Amsterdam Comparison
Human and LLM Complexity Evaluations
A descriptive comparison of published human assessments and blinded LLM assessments for 12 Amsterdam Execution Layer EIPs, evaluated under the same checklist revision so criteria and totals are directly comparable.
Complexity profiles per EIP
Human and LLM revision-1 profiles for each EIP on one shared scale, ranked by the Human total. Δ is LLM minus Human. Open an EIP for its per-criterion comparison with the rationale from both evaluators.
Where the Evaluators Differ by Criterion
Mean per-criterion difference across the 12 EIPs, LLM minus Human, under the shared revision-1 checklist. Positive values mean the LLM scored the criterion higher on average.
| Security risks | +1.50 | 1.83 | 9 | 2 | 1 |
|---|---|---|---|---|---|
| Cross-EIP interactions | +1.17 | 1.17 | 8 | 0 | 4 |
| Performance risks | +1.00 | 1.00 | 7 | 0 | 5 |
| EVM Gas rule changes | −0.92 | 1.25 | 2 | 5 | 5 |
| Encoding changes (RLP/SSZ) | +0.50 | 0.50 | 2 | 0 | 10 |
| Edge/boundary conditions | +0.50 | 0.67 | 5 | 1 | 6 |
| New or modified transaction validity mechanisms | −0.42 | 0.58 | 1 | 4 | 7 |
| New fork activation mechanism | +0.42 | 0.42 | 2 | 0 | 10 |
| Patterns affecting pre-existing tests | −0.33 | 1.17 | 5 | 4 | 3 |
| Modified system contracts | −0.25 | 0.25 | 0 | 2 | 10 |
| New EVM gas refund | −0.25 | 0.25 | 0 | 2 | 10 |
| Engine API changes | −0.25 | 0.25 | 0 | 1 | 11 |
| Modified opcodes | +0.17 | 0.67 | 2 | 1 | 9 |
| Block syncing changes | +0.17 | 0.17 | 2 | 0 | 10 |
| Added system contracts | −0.08 | 0.08 | 0 | 1 | 11 |
| Blob gas accounting changes | −0.08 | 0.08 | 0 | 1 | 11 |
| Engine API encoding changes | +0.08 | 0.08 | 1 | 0 | 11 |
| Transition-tool interface changes | −0.08 | 0.42 | 1 | 2 | 9 |
| Added opcodes | 0 | 0.00 | 0 | 0 | 12 |
| Added precompiles | 0 | 0.00 | 0 | 0 | 12 |
| Modified precompiles | 0 | 0.00 | 0 | 0 | 12 |
| New transaction types | 0 | 0.00 | 0 | 0 | 12 |
| New block / header fields | 0 | 0.00 | 0 | 0 | 12 |
| Cryptography | 0 | 0.00 | 0 | 0 | 12 |
Totals per EIP
| EIP-7928 | Block-Level Access Lists | 29 | 26 | −3 | High | 40 | No |
| EIP-8037 | State Creation Gas Cost Increase | 28 | 21 | −7 | High | 35 | No |
| EIP-8038 | State-access gas cost update | 20 | 17 | −3 | High vs Medium | 17 | No |
| EIP-2780 | Resource-based intrinsic transaction gas | 13 | 20 | +7 | Medium vs High | 25 | No |
| EIP-7778 | Block Gas Accounting without Refunds | 10 | 8 | −2 | Medium vs Low | 13 | No |
| EIP-7708 | ETH transfers emit a log | 9 | 13 | +4 | Low vs Medium | 19 | Yes |
| EIP-7610 | Revert creation in case of non-empty storage | 7 | 10 | +3 | Low vs Medium | 9 | No |
| EIP-7843 | SLOTNUM opcode | 7 | 17 | +10 | Low vs Medium | 23 | Yes |
| EIP-7981 | Increase Access List Cost | 6 | 11 | +5 | Low vs Medium | 11 | No |
| EIP-8024 | Backward compatible SWAPN, DUPN, EXCHANGE | 6 | 11 | +5 | Low vs Medium | 13 | No |
| EIP-7976 | Increase Calldata Floor Cost | 5 | 12 | +7 | Low vs Medium | 10 | No |
| EIP-7997 | Deterministic Factory Contract | 5 | 13 | +8 | Low vs Medium | 14 | No |
What Differed Systematically?
- Low human scores were pulled toward the middle. Under the shared v1 rubric, all seven EIPs rated Low by humans were rated Medium by the LLM, usually through a larger number of small nonzero scores.
- The LLM applied a “risk-review tax.” Much of the excess came from security review, cross-EIP dependencies, edge cases, and performance testing where human assessors more often required evidence of substantial additional work.
- High-end misses had a different shape. The LLM sometimes undercounted concrete gas-accounting, Engine API, validity, or additive per-mechanism burdens while still adding risk-oriented points elsewhere.
- V2 shifted scores upward. Its apparent improvement in tier agreement cannot be separated from the expanded rubric and changed thresholds.
The strongest supported use is as a structured second opinion: retain the LLM’s criterion-level flags and ranking signal, then have a human approve the evidence, aggregation, and final score.
Criterion legend for the revision-1 checklist
Every stacked bar, comparison matrix, and criterion table on this site uses the same criterion colours, abbreviations, and order. Colour marks the criterion group; the abbreviation and name identify the criterion. Scores are 0–3 per criterion (4 is exceptional; cross-EIP interactions is uncapped).
EVM surface
Opcodes, precompiles, and system contracts that are added or modified.
- Added opcodesIntroduces new opcodes
- Modified opcodesModifies pre-existing opcodes
- Added precompilesIntroduces new precompiles
- Modified precompilesModifies pre-existing precompiles logic or gas-accounting
- Added system contractsIntroduces new system contract, stateful or not
- Modified system contractsModifies pre-existing system contracts
Gas and accounting
Execution, blob, and state gas rules, refunds, and where charges happen inside opcodes.
- EVM Gas rule changesNew EVM gas accounting rules
- Blob gas accounting changesNew Blob gas accounting rules which potentially affect pre-existing tests
- New EVM gas refundNew gas-refund mechanism
Blocks, transactions, and encoding
Transaction types and validity, block and header fields, encodings, syncing, and activation-time changes.
- New transaction typesIntroduces a new transaction type
- New or modified transaction validity mechanismsCreates new or modifies pre-existing transaction types' validation mechanisms
- New block / header fieldsIntroduces new block or block header fields
- Encoding changes (RLP/SSZ)Introduces encoding changes at the transaction/block/interfaces level
- Block syncing changesModifies block RLP validation mechanisms that require test client syncing.
- New fork activation mechanismModifies state, internal variables, or similar, at the fork activation block
Client interfaces
Engine API and transition-tool interface changes.
- Engine API changesIntroduces new fields to the Engine API directives
- Engine API encoding changes · Checklist revision 1 onlyEngine API encoding changes (the revision-1 template defines no anchor text for this row).
- Transition-tool interface changesModifies or adds new fields to the transition tool interface.
Testing impact
Rework, new invariants, and new primitives required in the test framework.
- Patterns affecting pre-existing testsImplements a new validation mechanism or rule that translates in reworking pre-existing tests
Risk and validation
Security, performance, boundary conditions, and cryptography that need validation.
- Security risksIntroduces or modifies mechanisms that could compromise the security of the chain, users, validators, or other stakeholders, if not implemented properly.
- Performance risksIntroduces or modifies mechanisms and requires performance validation.
- Edge/boundary conditionsFeature contains edge/boundary conditions.
- CryptographyIntroduces new cryptography mechanisms or modifies existing functionality that involves cryptography
Coordination
Cross-EIP interactions and behavior that clients must agree on before tests exist.
- Cross-EIP interactionsIntroduces or modifies mechanisms that affect other EIPs in either the same or past forks.