Retrospective LLM-Based Complexity Evaluations

Retrospective Fork Evaluation

Amsterdam / Glamsterdam

13 Execution Layer EIPs were included by the evaluation cutoff, with a predicted complexity score sum of 243. Another 2 EIPs were added later, contributing 44 to the final-scope total of 287.

Right-censored and potentially in-sample. Amsterdam’s observation window ends before its final outcome.
Scope timing: The primary retrospective prediction includes only EIPs present by the evaluation cutoff. The later additions remain assessed and reported separately: EIP-8246 (+12), EIP-8282 (+32).
EIPs by cutoff
13
score 243
Added after cutoff
2
+44 · final scope 287
Under-specified at cutoff
10 / 15
Human checklists
12 of 15
  • Complete 12
  • Not available 3
12 same-rubric Human/LLM comparisons

Where the Complexity Comes From

Each segment sums one criterion across the fork's scored LLM assessments. Hover or focus a segment for its points, share, and the number of contributing EIPs.

View
By cutoff243 points · 13 EIPs
Added later44 points · 2 EIPs
Composition table: points per criterion and fork
Amsterdam / Glamsterdam: criterion points included by the cutoff and added afterward
CriterionBy cutoffAdded later
Added opcodes4 · 2% · 2 EIPs0
Modified opcodes9 · 4% · 3 EIPs3 · 7% · 1 EIP
Added system contracts2 · 1% · 1 EIP3 · 7% · 1 EIP
Modified system contracts2 · 1% · 1 EIP2 · 5% · 1 EIP
EVM Gas rule changes13 · 5% · 8 EIPs1 · 2% · 1 EIP
State-access ordering within opcode execution2 · 1% · 1 EIP0
State gas accounting changes3 · 1% · 1 EIP0
New or modified transaction validity mechanisms11 · 5% · 5 EIPs0
New block / header fields6 · 2% · 2 EIPs0
Encoding changes (RLP/SSZ)6 · 2% · 2 EIPs3 · 7% · 1 EIP
Block syncing changes4 · 2% · 2 EIPs0
New fork activation mechanism3 · 1% · 1 EIP3 · 7% · 1 EIP
Engine API changes3 · 1% · 2 EIPs0
Transition-tool interface changes7 · 3% · 4 EIPs0
Patterns affecting pre-existing tests27 · 11% · 13 EIPs4 · 9% · 2 EIPs
New invariant on pre-existing tests10 · 4% · 5 EIPs1 · 2% · 1 EIP
New test-framework primitives6 · 2% · 3 EIPs1 · 2% · 1 EIP
Security risks26 · 11% · 12 EIPs5 · 11% · 2 EIPs
Performance risks23 · 9% · 11 EIPs3 · 7% · 1 EIP
Edge/boundary conditions28 · 12% · 13 EIPs6 · 14% · 2 EIPs
Cryptography1 · 0% · 1 EIP1 · 2% · 1 EIP
Cross-EIP interactions28 · 12% · 11 EIPs6 · 14% · 2 EIPs
Unspecified behavior requiring cross-client consensus19 · 8% · 9 EIPs2 · 5% · 1 EIP
Total24344

Complexity Assessments

LLM rows were automated using the GPT-5.6 Sol LLM at xhigh reasoning effort, evaluating the EIP revision linked in the Assessment Input column against version 2 of STEEL’s complexity-assignment template. Human rows are STEEL reviewers’ published checklists; their status badge links to the pull request or merged report. Select rows from either evaluator to compare them, or open an EIP for its full assessment.

Evaluator
27 assessments: 15 LLM, 12 Human
Amsterdam / Glamsterdam complexity assessments, one row per evaluator
Select for comparisonComplexity profile
EIP-2780 Resource-based intrinsic transaction gasIncluded by cutoffLLMChecklist v2Complete25HighYes
EIP-2780 Resource-based intrinsic transaction gasIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete20HighYes
EIP-2780 Resource-based intrinsic transaction gasIncluded by cutoffNot recordedHumanChecklist v1Complete13Medium
EIP-7610 Revert creation in case of non-empty storageIncluded by cutoffLLMChecklist v2Complete9LowNo
EIP-7610 Revert creation in case of non-empty storageIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete10MediumYes
EIP-7610 Revert creation in case of non-empty storageIncluded by cutoffNot recordedHumanChecklist v1Complete7Low
EIP-7708 ETH transfers emit a logIncluded by cutoffLLMChecklist v2Complete19MediumYes
EIP-7708 ETH transfers emit a logIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete13MediumYes
EIP-7708 ETH transfers emit a logIncluded by cutoffNot recordedHumanChecklist v1Complete9Low
EIP-7778 Block Gas Accounting without RefundsIncluded by cutoffLLMChecklist v2Complete13MediumYes
EIP-7778 Block Gas Accounting without RefundsIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete8LowYes
EIP-7778 Block Gas Accounting without RefundsIncluded by cutoffNot recordedHumanChecklist v1Complete10Medium
EIP-7843 SLOTNUM opcodeIncluded by cutoffLLMChecklist v2Complete23HighYes
EIP-7843 SLOTNUM opcodeIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete17MediumYes
EIP-7843 SLOTNUM opcodeIncluded by cutoffNot recordedHumanChecklist v1Complete7Low
EIP-7928 Block-Level Access ListsIncluded by cutoffLLMChecklist v2Complete40HighYes
EIP-7928 Block-Level Access ListsIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete26HighYes
EIP-7928 Block-Level Access ListsIncluded by cutoffNot recordedHumanChecklist v1Complete29High
EIP-7954 Increase Maximum Contract SizeIncluded by cutoffLLMChecklist v2Complete14MediumNo
EIP-7976 Increase Calldata Floor CostIncluded by cutoffLLMChecklist v2Complete10LowYes
EIP-7976 Increase Calldata Floor CostIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete12MediumYes
EIP-7976 Increase Calldata Floor CostIncluded by cutoffNot recordedHumanChecklist v1Complete5Low
EIP-7981 Increase Access List CostIncluded by cutoffLLMChecklist v2Complete11LowNo
EIP-7981 Increase Access List CostIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete11MediumYes
EIP-7981 Increase Access List CostIncluded by cutoffNot recordedHumanChecklist v1Complete6Low
EIP-7997 Deterministic Factory ContractIncluded by cutoffLLMChecklist v2Complete14MediumNo
EIP-7997 Deterministic Factory ContractIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete13MediumYes
EIP-7997 Deterministic Factory ContractIncluded by cutoffNot recordedHumanChecklist v1Complete5Low
EIP-8024 Backward compatible SWAPN, DUPN, EXCHANGEIncluded by cutoffLLMChecklist v2Complete13MediumYes
EIP-8024 Backward compatible SWAPN, DUPN, EXCHANGEIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete11MediumYes
EIP-8024 Backward compatible SWAPN, DUPN, EXCHANGEIncluded by cutoffNot recordedHumanChecklist v1Complete6Low
EIP-8037 State Creation Gas Cost IncreaseIncluded by cutoffLLMChecklist v2Complete35HighYes
EIP-8037 State Creation Gas Cost IncreaseIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete21HighYes
EIP-8037 State Creation Gas Cost IncreaseIncluded by cutoffNot recordedHumanChecklist v1Complete28High
EIP-8038 State-access gas cost updateIncluded by cutoffLLMChecklist v2Complete17MediumYes
EIP-8038 State-access gas cost updateIncluded by cutoffLLMChecklist v1
re-run for the human comparison
Complete17MediumYes
EIP-8038 State-access gas cost updateIncluded by cutoffNot recordedHumanChecklist v1Complete20High
EIP-8246 Remove SELFDESTRUCT BurnAdded after cutoffLLMChecklist v2Complete12MediumNo
EIP-8282 Builder Execution RequestsAdded after cutoffLLMChecklist v2Complete32HighYes

How the Historical Refs Were Selected

These timelines support the assessment table by showing the fork chronology and the revision and inclusion history used to choose each input. They are process evidence, not additional complexity results. Each chart has its own time axis; horizontal positions should only be compared within a chart.

Fork Milestones and Relevant Devnets

Fork-level scope, devnet, testnet, and activation milestones on an independent time axis.

EIP Revision Histories and Selected Assessment Refs

The main purpose of this panel is to show where the selected assessment ref (REF) falls in each EIP’s revision history. The other traces and markers provide context for that selection. Every EIP row uses the same date scale, shown once above the panel.

Aligned EIP revision histories, inclusion-state transitions, selected assessment refs, and the fork evaluation cutoff on one shared time axis.
Criterion legend and glossary

Every stacked bar, comparison matrix, and criterion table on this site uses the same criterion colours, abbreviations, and order. Colour marks the criterion group; the abbreviation and name identify the criterion. Scores are 0–3 per criterion (4 is exceptional; cross-EIP interactions is uncapped).

EVM surface

Opcodes, precompiles, and system contracts that are added or modified.

  • Added opcodes
    Introduces new opcodes
  • Modified opcodes
    Modifies pre-existing opcodes
  • Added precompiles
    Introduces new precompiles
  • Modified precompiles
    Modifies pre-existing precompiles logic or gas-accounting
  • Added system contracts
    Introduces new system contract, stateful or not
  • Modified system contracts
    Modifies pre-existing system contracts

Gas and accounting

Execution, blob, and state gas rules, refunds, and where charges happen inside opcodes.

  • EVM Gas rule changes
    New EVM gas accounting rules
  • State-access ordering within opcode execution · Checklist revision 2 only
    Changes *where inside an opcode's execution* state is accessed, or where gas is charged relative to that access. Because a state access is recorded in the block-level access list only if execution had enough gas to reach it, this ordering is consensus-critical: moving it changes the BAL at every gas boundary of every affected opcode.
  • Blob gas accounting changes
    New Blob gas accounting rules which potentially affect pre-existing tests
  • State gas accounting changes · Checklist revision 2 only
    New state gas accounting rules. State gas is the cost of *writing* state, as opposed to accessing or executing it: `StateGasCosts`, `COST_PER_STATE_BYTE`, the block-level state gas budget, and the spill path into execution gas.
  • New EVM gas refund
    New gas-refund mechanism

Blocks, transactions, and encoding

Transaction types and validity, block and header fields, encodings, syncing, and activation-time changes.

  • New transaction types
    Introduces a new transaction type
  • New or modified transaction validity mechanisms
    Creates new or modifies pre-existing transaction types' validation mechanisms
  • New block / header fields
    Introduces new block or block header fields
  • Encoding changes (RLP/SSZ)
    Introduces encoding changes at the transaction/block/interfaces level
  • Block syncing changes
    Modifies block RLP validation mechanisms that require test client syncing.
  • New fork activation mechanism
    Modifies state, internal variables, or similar, at the fork activation block

Client interfaces

Engine API and transition-tool interface changes.

  • Engine API changes
    Introduces new fields to the Engine API directives
  • Engine API encoding changes · Checklist revision 1 only
    Engine API encoding changes (the revision-1 template defines no anchor text for this row).
  • Transition-tool interface changes
    Modifies or adds new fields to the transition tool interface.

Testing impact

Rework, new invariants, and new primitives required in the test framework.

  • Patterns affecting pre-existing tests
    Implements a new validation mechanism or rule that translates in reworking pre-existing tests
  • New invariant on pre-existing tests · Checklist revision 2 only
    Tests that are **not about this EIP** must nonetheless assert something this EIP produces. Their logic does not change; they gain a new thing to check.
  • New test-framework primitives · Checklist revision 2 only
    Requires new abstractions in the test framework itself — expectation types, modifiers, helpers — beyond writing test functions with what already exists.

Risk and validation

Security, performance, boundary conditions, and cryptography that need validation.

  • Security risks
    Introduces or modifies mechanisms that could compromise the security of the chain, users, validators, or other stakeholders, if not implemented properly.
  • Performance risks
    Introduces or modifies mechanisms and requires performance validation.
  • Edge/boundary conditions
    Feature contains edge/boundary conditions.
  • Cryptography
    Introduces new cryptography mechanisms or modifies existing functionality that involves cryptography

Coordination

Cross-EIP interactions and behavior that clients must agree on before tests exist.

  • Cross-EIP interactions
    Introduces or modifies mechanisms that affect other EIPs in either the same or past forks.
  • Unspecified behavior requiring cross-client consensus · Checklist revision 2 only
    The EIP text does not determine the answer for cases a test can construct. Clients must agree on a previously unspecified detail before tests can be baselined. The cost here is coordination and re-baselining, not test writing.