Retrospective LLM-Based Complexity Evaluations

Primary retrospective result

Does Predicted Complexity Track Time to Mainnet?

At fork level, the study compares the time from development underway in earnest—the first devnet with at least two independent execution-layer clients—to mainnet against three summaries of predicted complexity.

Five forks, descriptive only. Amsterdam uses the projected 15 December 2026 mainnet date and is potentially in-sample. These plots do not establish that complexity caused delivery time; one fork can materially change the apparent pattern.

Predicted Complexity by Fork

The primary prediction is the score sum for EIPs included by each fork’s evaluation cutoff. EIPs added afterward could not have informed that initial prediction, so the stacked chart reports their complexity as a separate increment. Both subtotals reflect the number and splitting of EIPs.

Shanghai4 by cutoff · 1 added later
Cancun5 by cutoff · 1 added later
Prague8 by cutoff · 3 added later
Osaka8 by cutoff · 4 added later
Amsterdam13 by cutoff · 2 added later
Values: score at cutoff, added-later score, and final-scope score per fork
Predicted Execution Layer complexity at and after each fork’s evaluation cutoff
ForkEIPs by cutoffScore at cutoffEIPs added laterAdded-later scoreFinal-scope score
Shanghai / Shapella4651368
Cancun / Dencun512619135
Prague / Pectra8216338254
Osaka / Fusaka8105435140
Amsterdam / Glamsterdam13243244287

The at-cutoff subtotal is used in the shipping-time comparison; later additions remain visible as a separate segment. Hegotá is excluded because it is a forward-looking evaluation. Hover or focus a segment for its EIP count and share.

Which Kinds of Complexity Made Each Fork Heavy?

Each bar stacks the criterion scores of the EIPs included by the fork’s evaluation cutoff, so a segment is that criterion’s total contribution to the fork’s predicted complexity. Absolute mode keeps the shared scale used elsewhere on this page; composition mode normalizes each fork to 100% so the mix can be compared independently of size. Colours, names, and order match every per-EIP bar on the site.

View
Shanghai65 points · 4 EIPs
Cancun126 points · 5 EIPs
Prague216 points · 8 EIPs
Osaka105 points · 8 EIPs
Amsterdam243 points · 13 EIPs
Composition table: points per criterion and fork
Criterion points included by each fork’s evaluation cutoff
CriterionShanghaiCancunPragueOsakaAmsterdam
Added opcodes1 · 2% · 1 EIP6 · 5% · 4 EIPs004 · 2% · 2 EIPs
Modified opcodes3 · 5% · 1 EIP3 · 2% · 1 EIP3 · 1% · 1 EIP09 · 4% · 3 EIPs
Added precompiles01 · 1% · 1 EIP5 · 2% · 2 EIPs1 · 1% · 1 EIP0
Modified precompiles0003 · 3% · 2 EIPs0
Added system contracts02 · 2% · 1 EIP4 · 2% · 2 EIPs02 · 1% · 1 EIP
Modified system contracts002 · 1% · 1 EIP02 · 1% · 1 EIP
EVM Gas rule changes5 · 8% · 3 EIPs6 · 5% · 4 EIPs9 · 4% · 4 EIPs3 · 3% · 3 EIPs13 · 5% · 8 EIPs
State-access ordering within opcode execution2 · 3% · 1 EIP2 · 2% · 1 EIP3 · 1% · 2 EIPs02 · 1% · 1 EIP
Blob gas accounting changes03 · 2% · 1 EIP02 · 2% · 2 EIPs0
State gas accounting changes00003 · 1% · 1 EIP
New transaction types03 · 2% · 1 EIP3 · 1% · 1 EIP00
New or modified transaction validity mechanisms2 · 3% · 1 EIP3 · 2% · 1 EIP5 · 2% · 2 EIPs2 · 2% · 1 EIP11 · 5% · 5 EIPs
New block / header fields3 · 5% · 1 EIP6 · 5% · 2 EIPs9 · 4% · 3 EIPs06 · 2% · 2 EIPs
Encoding changes (RLP/SSZ)3 · 5% · 1 EIP6 · 5% · 2 EIPs15 · 7% · 5 EIPs6 · 6% · 2 EIPs6 · 2% · 2 EIPs
Block syncing changes2 · 3% · 1 EIP3 · 2% · 2 EIPs9 · 4% · 3 EIPs1 · 1% · 1 EIP4 · 2% · 2 EIPs
New fork activation mechanism03 · 2% · 1 EIP6 · 3% · 2 EIPs3 · 3% · 1 EIP3 · 1% · 1 EIP
Engine API changes1 · 2% · 1 EIP2 · 2% · 2 EIPs4 · 2% · 3 EIPs03 · 1% · 2 EIPs
Transition-tool interface changes1 · 2% · 1 EIP4 · 3% · 2 EIPs10 · 5% · 4 EIPs07 · 3% · 4 EIPs
Patterns affecting pre-existing tests7 · 11% · 4 EIPs9 · 7% · 5 EIPs16 · 7% · 7 EIPs11 · 10% · 8 EIPs27 · 11% · 13 EIPs
New invariant on pre-existing tests3 · 5% · 1 EIP4 · 3% · 2 EIPs9 · 4% · 4 EIPs1 · 1% · 1 EIP10 · 4% · 5 EIPs
New test-framework primitives2 · 3% · 1 EIP3 · 2% · 2 EIPs6 · 3% · 4 EIPs2 · 2% · 1 EIP6 · 2% · 3 EIPs
Security risks7 · 11% · 4 EIPs11 · 9% · 5 EIPs20 · 9% · 8 EIPs13 · 12% · 7 EIPs26 · 11% · 12 EIPs
Performance risks3 · 5% · 2 EIPs10 · 8% · 5 EIPs19 · 9% · 8 EIPs13 · 12% · 7 EIPs23 · 9% · 11 EIPs
Edge/boundary conditions8 · 12% · 4 EIPs15 · 12% · 5 EIPs22 · 10% · 8 EIPs17 · 16% · 8 EIPs28 · 12% · 13 EIPs
Cryptography04 · 3% · 2 EIPs3 · 1% · 2 EIPs3 · 3% · 2 EIPs1 · 0% · 1 EIP
Cross-EIP interactions7 · 11% · 4 EIPs8 · 6% · 4 EIPs14 · 6% · 7 EIPs12 · 11% · 7 EIPs28 · 12% · 11 EIPs
Unspecified behavior requiring cross-client consensus5 · 8% · 2 EIPs9 · 7% · 4 EIPs20 · 9% · 8 EIPs12 · 11% · 7 EIPs19 · 8% · 9 EIPs
Total65126216105243
Criterion legend for the fork composition bars

Every stacked bar, comparison matrix, and criterion table on this site uses the same criterion colours, abbreviations, and order. Colour marks the criterion group; the abbreviation and name identify the criterion. Scores are 0–3 per criterion (4 is exceptional; cross-EIP interactions is uncapped).

EVM surface

Opcodes, precompiles, and system contracts that are added or modified.

  • Added opcodes
    Introduces new opcodes
  • Modified opcodes
    Modifies pre-existing opcodes
  • Added precompiles
    Introduces new precompiles
  • Modified precompiles
    Modifies pre-existing precompiles logic or gas-accounting
  • Added system contracts
    Introduces new system contract, stateful or not
  • Modified system contracts
    Modifies pre-existing system contracts

Gas and accounting

Execution, blob, and state gas rules, refunds, and where charges happen inside opcodes.

  • EVM Gas rule changes
    New EVM gas accounting rules
  • State-access ordering within opcode execution · Checklist revision 2 only
    Changes *where inside an opcode's execution* state is accessed, or where gas is charged relative to that access. Because a state access is recorded in the block-level access list only if execution had enough gas to reach it, this ordering is consensus-critical: moving it changes the BAL at every gas boundary of every affected opcode.
  • Blob gas accounting changes
    New Blob gas accounting rules which potentially affect pre-existing tests
  • State gas accounting changes · Checklist revision 2 only
    New state gas accounting rules. State gas is the cost of *writing* state, as opposed to accessing or executing it: `StateGasCosts`, `COST_PER_STATE_BYTE`, the block-level state gas budget, and the spill path into execution gas.
  • New EVM gas refund
    New gas-refund mechanism

Blocks, transactions, and encoding

Transaction types and validity, block and header fields, encodings, syncing, and activation-time changes.

  • New transaction types
    Introduces a new transaction type
  • New or modified transaction validity mechanisms
    Creates new or modifies pre-existing transaction types' validation mechanisms
  • New block / header fields
    Introduces new block or block header fields
  • Encoding changes (RLP/SSZ)
    Introduces encoding changes at the transaction/block/interfaces level
  • Block syncing changes
    Modifies block RLP validation mechanisms that require test client syncing.
  • New fork activation mechanism
    Modifies state, internal variables, or similar, at the fork activation block

Client interfaces

Engine API and transition-tool interface changes.

  • Engine API changes
    Introduces new fields to the Engine API directives
  • Transition-tool interface changes
    Modifies or adds new fields to the transition tool interface.

Testing impact

Rework, new invariants, and new primitives required in the test framework.

  • Patterns affecting pre-existing tests
    Implements a new validation mechanism or rule that translates in reworking pre-existing tests
  • New invariant on pre-existing tests · Checklist revision 2 only
    Tests that are **not about this EIP** must nonetheless assert something this EIP produces. Their logic does not change; they gain a new thing to check.
  • New test-framework primitives · Checklist revision 2 only
    Requires new abstractions in the test framework itself — expectation types, modifiers, helpers — beyond writing test functions with what already exists.

Risk and validation

Security, performance, boundary conditions, and cryptography that need validation.

  • Security risks
    Introduces or modifies mechanisms that could compromise the security of the chain, users, validators, or other stakeholders, if not implemented properly.
  • Performance risks
    Introduces or modifies mechanisms and requires performance validation.
  • Edge/boundary conditions
    Feature contains edge/boundary conditions.
  • Cryptography
    Introduces new cryptography mechanisms or modifies existing functionality that involves cryptography

Coordination

Cross-EIP interactions and behavior that clients must agree on before tests exist.

  • Cross-EIP interactions
    Introduces or modifies mechanisms that affect other EIPs in either the same or past forks.
  • Unspecified behavior requiring cross-client consensus · Checklist revision 2 only
    The EIP text does not determine the answer for cases a test can construct. Clients must agree on a previously unspecified detail before tests can be baselined. The cost here is coordination and re-baselining, not test writing.

Fork Shipping Time Versus Predicted Complexity

Fork shipping time compared with the sum of Execution Layer complexity scores that could have been predicted at each fork’s evaluation cutoff.
Fork shipping time compared with the at-cutoff sum of scores from EIPs assessed as High complexity.
Fork shipping time compared with the highest-scoring Execution Layer EIP present by each fork’s evaluation cutoff.
Fork-level values plotted above
Shanghai / Shapellawithdrawals-devnet-0
2022-12-09		2023-04-121246553EIP-4895: 303
Cancun / Dencundencun-devnet-4
2023-01-23		2024-03-1341512686EIP-4844: 489
Prague / Pectrapectra-devnet-0
2024-05-17		2025-05-07355216165EIP-7002: 3838
Osaka / Fusakafusaka-devnet-0
2025-05-26		2025-12-0319110524EIP-7594: 2435
Amsterdam / Glamsterdam (projected)bal-devnet-0
2025-11-04		2026-12-15405243123EIP-7928: 4044

Cancun is the only fork whose ≥2-EL start differs from its first EL devnet: devnets 1–3 ran a single patched geth fork, so this clock begins at dencun-devnet-4. Cancun also largely distinguishes the total-load and critical-path readings in this five-fork sample.

EIP-level observed proxies

As a separate view, each retrospective fork–EIP relationship is compared with two score-blind counts derived from public artifacts:

Amsterdam is omitted from these plots because it had not reached mainnet when the data was frozen, so its counts were still incomplete.

Specification rework across 34 fork–EIP relationships in the four shipped forks.
New EIP interactions across 34 fork–EIP relationships in the four shipped forks.
Descriptive rank associations across shipped forks
MetricNPooled Spearman ρ
Specification rework340.75
New EIP interactions340.56
Browse all EIP assessmentsCompare human and LLM evaluations