Evaluating if Conflux needs to support execution-spec-tests

[darwintree]

Many Ethereum clients rely on https://github.com/ethereum/execution-spec-tests to check if their EVM implementation is correct. It would be useful if Conflux eSpace can run the tests.

How does the spec-tests work

Generating Fixtures

The spec-tests library contains a lot of test cases. But the repo doesn't directly assume the execution result within tests. After running tests in spec-tests repo, a lot of json files called "fixtures" would be generated. Each json contains many object, which may contains:

• blockchain pre state
• blocks after the pre state
• blockchain post state

For example,

{
    "tests/berlin/eip2930_access_list/test_acl.py::test_access_list[fork_Berlin-blockchain_test]": {
        "network": "Berlin",
        "genesisBlockHeader": {
            "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "uncleHash": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
            "coinbase": "0x0000000000000000000000000000000000000000",
            "stateRoot": "0xdcddfa4aea2ba8d951face17b28ca6e2daff3f94996ea0c6d606b168be11e1e3",
            "transactionsTrie": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "receiptTrie": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
            "difficulty": "0x020000",
            "number": "0x00",
            "gasLimit": "0x016345785d8a0000",
            "gasUsed": "0x00",
            "timestamp": "0x00",
            "extraData": "0x00",
            "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "nonce": "0x0000000000000000",
            "hash": "0xc02537631331c93afab70bc90cf4c89353fb32d39925747e3fd9b8d1dd463a1f"
        },
        "pre": {
            "0x0000000000000000000000000000000000001000": {
                "nonce": "0x01",
                "balance": "0x03",
                "code": "0x5854505854",
                "storage": {}
            },
            "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b": {
                "nonce": "0x00",
                "balance": "0x300000",
                "code": "0x",
                "storage": {}
            }
        },
        "postState": {
            "0x0000000000000000000000000000000000001000": {
                "nonce": "0x01",
                "balance": "0x04",
                "code": "0x5854505854",
                "storage": {}
            },
            "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba": {
                "nonce": "0x00",
                "balance": "0x1bc16d674ecb26ce",
                "code": "0x",
                "storage": {}
            },
            "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b": {
                "nonce": "0x01",
                "balance": "0x2cd931",
                "code": "0x",
                "storage": {}
            }
        },
        "lastblockhash": "0x9700f628797ac43e4d80c316d84d207f7c61eef134ae71cd60eb6e7e5914eb09",
        "config": {
            "network": "Berlin",
            "chainid": "0x01"
        },
        "genesisRLP": "0xf901fdf901f8a00000000000000000000000000000000000000000000000000000000000000000a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347940000000000000000000000000000000000000000a0dcddfa4aea2ba8d951face17b28ca6e2daff3f94996ea0c6d606b168be11e1e3a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000830200008088016345785d8a0000808000a00000000000000000000000000000000000000000000000000000000000000000880000000000000000c0c0",
        "blocks": [
            {
                "blockHeader": {
                    "parentHash": "0xc02537631331c93afab70bc90cf4c89353fb32d39925747e3fd9b8d1dd463a1f",
                    "uncleHash": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
                    "coinbase": "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
                    "stateRoot": "0x2bda9179e1c1acef5708726a7b97c95278d569a6a2fc123dd8b54db97c41908c",
                    "transactionsTrie": "0x2182b0a8aaa5484a3fdc67d43b96180cdbd09465ec8db206db63c9b071b33f0f",
                    "receiptTrie": "0xc166d1f62ef90bdef78d79a4b28579b279621fbcd156aba3060f242d5feeda67",
                    "bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
                    "difficulty": "0x020000",
                    "number": "0x01",
                    "gasLimit": "0x016345785d8a0000",
                    "gasUsed": "0x7342",
                    "timestamp": "0x03e8",
                    "extraData": "0x00",
                    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
                    "nonce": "0x0000000000000000",
                    "hash": "0x9700f628797ac43e4d80c316d84d207f7c61eef134ae71cd60eb6e7e5914eb09"
                },
                "transactions": [
                    {
                        "type": "0x01",
                        "chainId": "0x01",
                        "nonce": "0x00",
                        "gasPrice": "0x07",
                        "gasLimit": "0x04ef00",
                        "to": "0x0000000000000000000000000000000000001000",
                        "value": "0x01",
                        "data": "0x",
                        "accessList": [
                            {
                                "address": "0x0000000000000000000000000000000000000000",
                                "storageKeys": [
                                    "0x0000000000000000000000000000000000000000000000000000000000000000"
                                ]
                            }
                        ],
                        "v": "0x00",
                        "r": "0x81a25ffdb3797e6428f854c642e1884ee7b7be0c4ccbb7e989b70039b87e4450",
                        "s": "0x2d25e40f45271d5f77735d06550dff43bf328190de034157cefec446855513d6",
                        "sender": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b"
                    }
                ],
                "uncleHeaders": [],
                "rlp": "0xf902a2f901fca0c02537631331c93afab70bc90cf4c89353fb32d39925747e3fd9b8d1dd463a1fa01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347942adc25665018aa1fe0e6bc666dac8fc2697ff9baa02bda9179e1c1acef5708726a7b97c95278d569a6a2fc123dd8b54db97c41908ca02182b0a8aaa5484a3fdc67d43b96180cdbd09465ec8db206db63c9b071b33f0fa0c166d1f62ef90bdef78d79a4b28579b279621fbcd156aba3060f242d5feeda67b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000830200000188016345785d8a00008273428203e800a00000000000000000000000000000000000000000000000000000000000000000880000000000000000f8a0b89e01f89b0180078304ef009400000000000000000000000000000000000010000180f838f7940000000000000000000000000000000000000000e1a0000000000000000000000000000000000000000000000000000000000000000080a081a25ffdb3797e6428f854c642e1884ee7b7be0c4ccbb7e989b70039b87e4450a02d25e40f45271d5f77735d06550dff43bf328190de034157cefec446855513d6c0",
                "blocknumber": "1"
            }
        ],
        "sealEngine": "NoProof",
        "_info": {
            "hash": "0xccc59b7bf0e30ef10c257342f5f8a3d054ae314db6ff564463784a1e66bffca1",
            "comment": "`execution-spec-tests` generated test",
            "filling-transition-tool": "evm version 1.13.5-stable",
            "description": "Test function documentation:\nTest type 1 transaction.",
            "url": "https://github.com/ethereum/execution-spec-tests/tree/v4.0.0/tests/berlin/eip2930_access_list/test_acl.py#L19",
            "fixture_format": "blockchain_test",
            "reference-spec": "https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2930.md",
            "reference-spec-version": "c9db53a936c5c9cbe2db32ba0d1b86c4c6e73534"
        }
    },
    ...
}

The default implementation comes from https://github.com/ethereum/execution-specs or https://github.com/ethereum/execution-spec-tests/tree/main?tab=readme-ov-file#transition-tool-support

Consume Fixtures

The actual clients, for example geth, will "consume" the generated fixtures: load the pre state, play the blocks and check if their post state matches the claimed post state. In this way the clients would know if their implementation would match the expected behaviour.

The tools can be found here: https://ethereum.github.io/execution-spec-tests/main/consuming_tests/

What is Needed for Conflux to Consume the Tests

• Conflux's implementation slightly differs from standard EVM, including state root conputation, opcode cost, and something else. So corresponding changes is required to directly use the project by either or combining
  • fork the standard implementation(https://github.com/ethereum/execution-specs) and modify corresponding parameters before generating fixtures
  • disable or modify certain tests
  • implement or fork corresponding Ethereum test consumption tool
• Conflux's execution engine/RPC interface yet does not support the basic features needed to consume the tests, including
  • load a specified state
  • play blocks