JSON RPC Error: failure to recognize C-Chain commands found under the debug_* namespace "the method debug_traceTransaction does not exist/is not available"

[keennay]

Describe the bug
Hello. A bootstrapped, fully synced archive node of AvalancheGo fails to recognize C-Chain commands found under the debug_* namespace, despite "debug-tracer" enabled within the "eth-apis" section of the provided config.json file. This problem exists for both fully synced Mainnet and Fuji nodes, across versions v1.7.5 (source) and v1.7.7 (binary) of AvalancheGo, tested on both CentOS 7 and Rocky Linux 8.4 machines. It was confirmed AvalancheGo was able to read the config.json file since debug logging was viewable from the avalanchego console.

Configuration of the C-Chain config file was followed from this document:
https://docs.avax.network/build/references/avalanchego-config-flags/#debug-tracer

To Reproduce
Download the AvalancheGo 1.7.7 binary, download the config.json file from Gabriel Cardona found below (while also setting "pruning-enabled" to false), then run using the commands below the config.json link:
https://gist.github.com/cgcardona/9628ca63d18991922db52962b10ddc91

avalanchego/avalanchego --index-enabled=true --db-dir=/data/avax --http-host=127.0.0.1 --http-port=9650 --staking-port=9651 --api-ipcs-enabled=true --config-file=/data/avax/mainnet/config.json

Expected behavior
Running the below command should show a full trace for the Mainnet transaction hash: 0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c

curl -X POST --data '{ 
   "jsonrpc": "2.0", 
   "method": "debug_traceTransaction", 
   "params": ["0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c"], 
   "id": "0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c"
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/bc/C/rpc

Instead the below is received:
{"jsonrpc":"2.0","id":"0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c","error":{"code":-32601,"message":"the method debug_traceTransaction does not exist/is not available"}}

Operating System
CentOS 7
Rocky Linux 8.4

Additional context
The problem was initially found after building AvalancheGo v1.7.5 from source on a CentOS 7 machine, after it was fully bootstrapped. This procedure was then repeated on a Rocky Linux 8.4 machine using the binary of AvalancheGo v.1.7.7, and a fresh sync after that too was fully bootstrapped after some days, and the same error was seen.

I have been asking in the Avalanche Discord #developers-chat channel and working with Gabriel Cardona, jpop32, and others with no resolution:
https://discordapp.com/channels/578992315641626624/578992911991963648/954517681405583390

Another user has the same problem here:
https://discordapp.com/channels/578992315641626624/578992911991963648/954498144605986857

Any help would be greatly appreciated, thanks!

[jsvisa]

have you added those APIs into your config.json's eth-apis:

    "private-debug",
    "debug-tracer",

mine looks as below:

{
  "snowman-api-enabled": true,
  "coreth-admin-api-enabled": true,
  "coreth-admin-api-dir": "",
  "eth-apis": [
    "public-eth",
    "public-eth-filter",
    "private-admin",
    "public-debug",
    "private-debug",
    "debug-tracer",
    "net",
    "web3",
    "internal-public-eth",
    "internal-public-blockchain",
    "internal-public-tx-pool",
    "internal-public-transaction-pool",
    "internal-public-account",
    "internal-public-debug",
    "internal-private-debug",
    "internal-private-personal"
  ],
  "continuous-profiler-dir": "",
  "continuous-profiler-frequency": 900000000000,
  "continuous-profiler-max-files": 5,
  "rpc-gas-cap": 50000000,
  "rpc-tx-fee-cap": 100,
  "preimages-enabled": true,
  "pruning-enabled": false,
  "snapshot-async": true,
  "snapshot-verification-enabled": false,
  "metrics-enabled": true,
  "metrics-expensive-enabled": false,
  "local-txs-enabled": false,
  "api-max-duration": 0,
  "ws-cpu-refill-rate": 0,
  "ws-cpu-max-stored": 0,
  "api-max-blocks-per-request": 0,
  "allow-unfinalized-queries": false,
  "allow-unprotected-txs": false,
  "keystore-directory": "",
  "keystore-external-signer": "",
  "keystore-insecure-unlock-allowed": false,
  "remote-tx-gossip-only-enabled": false,
  "tx-regossip-frequency": 60000000000,
  "tx-regossip-max-size": 15,
  "log-level": "info",
  "offline-pruning-enabled": false,
  "offline-pruning-bloom-filter-size": 512,
  "offline-pruning-data-directory": ""
}

And the debug_traceXxx works fine.

[keennay]

Hey @jsvisa, I'm using your exact conf.json file for my node and seeing the same problem. Are you testing this against a local node or Mainnet / Fuji? Gabriel in Discord had reported the same: the debug_* calls worked on his local node, yet he didn't have a Mainnet or Fuji node to check.

$ curl -X POST --data '{ 
   "jsonrpc": "2.0", 
   "method": "debug_traceTransaction", 
   "params": ["0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c"], 
   "id": "0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c"
}' -H 'content-type:application/json;' 127.0.0.1:9650/ext/bc/C/rpc
{"jsonrpc":"2.0","id":"0x3ff2cbfb93f71816a76c352e063d3f6bd45c1904d8c64f70d2e7c4bda064551c","error":{"code":-32601,"message":"the method debug_traceTransaction does not exist/is not available"}}

[jsvisa]

I'm running with AvalancheGo 1.7.8 and in mainnet.

[keennay]

Thanks @jsvisa, may I ask which arguments you're using to run AvalancheGo, same as what I have? And is the config.json file placed in a special directory?

[jsvisa]

@ymonye

/usr/local/avalanche/bin/avalanchego \
    --chain-config-dir=/md0/avalanche/etc/chains \
    --db-dir=/md0/avalanche/db \
    --db-type=leveldb \
    --index-enabled \
    --api-admin-enabled=true --api-health-enabled=true --api-info-enabled=true --api-metrics-enabled=true \
    --http-allowed-origins='*' \
    --http-host=0.0.0.0 \
    --http-port=9650

[keennay]

Thanks so much! Looks like --chain-config-dir= returned the desired results for reading the eth-api data, versus --config-file=

Resolved

[khaihkd]

@jsvisa How can I call method debug_traceTransaction in a subnet?

I added

"private-debug",
 "debug-tracer",

To the C chain config and I can call this function in C chain, but it does not work on subnet. What I need todo? Thanks

[jsvisa]

I'm not sure, but the trace api maybe only available for c-chain