The query "eth/v1/validator/duties/attester" API is very slow

[zhy827827]

Hellooo! 😄

time curl --request POST --url http://127.0.0.1:3500/eth/v1/validator/duties/attester/250170 --header 'Content-Type: application/json' --data '["1002966", "978009", "1003031", "978012", "616694", "943304", "1002989", "657396", "632648", "657385", "968524", "1002990", "616683", "968505", "616688", "1002997", "616697", "866841", "977182", "1003012", "1002972", "943309", "729596", "1008686", "1003026", "1019623", "1019611", "657387", "1019612", "968506", "657390", "977206", "968526", "1002992", "978024", "1019601", "1019607", "1003034", "943313", "978025", "968523", "977189", "1003011", "1019628", "943317", "1019619", "1019604", "1003016", "943312", "1013619", "616685", "1002991", "1019600", "948171", "770998", "977193", "616696", "978023", "616699", "977181", "977208", "657397", "1002971", "1019617", "1023981", "1002994", "1003020", "833413", "943306", "968520", "977195", "1003039", "554491", "1003028", "616678", "1003002", "657399", "1003022", "1019621", "1019616", "977209", "978028", "1002995", "657389", "989531", "1002963", "977204", "968522", "1002974", "1019606", "877504", "977223", "805649", "616687", "1003004", "1019603", "1019618", "616689", "1002998", "977221", "978015", "1023980", "1002979", "1002985", "657403", "657404", "1002968", "943311", "1019609", "978029", "989530", "1002999", "943314", "977205", "1003005", "968511", "977198", "1003019", "943302", "1003000", "1003045", "968509", "1002981", "948176", "978016", "977207", "1019614", "1003021", "1002988", "616690", "1003001", "694856", "657405", "1019599", "968516", "1002982", "977212", "1003007", "968508", "616691", "805647", "977202", "978019", "616695", "1019626", "943316", "657395", "645639", "968514", "978021", "616698", "979424", "1003003", "968513", "978017", "977192", "1002975", "1019613", "977188", "1019602", "1003051", "657388", "1003038", "943319", "978027", "1002983", "977211", "843998", "657394", "616679", "657401", "977197", "977183", "1036391", "977186", "977214", "968527", "1003041", "977220", "1002970", "1003042", "989532", "616686", "1013618", "948174", "977203", "968521", "977196", "1002986", "978030", "1003013", "1003052", "1003009", "1002996", "694857", "1003046", "978014", "977194", "1003010", "978010", "943308", "978008", "1003050", "977201", "943305", "1002969", "805648", "977213", "978031", "968515", "1003027", "1003032", "977210", "977184", "977187", "1002961", "977179", "657398", "1003044", "1003029", "616692", "943315", "657391", "968517", "1002976", "657402", "977191", "1003043", "1003008", "1019620", "1019624", "1002964", "968525", "616681", "962205", "1003048", "1003023", "948173", "1002987", "968512", "866840", "968510", "1003015", "1002980", "1003047", "1003018", "1002973", "1019608", "805646", "1019598", "977199", "729595", "657386", "1002984", "943318", "977219", "1019625", "1008689", "978011", "1003040", "1003036", "657400", "977178", "978032", "1002978", "616682", "978020", "977180", "1003033", "657406", "657393", "1003049", "943320", "1019610", "977185", "1002977", "1008687", "977216", "1003035", "1003053", "943301", "1003030", "948172", "1019627", "948170", "978022", "1003014", "1008685", "1019622", "616684", "977218", "968518", "877506", "657392", "977215", "1013620", "1003037", "616693", "1008688", "1002962", "1019615", "1003017", "978013", "1002993", "943303", "977200", "1003025", "978026", "1002965", "616680", "977222", "978018", "1008684", "1002967", "567565", "705890", "943307", "948175", "977224", "977190", "1003024", "1019605", "979425", "968507", "943310", "977217", "968519", "1003006"]'

real	2m27.117s
user	0m0.054s
sys	0m0.056s

!!!!!Prysm took 137 seconds to retrieve the data, but Lighthouse only needs 1 second to retrieve it

Prysm version:v4.1.1

Start Prysm Command: /opt/prysm/dist/beacon-chain-v4.1.1-linux-amd64 --verbosity debug --execution-endpoint=http://127.0.0.1:8551 --datadir /opt/prysm/data --grpc-gateway-host 0.0.0.0 --p2p-max-peers 100 --min-sync-peers 5 --jwt-secret /opt/prysm/data/jwtsecret

#Is there any way to optimize?

[nisdas]

This sounds like a bug, cc @rkapka

[zhy827827]

time curl --request POST --url http://127.0.0.1:3500/eth/v1/validator/duties/attester/250170 --header 'Content-Type: application/json' --data-raw '["943310", "977217", "968519", "1003006"]'

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1221 100 1180 0 41 8 0 0:02:27 0:02:27 --:--:-- 308
{"dependent_root":"0x27dc287358e3eab436efc4b3601aaee0ebf6637c422939af3f8c30ce0b5080c9","execution_optimistic":false,"data":[{"pubkey":"0xb2d342804a006b372f2d5885dabb38c3a3c78e600d1522134cf33daa98faed9700f62291a9495729b278076d3aeecd23","validator_index":"943310","committee_index":"55","committee_length":"435","committees_at_slot":"64","validator_committee_index":"369","slot":"8005463"},{"pubkey":"0xacd43e7f7e403c99c9299e187f885bb2029a8fe886dec591bd6c4fa6f6a042c773da1fb2f8f3f9ac4370dab6529715b9","validator_index":"977217","committee_index":"2","committee_length":"435","committees_at_slot":"64","validator_committee_index":"334","slot":"8005464"},{"pubkey":"0xa5d0bdddd535c4aa667b1022a7881e48565c426b1e682ff012362ee752a78773e6ed3f28ae600966344b37d881e9ceb0","validator_index":"968519","committee_index":"21","committee_length":"435","committees_at_slot":"64","validator_committee_index":"91","slot":"8005451"},{"pubkey":"0x84f674ca43562926f748393b8ba9ed39e6fb1a5101e31008243e0b20cb09b7e204fe622efef021680bcc3194796cbac0","validator_index":"1003006","committee_index":"6","committee_length":"435","committees_at_slot":"64","validator_committee_index":"214","slot":"8005454"}]}: command not found

real 2m27.239s
user 0m0.056s
sys 0m0.043s`

[rkapka]

Hi @zhy827827 , was the epoch that you asked for an epoch in the past (even 1 epoch in the past)? If yes, then the answer is pretty clear: we save past states every 2048 slots by default and because of this we had to replay states from the latest saved one to the requested one, which currently takes a lot of time.

[potuz]

Hi @zhy827827 , was the epoch that you asked for an epoch in the past (even 1 epoch in the past)? If yes, then the answer is pretty clear: we save past states every 2048 slots by default and because of this we had to replay states from the latest saved one to the requested one, which currently takes a lot of time.

you shouldn't need to replay that long as you have the finalized state in memory. Over two epochs in the past is more arguable, but still we keep several checkpoint states in the cache for a while

[zhy827827]

@rkapka @potuz Thank you!!Wishing you a pleasant work experience