BSC - No block headers to write in this log period block number #5703

breezytm · 2022-10-11T15:25:26Z

System information

Erigon version: erigon version 2.27.0-stable-6d8b304e

OS & Version: Linux

Commit hash : 6d8b304

Expected behaviour

Syncing to the latest block

Actual behaviour

Error: No block headers to write in this log period

bsc-node-000            | [INFO] [10-11|15:02:13.458] [p2p] GoodPeers                          eth66=86
bsc-node-000            | [INFO] [10-11|15:02:13.470] [txpool] stat                            block=0 pending=1 baseFee=0 queued=16 alloc=3.8GB sys=65.4GB
bsc-node-000            | [INFO] [10-11|15:02:31.125] [2/16 Headers] No block headers to write in this log period block number=20304127 blk/second=0.000
bsc-node-000            | [INFO] [10-11|15:02:51.125] [2/16 Headers] No block headers to write in this log period block number=20304127 blk/second=0.000
bsc-node-000            | [INFO] [10-11|15:03:11.126] [2/16 Headers] No block headers to write in this log period block number=20304127 blk/second=0.000
bsc-node-000            | [INFO] [10-11|15:03:13.468] [txpool] stat                            block=0 pending=1 baseFee=0 queued=16 alloc=3.3GB sys=65.4GB
bsc-node-000            | [INFO] [10-11|15:03:31.125] [2/16 Headers] No block headers to write in this log period block number=20304127 blk/second=0.000
bsc-node-000            | [INFO] [10-11|15:03:51.126] [2/16 Headers] No block headers to write in this log period block number=20304127 blk/second=0.000

Steps to reproduce the behaviour

    command:
      - erigon
      - --datadir=/root/.local/share/erigon
      - --chain=bsc
      - --port=29505
      - --private.api.addr=0.0.0.0:9090
      - --http=false
      - --db.pagesize=16kb
      - --prune=hrtc
      - --torrent.port=29506
      - --torrent.download.rate=1000mb
      - --torrent.download.slots=6
      - --downloader.verify
      - --batchSize=512M
      - --etl.bufferSize=512M
      - --healthcheck
      - --verbosity=3

The text was updated successfully, but these errors were encountered:

nikolinsko · 2022-10-12T09:29:22Z

additional log:

[WARN] [10-12|11:18:19.176] [7/16 Execution] Execution failed        block=22107425 hash=0x71044292c200a2138a4b1cfe258bbeb3998e1a6cc1a06f0a380522c4527702c2 err="expected system tx (hash 0xf1753cfe253f554860cac1c86ef60fcb5bf094116afd0ddeff8548e54b30d29f, nonce 831138, to 0x0000000000000000000000000000000000001000, value 0x161441e802c87a4, gas 9223372036854775807, gasPrice 0x0, data f340fa010000000000000000000000009f8ccdafcc39f3c7d6ebf637c9151673cbc36b88), actual tx (hash 0xe053c8c705c470a3032854fa2d36b41ca638d7972d48f9c9ec344bbc8a05bca8, nonce 831138, to 0x0000000000000000000000000000000000001000, value 0x16144381caf53a4, gas 9223372036854775807, gasPrice 0x0, data f340fa010000000000000000000000009f8ccdafcc39f3c7d6ebf637c9151673cbc36b88)"
[INFO] [10-12|11:18:19.177] UnwindTo                                 block=22107424 bad_block_hash=0x71044292c200a2138a4b1cfe258bbeb3998e1a6cc1a06f0a380522c4527702c2

Erigon version: v2.28.0

mikolajsobolewski · 2022-10-12T10:04:45Z

@AdminAnticaptcha please upgrade to v2.28.0 this is the BSC hardfork Moran

nikolinsko · 2022-10-12T10:30:12Z

@AdminAnticaptcha did you do something in particular?

@mikolajsobolewski

i'm at the following commit / tag:

git_branch=heads/v2.28.0 git_tag=v2.28.0 git_commit=355837aee5ea2374bf420eff6e93360deb83d3f7

after:

[WARN] [10-12|12:17:08.820] [7/16 Execution] Execution failed        block=22107425 hash=0x71044292c200a2138a4b1cfe258bbeb3998e1a6cc1a06f0a380522c4527702c2 err="expected system tx (hash 0xf1753cfe253f554860cac1c86ef60fcb5bf094116afd0ddeff8548e54b30d29f, nonce 831138, to 0x0000000000000000000000000000000000001000, value 0x161441e802c87a4, gas 9223372036854775807, gasPrice 0x0, data f340fa010000000000000000000000009f8ccdafcc39f3c7d6ebf637c9151673cbc36b88), actual tx (hash 0xe053c8c705c470a3032854fa2d36b41ca638d7972d48f9c9ec344bbc8a05bca8, nonce 831138, to 0x0000000000000000000000000000000000001000, value 0x16144381caf53a4, gas 9223372036854775807, gasPrice 0x0, data f340fa010000000000000000000000009f8ccdafcc39f3c7d6ebf637c9151673cbc36b88)"
[INFO] [10-12|12:17:08.820] UnwindTo                                 block=22107424 bad_block_hash=0x71044292c200a2138a4b1cfe258bbeb3998e1a6cc1a06f0a380522c4527702c2
[INFO] [10-12|12:17:08.820] [7/16 Execution] Completed on            block=22107424
[INFO] [10-12|12:17:12.526] [2/16 Headers] Waiting for headers...    from=22107424

the following is looped

[INFO] [10-12|12:23:12.527] [2/16 Headers] No block headers to write in this log period block number=22107424 blk/second=0.000
[INFO] [10-12|12:23:12.527] Req/resp stats                           req=5 reqMin=22107528 reqMax=22108296 skel=15 skelMin=22108960 skelMax=22403680 resp=100 respMin=22107721 respMax=22109074 dups=30
[INFO] [10-12|12:23:12.527] [Downloader] Queue sizes                 anchors=1 links=1354 persisted=64055
[INFO] [10-12|12:23:12.527] {22107721-22109074 links=1354 (22107721-22109074)} => 14663dfdaf1c69c514afeecc726a10af8e9be6766bfa1b1f9248285873388ec9, anchorQueue.idx=0, next retry in 1.691698652s

integration shows:

Snapshots                        22107422        0
Headers                          22107424        0
BlockHashes                      22107424        0
Bodies                           22107424        0
Senders                          22107424        0
Execution                        22107424        22107422
Translation                      0               0
HashState                        22107422        0
IntermediateHashes               22107422        22107422
AccountHistoryIndex              22107422        22107422
StorageHistoryIndex              22107422        22107422
LogIndex                         22107422        22107422
CallTraces                       22107422        22107422
TxLookup                         22107422        22102422
Finish                           22107422        0

prune distance: --prune.h.older=5000 --prune.r.older=5000 --prune.t.older=5000 --prune.c.older=5000

sequence: EthTx=3541637689, NonCanonicalTx=230018

first header in db: 1
first body in db: 1


snapsthos: blocks=0, segments=0, indices=0

affixpin · 2022-10-12T10:39:43Z

The same issue, 2.28.0 update doesn't help

tinoh9 · 2022-10-12T12:47:38Z

@AdminAnticaptcha did you modify anything with your synced node?

chetanbothra · 2022-10-12T14:42:01Z

Stuck at same issue, was earlier working with 2.27.0 and before 5hrs got this issue. Upgraded erigon to 2.28.0 and still stuck at same place

erigon-erigon-1  | [INFO] [10-12|14:41:38.054] [2/16 Headers] No block headers to write in this log period block number=22107424 blk/second=0.000
erigon-erigon-1  | [INFO] [10-12|14:41:38.054] Req/resp stats                           req=31 reqMin=22107423 reqMax=22112585 skel=14 skelMin=22108960 skelMax=22403680 resp=77 respMin=22108000 respMax=22114132 dups=85
erigon-erigon-1  | [INFO] [10-12|14:41:38.054] [Downloader] Queue sizes                 anchors=1 links=6134 persisted=59168
erigon-erigon-1  | [INFO] [10-12|14:41:38.054] {22108000-22114132 links=6134 (22108000-22114130-22114132)} => 3cc9c62721e50b37936edbcc888fc5171c056fc449eaa02666adff10d5cc1d81, anchorQueue.idx=0, next retry in 38.281912ms

chetanbothra · 2022-10-12T15:56:43Z

I just ran ./build/bin/integration stage_exec --datadir <your_datadir> --chain bsc --unwind 10 and start erigon and seems to be working as expected

@poloplayingsolo @nikolinsko @AdminAnticaptcha @breezytm can u try this once and see if this is working for u.

breezytm · 2022-10-12T16:10:27Z

I just ran ./build/bin/integration stage_exec --datadir <your_datadir> --chain bsc --unwind 10 and start erigon and seems to be working as expected

@poloplayingsolo @nikolinsko @AdminAnticaptcha @breezytm can u try this once and see if this is working for u.

In my case I am not sure if there is any data to unwind because this is a new deployment. It never synced. It does that right after finish the snapshot download.

nikolinsko · 2022-10-12T16:55:13Z

I just ran ./build/bin/integration stage_exec --datadir <your_datadir> --chain bsc --unwind 10 and start erigon and seems to be working as expected

@poloplayingsolo @nikolinsko @AdminAnticaptcha @breezytm can u try this once and see if this is working for u.

yup, seems like that fixes the issue.

Thanks for the tip.

chetanbothra · 2022-10-12T19:01:11Z

@breezytm still give a try. As i was stuck at same issue. U dnt need full sync to get complete to unwind.

a26nine · 2022-10-12T21:25:19Z

Did anyone find a workaround for this? Unwinding doesn't help.

I synced multiple nodes from scratch and now it's stuck at No block headers to write in this log period block number=xxxxx. On every restart, it adds a few blocks and then gets stuck again.

a26nine · 2022-10-13T04:26:14Z

Adding the --snap.stop flag and restarting erigon fixed this for me — multiple nodes.

tinoh9 · 2022-10-13T07:50:25Z

@a26nine I have tried your flag but it's still stucked, did you input any additional flag?

AlexeyAkhunov · 2022-10-13T09:18:58Z

Perhaps boot nodes need to be updated for BSC, can someone check

a26nine · 2022-10-13T12:04:33Z

@a26nine I have tried your flag but it's still stucked, did you input any additional flag?

I did add --downloader.verify earlier for another issue — don't know whether that would help.

breezytm · 2022-10-13T13:11:41Z

@breezytm still give a try. As i was stuck at same issue. U dnt need full sync to get complete to unwind.

This is what I am getting

bsc-node-000            | [INFO] [10-13|13:08:38.253] [Downloader] Queue sizes                 anchors=1 links=801 persisted=65536
bsc-node-000            | [INFO] [10-13|13:08:38.253] {22139860-22140660 links=801 (22139860-22140660)} => 50669e6f4e517c35cedba5a40781c83c5286da9bc21cfc08b9ebc62d5d8824ce, anchorQueue.idx=0, next retry in 3.591337491s 
bsc-node-000            | [INFO] [10-13|13:08:58.252] [2/16 Headers] No block headers to write in this log period block number=21499999 blk/second=0.000
bsc-node-000            | [INFO] [10-13|13:08:58.252] Req/resp stats                           req=5 reqMin=22139283 reqMax=22139859 skel=13 skelMin=21501535 skelMax=21796255 resp=32 respMin=22140666 respMax=22140667 dups=7
bsc-node-000            | [INFO] [10-13|13:08:58.252] [Downloader] Queue sizes                 anchors=1 links=1192 persisted=65536
bsc-node-000            | [INFO] [10-13|13:08:58.252] {22139476-22140667 links=1192 (22139476-22140667)} => b9813bf7c311b2bff7a5945d221381ed36d2bb308d33a71d90eda071242068a5, anchorQueue.idx=0, next retry in 1.305319751s 
bsc-node-000            | [INFO] [10-13|13:09:18.252] [2/16 Headers] No block headers to write in this log period block number=21499999 blk/second=0.000
bsc-node-000            | [INFO] [10-13|13:09:18.252] Req/resp stats                           req=4 reqMin=22139283 reqMax=22139475 skel=17 skelMin=21501535 skelMax=21796255 resp=43 respMin=22140638 respMax=22140674 dups=14
bsc-node-000            | [INFO] [10-13|13:09:18.252] [Downloader] Queue sizes                 anchors=1 links=1199 persisted=65536
bsc-node-000            | [INFO] [10-13|13:09:18.253] {22139476-22140674 links=1199 (22139476-22140674)} => b9813bf7c311b2bff7a5945d221381ed36d2bb308d33a71d90eda071242068a5, anchorQueue.idx=0, next retry in 3.133441457s 
bsc-node-000            | [INFO] [10-13|13:09:28.888] [txpool] stat                            block=21499999 pending=0 baseFee=0 queued=91 alloc=2.6GB sys=5.1GB
bsc-node-000            | [INFO] [10-13|13:09:38.252] [2/16 Headers] No block headers to write in this log period block number=21499999 blk/second=0.000
bsc-node-000            | [INFO] [10-13|13:09:38.252] Req/resp stats                           req=6 reqMin=22138707 reqMax=22139475 skel=15 skelMin=21501535 skelMax=21796255 resp=46 respMin=22140671 respMax=22140681 dups=15
bsc-node-000            | [INFO] [10-13|13:09:38.252] [Downloader] Queue sizes                 anchors=1 links=1782 persisted=65536
bsc-node-000            | [INFO] [10-13|13:09:38.252] {22138900-22140681 links=1782 (22138900-22140681)} => 56042078e493a16e1caa5f922eea21a9777832bdfa601c512a9a5c03a289ed70, anchorQueue.idx=0, next retry in -3.992642ms

yixun-alpha · 2022-10-14T03:53:41Z

Just to chime in that I have the exact same issues as above on v2.28.0, have tried all workarounds in the thread (downloader.verify, snap.stop, unwind etc) without success

PolarSyncZ · 2022-10-15T08:20:33Z

I had this same issue with a new node I was spinning up from snapshots. I tried all sorts of things and was finally able to resolve it by deleting all of the snapshot files and restarting erigon then letting them re-download and index etc. It took about 24 hours but it solved the issue and my node is now functioning normally.

allada · 2022-10-15T14:59:42Z

As @PolarSyncZ said, I'd been working with them to help get their node online. I'll be updating the s3 snapshot folder in https://github.com/allada/bsc-archive-snapshot. I'll post back here with the s3 command you'all can use to download just the snapshots tomorrow-ish or you can delete the snapshots folder and torrent the files yourself.

allada · 2022-10-16T20:20:19Z

For those that want to test if your snapshot folder will fix the issue, here's what should fix the issue:

make integration
./build/bin/integration stage_exec --datadir <your_datadir> --chain bsc --unwind 1000 # You might be able to get away with less blocks
rm -rf <your_datadir>/snapshots/*
cd <your_datadir>/snapshots && aws s3 --request-payer=requester sync s3://public-blockchain-snapshots/bsc/erigon-snapshots-folder-latest/ ./

Note: You will need an AWS account configured on the computer it is running on, as your aws account will be billed for the download costs, which is about $50USD in data xfer costs unless you are running the instance in AWS's us-west-2 data center, which it'll be about $0.10USD.

f0rmatting · 2022-10-17T00:41:22Z

For those that want to test if your snapshot folder will fix the issue, here's what should fix the issue:

make integration

./build/bin/integration stage_exec --datadir <your_datadir> --chain bsc --unwind 1000 # You might be able to get away with less blocks

rm -rf <your_datadir>/snapshots/*

cd <your_datadir>/snapshots && aws s3 --request-payer=requester sync s3://public-blockchain-snapshots/bsc/erigon-snapshots-folder-latest/ ./

Note: You will need an AWS account configured on the computer it is running on, as your aws account will be billed for the download costs, which is about $50USD in data xfer costs unless you are running the instance in AWS's us-west-2 data center, which it'll be about $0.10USD.

Hi @allada, thanks for you snapshots file, but maybe s3 sync need "ListObjectsV2" permission
When I use "aws s3 --request-payer=requester sync" return:

An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

allada · 2022-10-17T00:47:06Z

Hi @allada, thanks for you snapshots file, but maybe s3 sync need "ListObjectsV2" permission

You need to make sure the account you are using has the ability to charge your account for S3 requests. These permissions are set in your account, and IIRC you can just allow s3:ListBucket and other related permissions to resource: arn:aws:s3:::public-blockchain-snapshots and I think it'll work.

f0rmatting · 2022-10-17T01:05:34Z

Hi @allada, thanks for you snapshots file, but maybe s3 sync need "ListObjectsV2" permission

You need to make sure the account you are using has the ability to charge your account for S3 requests. These permissions are set in your account, and IIRC you can just allow s3:ListBucket and other related permissions to resource: arn:aws:s3:::public-blockchain-snapshots and I think it'll work.

@allada Thanks, it's my problem.
It's syncing now.

nikolinsko · 2022-10-17T21:31:29Z

fyi: there is also https://github.com/BNB48Club/bsc-snapshots. Seems to be from a validator, so as close as trustworthy as you can get on bsc :D

1scrooge · 2022-10-18T07:34:53Z

I have the same issue
Erigon version is 2.28.1
Last log:

erigon_1  | [INFO] [10-18|07:24:55.556] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:25:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:25:24.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:25:44.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:25:55.557] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.5GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:26:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:26:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:26:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:26:55.535] [p2p] GoodPeers                          eth66=20
erigon_1  | [INFO] [10-18|07:26:55.557] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:27:04.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:27:24.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:27:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:27:55.556] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.3GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:28:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:28:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:28:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:28:55.535] [p2p] GoodPeers                          eth66=20
erigon_1  | [INFO] [10-18|07:28:55.555] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:29:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:29:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:29:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:29:55.556] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.3GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:30:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:30:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:30:44.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:30:55.535] [p2p] GoodPeers                          eth66=20
erigon_1  | [INFO] [10-18|07:30:55.555] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:31:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000

Restart doesn't help

allada · 2022-10-18T14:34:05Z

@1scrooge, read above for fix. There is unlikely to be any "fix" by erigon since users can fix it themselves.

amurde · 2022-10-20T10:39:07Z

Same version , same problem for me.
Tried to unwind stages and re-download snapshots. Stuck at the same message but different block.

nftgeek · 2022-10-22T03:55:43Z

I have the same issue Erigon version is 2.28.1 Last log:

erigon_1  | [INFO] [10-18|07:24:55.556] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:25:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:25:24.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:25:44.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:25:55.557] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.5GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:26:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:26:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:26:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:26:55.535] [p2p] GoodPeers                          eth66=20
erigon_1  | [INFO] [10-18|07:26:55.557] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:27:04.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:27:24.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:27:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:27:55.556] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.3GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:28:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:28:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:28:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:28:55.535] [p2p] GoodPeers                          eth66=20
erigon_1  | [INFO] [10-18|07:28:55.555] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:29:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:29:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:29:44.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:29:55.556] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.3GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:30:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:30:24.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:30:44.635] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000
erigon_1  | [INFO] [10-18|07:30:55.535] [p2p] GoodPeers                          eth66=20
erigon_1  | [INFO] [10-18|07:30:55.555] [txpool] stat                            block=19999999 pending=1 baseFee=0 queued=146 alloc=4.2GB sys=7.0GB
erigon_1  | [INFO] [10-18|07:31:04.634] [2/16 Headers] No block headers to write in this log period block number=20348671 blk/second=0.000

Restart doesn't help

Hi @1scrooge @amurde Double check your firewall config. I fixed this issue by allowing tcp/udp ports of downloader and sentry containers.

alexmance · 2022-12-27T18:54:13Z

We had the same issue as you @luwang920.

We were able to move forward (or it seems so) with the following command: ./integration stage_exec --datadir=/mnt/erigon/ --chain='bor-mainnet' --unwind 100
(we tried first with 10, then 20 blocks, nothing changed. Jumped to 100 and saw some changes)

version: erigon version 2.31.0-stable-a53e8a3e

luwang920 · 2022-12-28T07:01:15Z

Hi alexmance
Thanks , it didn't work for me ..
I tried to roll back several times...

alexmance · 2022-12-28T08:49:42Z

Hi @luwang920
Just take a look at our servers.. and after 4 hours it crashed again and we are stuck in the same block... sorry man.
We decided to re-sync from the official snapshot today, as it has been a few days with this behavior...

nikolinsko · 2022-12-28T12:03:58Z

this is still an issue for me on

[INFO] [12-28|13:00:34.744] Build info                               git_branch=heads/v2.32.0 git_tag=v2.32.0 git_commit=021891a311430ffefde77fcceda567ba324e9abe

[INFO] [12-28|12:44:24.951] [2/16 Headers] No block headers to write in this log period block number=24199022
[INFO] [12-28|12:44:24.951] Req/resp stats                           req=0 reqMin=0 reqMax=0 skel=17 skelMin=24199023 skelMax=24493935 resp=32 respMin=24199023 respMax=24293803 dups=941
[DBUG] [12-28|12:44:24.951] [Downloader] Queue sizes                 anchors=0 links=97497 persisted=65535
[DBUG] [12-28|12:44:26.084] [Downloader] Request skeleton            anchors=0 top seen height=24293804 highestInDb=24199022
[DBUG] [12-28|12:44:27.085] [Downloader] Request skeleton            anchors=0 top seen height=24293804 highestInDb=24199022
[DBUG] [12-28|12:44:28.086] [Downloader] Request skeleton            anchors=0 top seen height=24293804 highestInDb=24199022
[DBUG] [12-28|12:44:29.179] [Downloader] Request skeleton            anchors=0 top seen height=24293805 highestInDb=24199022
[DBUG] [12-28|12:44:30.180] [Downloader] Request skeleton            anchors=0 top seen height=24293805 highestInDb=24199022
[DBUG] [12-28|12:44:31.442] [Downloader] Request skeleton            anchors=0 top seen height=24293806 highestInDb=24199022
[DBUG] [12-28|12:44:32.536] [Downloader] Request skeleton            anchors=0 top seen height=24293806 highestInDb=24199022
[DBUG] [12-28|12:44:33.536] [Downloader] Request skeleton            anchors=0 top seen height=24293806 highestInDb=24199022
[DBUG] [12-28|12:44:35.213] [Downloader] Request skeleton            anchors=0 top seen height=24293807 highestInDb=24199022
[DBUG] [12-28|12:44:36.894] [Downloader] Request skeleton            anchors=0 top seen height=24293807 highestInDb=24199022
[DBUG] [12-28|12:44:36.986] [Downloader] new anchor too far in the past: 0, latest header in db: 24199022
[DBUG] [12-28|12:44:38.093] [Downloader] Request skeleton            anchors=0 top seen height=24293808 highestInDb=24199022
[DBUG] [12-28|12:44:39.139] [Downloader] Request skeleton            anchors=0 top seen height=24293808 highestInDb=24199022
[DBUG] [12-28|12:44:39.231] [Downloader] new anchor too far in the past: 0, latest header in db: 24199022
[INFO] [12-28|12:44:39.484] [txpool] stat                            block=24199023 pending=10000 baseFee=636 queued=2000 alloc=4.7GB sys=9.2GB

egorFiNE · 2022-12-29T07:16:21Z

Still happens daily to me with the BSC node on 2.32. unwind usually helps.

ChinW · 2023-01-05T13:18:16Z

also met the same log, I am not sure if it is just expected?

I have tried to run unwind=10 and then restarted the erigon, still see below log

[INFO] [01-05|21:14:41.714] [2/16 Headers] Waiting for headers...    from=22999999
[INFO] [01-05|21:15:01.715] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:15:21.716] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:15:39.629] [txpool] stat                            block=22999999 pending=0 baseFee=0 queued=15 alloc=3.0GB sys=3.3GB
[INFO] [01-05|21:15:41.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:01.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:21.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:21.715] Req/resp stats                           req=3 reqMin=24522988 reqMax=24523180 skel=0 skelMin=0 skelMax=0 resp=1 respMin=24523181 respMax=24523181 dups=0
[INFO] [01-05|21:16:39.601] [p2p] GoodPeers                          eth66=1 eth67=0
[INFO] [01-05|21:16:39.629] [txpool] stat                            block=22999999 pending=0 baseFee=0 queued=16 alloc=2.3GB sys=3.4GB
[INFO] [01-05|21:16:41.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:41.714] Req/resp stats                           req=2 reqMin=24522988 reqMax=24523180 skel=0 skelMin=0 skelMax=0 resp=1 respMin=0 respMax=0 dups=1
[INFO] [01-05|21:17:01.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:17:01.714] Req/resp stats                           req=3 reqMin=24522988 reqMax=24523180 skel=0 skelMin=0 skelMax=0 resp=1 respMin=0 respMax=0 dups=1
[INFO] [01-05|21:17:21.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:17:21.714] Req/resp stats                           req=10 reqMin=24521285 reqMax=24523205 skel=0 skelMin=0 skelMax=0 resp=14 respMin=24521478 respMax=24523206 dups=4

my build info Build info git_branch=stable git_tag=v2.32.0 git_commit=021891a311430ffefde77fcceda567ba324e9abe

luwang920 · 2023-01-05T13:25:15Z

also met the same log, I am not sure if it is just expected?

I have tried to run unwind=10 and then restarted the erigon, still see below log

[INFO] [01-05|21:14:41.714] [2/16 Headers] Waiting for headers...    from=22999999
[INFO] [01-05|21:15:01.715] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:15:21.716] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:15:39.629] [txpool] stat                            block=22999999 pending=0 baseFee=0 queued=15 alloc=3.0GB sys=3.3GB
[INFO] [01-05|21:15:41.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:01.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:21.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:21.715] Req/resp stats                           req=3 reqMin=24522988 reqMax=24523180 skel=0 skelMin=0 skelMax=0 resp=1 respMin=24523181 respMax=24523181 dups=0
[INFO] [01-05|21:16:39.601] [p2p] GoodPeers                          eth66=1 eth67=0
[INFO] [01-05|21:16:39.629] [txpool] stat                            block=22999999 pending=0 baseFee=0 queued=16 alloc=2.3GB sys=3.4GB
[INFO] [01-05|21:16:41.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:16:41.714] Req/resp stats                           req=2 reqMin=24522988 reqMax=24523180 skel=0 skelMin=0 skelMax=0 resp=1 respMin=0 respMax=0 dups=1
[INFO] [01-05|21:17:01.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:17:01.714] Req/resp stats                           req=3 reqMin=24522988 reqMax=24523180 skel=0 skelMin=0 skelMax=0 resp=1 respMin=0 respMax=0 dups=1
[INFO] [01-05|21:17:21.714] [2/16 Headers] No block headers to write in this log period block number=22999999
[INFO] [01-05|21:17:21.714] Req/resp stats                           req=10 reqMin=24521285 reqMax=24523205 skel=0 skelMin=0 skelMax=0 resp=14 respMin=24521478 respMax=24523206 dups=4

my build info Build info git_branch=stable git_tag=v2.32.0 git_commit=021891a311430ffefde77fcceda567ba324e9abe

I was stucking in this error for several days.
Unwind , and stop at the same block,
Unwind , and stop at the same block again..
Finally , I downloaded the latest offical snapshot, resync ..
My node is polygon archive mainnet , it took me 3 days to download and unzip the snapshot .....terrible...

Br
Lu

nandubatchu · 2023-01-11T21:33:41Z

@egorFiNE can you help me with the unwind steps you followed for bor-mainnet?

egorFiNE · 2023-01-12T10:17:54Z

@nandubatchu we gave up on erigon for bor-mainnet. It took something close to 20 days to sync and then threw "no block headers" daily or even more often.

mikolajsobolewski · 2023-01-12T12:07:41Z

@nandubatchu we gave up on erigon for bor-mainnet. It took something close to 20 days to sync and then threw "no block headers" daily or even more often.

We also think to give up on erigon for BSC. We have the same errors in mainnet and have no answers from Erigon developers.

egorFiNE · 2023-01-12T12:28:06Z

@mikolajsobolewski on BSC it's not that much of a PITA. It happens like twice per week. You unwind ten blocks, restart and in about 20 minutes or less it's up and running again.

mikolajsobolewski · 2023-01-12T12:48:41Z

@mikolajsobolewski on BSC it's not that much of a PITA. It happens like twice per week. You unwind ten blocks, restart and in about 20 minutes or less it's up and running again.

But if we use their geth fork always receives new blocks. I don't know how it doest that, but fact is fact. So i think, that it is the erigon error. We don't want to restart the node 5-6 times per day. it's really strange since 2022 Oct no one work solution.

egorFiNE · 2023-01-16T10:53:14Z

Updated to 2.34.0 and here we go again. :-(

nebojsa-ristovic · 2023-01-16T11:09:05Z

Same here for both Polygon and BSC Erigon node.

egorFiNE · 2023-01-16T11:23:14Z

...aaaand again.

egorFiNE · 2023-01-16T19:47:59Z

Now I'm having this issue multiple times per day on 2.34.0 on BSC BUT a simple restart helps, no need to unwind.

AskAlexSharov · 2023-01-17T12:59:49Z

Probably related: #6532

seonggwonyoon · 2023-01-27T08:15:05Z

Same on 2.36.1 (polygon and bsc)

akshat-pyor · 2023-03-03T09:50:45Z

@egorFiNE

Now I'm having this issue multiple times per day on 2.34.0 on BSC BUT a simple restart helps, no need to unwind.

facing the same issue (No block headers to write in this log period block number
) on version 2.39.0. for bsc. what is the way out here given that i restarted it multiple times.

Vitawt-The-Duke · 2023-03-13T20:49:00Z

Hello. I've been stuck with the same problem and fixed it with updating to the latest binary and unwinding:
(The client stopped getting new data for more than a couple of weeks I suppose)
My flow:

Timestamp 2023-03-13_20:47:02
Latest block at current node is: 26110021
Latest block at mainnet is: 26439285
diff is: 329264

Node parameters:

128GB of RAM
zraid0 (4x stripe SSD)
AMD Ryzen 9 5950X 16-Core Processor

Startup parameters:

#!/bin/bash
exec /go/erigon/build/bin/erigon --chain="bsc" \
    --snapshots=true \
    --txpool.disable \
    --db.pagesize="16k" \
    --datadir="/data/bsc" \
    --http --http.vhosts="*" --http.addr="0.0.0.0" --http.port="8545" \
    --http.api="net,eth,web3" \
    --ws \
    --metrics --metrics.addr="0.0.0.0" --metrics.port="6060"

tried to fix:
-update the client to the latest version
-unwind
and it helps!!

[WARN] [03-13|20:36:58.215] no log dir set, console logging only
[INFO] [03-13|20:36:58.216] Starting metrics server                  addr=http://0.0.0.0:6060/debug/metrics/prometheus
[WARN] [03-13|20:36:58.221] no log dir set, console logging only
[INFO] [03-13|20:36:58.221] Build info                               git_branch=HEAD git_tag=v2.40.1 git_commit=94b4cee5a898a47b9241ce824622af6350e5dc5b
[INFO] [03-13|20:36:58.221] Starting Erigon on BSC mainnet...
[INFO] [03-13|20:36:58.224] Maximum peer count                       ETH=100 total=100
[INFO] [03-13|20:36:58.224] starting HTTP APIs                       APIs=net,eth,web3
[INFO] [03-13|20:36:58.224] torrent verbosity                        level=WRN
[INFO] [03-13|20:37:00.327] Set global gas cap                       cap=50000000
[INFO] [03-13|20:37:00.333] [Downloader] Runnning with               ipv6-enabled=true ipv4-enabled=true download.rate=16mb upload.rate=4mb
[INFO] [03-13|20:37:00.333] Opening Database                         label=chaindata path=/data/bsc/chaindata
[INFO] [03-13|20:37:00.339] Initialised chain configuration          config="{ChainID: 56 Ramanujan: 0, Niels: 0, MirrorSync: 5184000, Bruno: 13082000, Euler: 18907621, Gibbs: 23846001, Nano: 21962149, Moran: 22107423, Gibbs: 23846001, Engine: parlia}" genesis=0x0d21840abff46b96c84b2ac9e10e4f5cdaeb5693cb665db62a2f3b02d2d57b5b
[INFO] [03-13|20:37:00.339] Effective                                prune_flags= snapshot_flags="--snapshots=true" history.v3=false
[INFO] [03-13|20:37:03.960] Initialising Ethereum protocol           network=56
[INFO] [03-13|20:37:04.242] Starting private RPC server              on=127.0.0.1:9090
[INFO] [03-13|20:37:04.242] new subscription to logs established
[INFO] [03-13|20:37:04.243] rpc filters: subscribing to Erigon events
[INFO] [03-13|20:37:04.243] new subscription to newHeaders established
[INFO] [03-13|20:37:04.244] Reading JWT secret                       path=/data/bsc/jwt.hex
[INFO] [03-13|20:37:04.244] HTTP endpoint opened for Engine API      url=127.0.0.1:8551 ws=true ws.compression=true
[INFO] [03-13|20:37:04.244] HTTP endpoint opened                     url=[::]:8545 ws=true ws.compression=true grpc=false
[INFO] [03-13|20:37:04.250] Started P2P networking                   version=68 self=enode://209ec59bf1d372caecb61836089e006c34f31dadefdc7e5abecebb45bd5735bc7ccf6b00c209a344fb584b2f6fe5bc83168fcf51643067ef2e7c262fd54a7eef@127.0.0.1:30304 name=erigon/v2.40.1-stable-94b4cee5/linux-amd64/go1.20.2
[INFO] [03-13|20:37:04.282] Started P2P networking                   version=67 self=enode://209ec59bf1d372caecb61836089e006c34f31dadefdc7e5abecebb45bd5735bc7ccf6b00c209a344fb584b2f6fe5bc83168fcf51643067ef2e7c262fd54a7eef@127.0.0.1:30303 name=erigon/v2.40.1-stable-94b4cee5/linux-amd64/go1.20.2
[INFO] [03-13|20:37:04.284] [1/15 Snapshots] Fetching torrent files metadata
[INFO] [03-13|20:37:04.287] [snapshots] Blocks Stat                  blocks=26020k indices=26020k alloc=2.8GB sys=3.0GB
[INFO] [03-13|20:37:04.287] [2/15 Headers] Waiting for headers...    from=26438754
[INFO] [03-13|20:37:24.288] [2/15 Headers] Wrote block headers       number=26439044 blk/second=14.500 alloc=3.1GB sys=3.3GB
[INFO] [03-13|20:37:44.288] [2/15 Headers] No block headers to write in this log period block number=26439044
[INFO] [03-13|20:38:44.288] [2/15 Headers] No block headers to write in this log period block number=26439044
[INFO] [03-13|20:39:03.960] [p2p] GoodPeers                          eth66=26
[INFO] [03-13|20:39:04.287] [2/15 Headers] No block headers to write in this log period block number=26439044
[INFO] [03-13|20:39:24.287] [2/15 Headers] No block headers to write in this log period block number=26439044

have updates after rewind:

[INFO] [03-13|20:48:24.288] [2/15 Headers] Wrote block headers       number=26439236 blk/second=9.600 alloc=2.6GB sys=3.6GB

Morning update (13 hrs passed)

[INFO] [03-14|10:14:46.840] [5/15 Bodies] No block bodies to write in this log period block number=26341579
[INFO] [03-14|10:15:03.960] [p2p] GoodPeers                          eth66=9 eth67=1 eth68=14
[INFO] [03-14|10:15:06.841] [5/15 Bodies] No block bodies to write in this log period block number=26341579
[INFO] [03-14|10:15:26.841] [5/15 Bodies] No block bodies to write in this log period block number=26341579
[INFO] [03-14|10:15:46.840] [5/15 Bodies] No block bodies to write in this log period block number=26341579

today update(1 day+ passed):

[INFO] [03-15|11:46:39.294] [7/15 Execution] Executed blocks         number=26263449 blk/s=12.2 tx/s=1748.5 Mgas/s=183.2 gasState=0.19 batch=237.3MB alloc=6.1GB sys=10.2GB
[INFO] [03-15|11:46:59.355] [7/15 Execution] Executed blocks         number=26263673 blk/s=11.2 tx/s=1626.2 Mgas/s=173.9 gasState=0.19 batch=245.9MB alloc=4.2GB sys=10.2GB
[INFO] [03-15|11:47:19.440] [7/15 Execution] Executed blocks         number=26263908 blk/s=11.7 tx/s=1764.3 Mgas/s=186.3 gasState=0.20 batch=254.7MB alloc=5.6GB sys=10.2GB
[INFO] [03-15|11:47:22.606] Committed State                          gas reached=111327847129 gasTarget=549755813888
[INFO] [03-15|11:47:26.056] [p2p] GoodPeers                          eth67=5 eth68=11 eth66=14
[INFO] [03-15|11:47:52.734] [] ETL [2/2] Loading                     into=PlainCodeHash current_prefix=bbaaa420
[INFO] [03-15|11:48:03.423] Write to db                              progress=0.1M/3.8M current table=PlainState
[INFO] [03-15|11:48:39.094] [] ETL [2/2] Loading                     into=PlainState current_prefix=2ab0e9e4
[INFO] [03-15|11:49:09.093] [] ETL [2/2] Loading                     into=PlainState current_prefix=36a23dfd

Update! It works!

Latest block at current node is: 26257353
Latest block at mainnet is: 26485629
diff is: 228276

jajaislanina · 2023-03-14T09:11:33Z

These are our Polygon nodes running Erigon 2.39. (current block over time)

Same condition happens on other networks (BSC, Sepolia etc.)

Right now, when we detect that the node is stalling (flat line on the graph) we need to restart it and it will start processing after ~10 minutes.
This is happening at least once per day - sometimes multiple times even.

github-actions · 2023-05-26T02:22:27Z

This issue is stale because it has been open for 40 days with no activity. Remove stale label or comment, or this will be closed in 7 days.

yperbasis · 2023-06-07T10:53:40Z

BSC is no longer supported by Erigon. Please consider https://github.com/node-real/bsc-erigon instead.

d0zingcat · 2023-08-12T01:30:58Z

In my situation, I utilized an older snapshot (e.g., erigon-05-23), and configured --nodiscover and --bootnodes=xxxx (the enodes were chosen from here based on their recentness, ensuring no lag between these nodes and other nodes on the mainnet). After removing them and adding --snapshots=false, the blocks began to sync correctly.

This solution was found in this document. An excerpt from the document is:

curl localhost:8545 -X POST --data '{"jsonrpc":"2.0","method":"eth_syncing","id":1}' -H "Content-Type: application/json"

In my particular situation, the result was:

{"jsonrpc":"2.0","id":1,"result":false}

However, the block number was 43057508, which could not possibly be the most recent one!

a26nine mentioned this issue Oct 13, 2022

BSC testnet No block headers to write in this log period block number #5698

Closed

PolarSyncZ mentioned this issue Oct 15, 2022

Sync stuck during BSC blockchain pause #5652

Closed

mikolajsobolewski mentioned this issue Jan 12, 2023

BSC testnet cannot be synchronized normally #6556

Closed

AskAlexSharov mentioned this issue Jan 17, 2023

BSC hard to find peers #6532

Closed

icculp mentioned this issue Feb 21, 2023

BSC Archival falling behind regularly #6917

Closed

calmbeing mentioned this issue Apr 15, 2023

[5/15 Bodies] No block bodies to write in this log period block number=22999999 node-real/bsc-erigon#21

Closed

github-actions bot added the Stale label May 26, 2023

yperbasis closed this as completed Jun 7, 2023

BSC - No block headers to write in this log period block number #5703

BSC - No block headers to write in this log period block number #5703

Comments

breezytm commented Oct 11, 2022

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

nikolinsko commented Oct 12, 2022 • edited

mikolajsobolewski commented Oct 12, 2022 • edited

nikolinsko commented Oct 12, 2022 • edited

affixpin commented Oct 12, 2022

tinoh9 commented Oct 12, 2022

chetanbothra commented Oct 12, 2022

chetanbothra commented Oct 12, 2022

breezytm commented Oct 12, 2022

nikolinsko commented Oct 12, 2022 • edited

chetanbothra commented Oct 12, 2022

a26nine commented Oct 12, 2022

a26nine commented Oct 13, 2022

tinoh9 commented Oct 13, 2022

AlexeyAkhunov commented Oct 13, 2022

a26nine commented Oct 13, 2022

breezytm commented Oct 13, 2022 • edited

yixun-alpha commented Oct 14, 2022

PolarSyncZ commented Oct 15, 2022 • edited

allada commented Oct 15, 2022

allada commented Oct 16, 2022

f0rmatting commented Oct 17, 2022 • edited

allada commented Oct 17, 2022 • edited

f0rmatting commented Oct 17, 2022

nikolinsko commented Oct 17, 2022

1scrooge commented Oct 18, 2022 • edited

allada commented Oct 18, 2022

amurde commented Oct 20, 2022

nftgeek commented Oct 22, 2022

alexmance commented Dec 27, 2022 • edited

luwang920 commented Dec 28, 2022

alexmance commented Dec 28, 2022

nikolinsko commented Dec 28, 2022

egorFiNE commented Dec 29, 2022

ChinW commented Jan 5, 2023 • edited

luwang920 commented Jan 5, 2023 • edited

nandubatchu commented Jan 11, 2023

egorFiNE commented Jan 12, 2023

mikolajsobolewski commented Jan 12, 2023

egorFiNE commented Jan 12, 2023

mikolajsobolewski commented Jan 12, 2023 • edited

egorFiNE commented Jan 16, 2023

nebojsa-ristovic commented Jan 16, 2023

egorFiNE commented Jan 16, 2023

egorFiNE commented Jan 16, 2023

AskAlexSharov commented Jan 17, 2023

seonggwonyoon commented Jan 27, 2023

akshat-pyor commented Mar 3, 2023

Vitawt-The-Duke commented Mar 13, 2023 • edited

jajaislanina commented Mar 14, 2023 • edited

github-actions bot commented May 26, 2023

yperbasis commented Jun 7, 2023

d0zingcat commented Aug 12, 2023

nikolinsko commented Oct 12, 2022 •

edited

mikolajsobolewski commented Oct 12, 2022 •

edited

nikolinsko commented Oct 12, 2022 •

edited

nikolinsko commented Oct 12, 2022 •

edited

breezytm commented Oct 13, 2022 •

edited

PolarSyncZ commented Oct 15, 2022 •

edited

f0rmatting commented Oct 17, 2022 •

edited

allada commented Oct 17, 2022 •

edited

1scrooge commented Oct 18, 2022 •

edited

alexmance commented Dec 27, 2022 •

edited

ChinW commented Jan 5, 2023 •

edited

luwang920 commented Jan 5, 2023 •

edited

mikolajsobolewski commented Jan 12, 2023 •

edited

Vitawt-The-Duke commented Mar 13, 2023 •

edited

jajaislanina commented Mar 14, 2023 •

edited