monero-node doesn't save blockchain on stop/resume from previous sync height

[Gravity6909]

App Name

monero-node

Operating System

TrueNAS SCALE 23.10.1

App Version

0.18.3.1

Application Events

n/a

Application Logs

Initial sync

2024-02-27T23:40:54.228239219-08:00 2024-02-28 07:40:54.224	I Monero 'Fluorine Fermi' (v0.18.3.1-release)
2024-02-27T23:40:54.228300414-08:00 2024-02-28 07:40:54.224	I Initializing cryptonote protocol...
2024-02-27T23:40:54.228308228-08:00 2024-02-28 07:40:54.224	I Cryptonote protocol initialized OK
2024-02-27T23:40:54.228314790-08:00 2024-02-28 07:40:54.225	I Initializing core...
2024-02-27T23:40:54.228321223-08:00 2024-02-28 07:40:54.226	I Loading blockchain from folder /root/.bitmonero/lmdb ...
2024-02-27T23:40:54.308474033-08:00 2024-02-28 07:40:54.303	I Loading checkpoints
2024-02-27T23:40:54.308505643-08:00 2024-02-28 07:40:54.303	I Pruning blockchain...
2024-02-27T23:40:54.308513067-08:00 2024-02-28 07:40:54.304	I Core initialized OK
2024-02-27T23:40:54.308519779-08:00 2024-02-28 07:40:54.304	I Initializing p2p server...
2024-02-27T23:40:54.308526292-08:00 2024-02-28 07:40:54.305	I p2p server initialized OK
2024-02-27T23:40:54.308532283-08:00 2024-02-28 07:40:54.305	I Initializing core RPC server...
2024-02-27T23:40:54.308548643-08:00 2024-02-28 07:40:54.305	I Binding on 127.0.0.1 (IPv4):18081
2024-02-27T23:40:55.170837166-08:00 2024-02-28 07:40:55.155	I core RPC server initialized OK on port: 18081
2024-02-27T23:40:55.170887750-08:00 2024-02-28 07:40:55.155	I Initializing restricted RPC server...
2024-02-27T23:40:55.170895144-08:00 2024-02-28 07:40:55.155	I Binding on 0.0.0.0 (IPv4):18089
2024-02-27T23:40:55.667328095-08:00 2024-02-28 07:40:55.665	I restricted RPC server initialized OK on port: 18089
2024-02-27T23:40:55.667384251-08:00 2024-02-28 07:40:55.665	I Starting core RPC server...
2024-02-27T23:40:55.667391685-08:00 2024-02-28 07:40:55.665	I core RPC server started ok
2024-02-27T23:40:55.667398137-08:00 2024-02-28 07:40:55.665	I Starting restricted RPC server...
2024-02-27T23:40:55.667404549-08:00 2024-02-28 07:40:55.665	I restricted RPC server started ok
2024-02-27T23:40:55.667424256-08:00 2024-02-28 07:40:55.665	I Public RPC port 18089 will be advertised to other peers over P2P
2024-02-27T23:40:55.667433643-08:00 2024-02-28 07:40:55.665	I Starting p2p net loop...
2024-02-27T23:40:56.670895747-08:00 2024-02-28 07:40:56.667	I 
2024-02-27T23:40:56.670942615-08:00 2024-02-28 07:40:56.667	I **********************************************************************
2024-02-27T23:40:56.670950169-08:00 2024-02-28 07:40:56.667	I The daemon will start synchronizing with the network. This may take a long time to complete.
2024-02-27T23:40:56.670971319-08:00 2024-02-28 07:40:56.667	I 
2024-02-27T23:40:56.670977741-08:00 2024-02-28 07:40:56.667	I You can set the level of process detailization through "set_log <level|categories>" command,
2024-02-27T23:40:56.670984053-08:00 2024-02-28 07:40:56.667	I where <level> is between 0 (no details) and 4 (very verbose), or custom category based levels (eg, *:WARNING).
2024-02-27T23:40:56.670993992-08:00 2024-02-28 07:40:56.667	I 
2024-02-27T23:40:56.670999762-08:00 2024-02-28 07:40:56.667	I Use the "help" command to see the list of available commands.
2024-02-27T23:40:56.671006074-08:00 2024-02-28 07:40:56.667	I Use "help <command>" to see a command's documentation.
2024-02-27T23:40:56.671011895-08:00 2024-02-28 07:40:56.667	I **********************************************************************
2024-02-27T23:40:58.456265906-08:00 2024-02-28 07:40:58.446	I [192.145.37.58:18080 OUT] Sync data returned a new top block candidate: 1 -> 3093926 [Your node is 3093925 blocks (9.8 years) behind] 
2024-02-27T23:40:58.456326460-08:00 2024-02-28 07:40:58.446	I SYNCHRONIZATION started
2024-02-27T23:41:00.150900175-08:00 2024-02-28 07:41:00.150	I Synced 101/3093926 (0%, 3093825 left)
2024-02-27T23:41:00.745322466-08:00 2024-02-28 07:41:00.745	I Synced 201/3093926 (0%, 3093725 left)
2024-02-27T23:41:00.783775520-08:00 2024-02-28 07:41:00.782	I Synced 301/3093926 (0%, 3093625 left)
2024-02-27T23:41:01.136203255-08:00 2024-02-28 07:41:01.136	I Synced 401/3093926 (0%, 3093525 left)
2024-02-27T23:41:01.197686044-08:00 2024-02-28 07:41:01.197	I Synced 501/3093926 (0%, 3093425 left)
2024-02-27T23:41:01.371887620-08:00 2024-02-28 07:41:01.371	I Synced 601/3093926 (0%, 3093325 left)
2024-02-27T23:41:01.595364668-08:00 2024-02-28 07:41:01.595	I Synced 701/3093926 (0%, 3093225 left)
2024-02-27T23:41:01.754827809-08:00 2024-02-28 07:41:01.754	I Synced 801/3093926 (0%, 3093125 left)
2024-02-27T23:41:01.845253557-08:00 2024-02-28 07:41:01.845	I Synced 901/3093926 (0%, 3093025 left)
2024-02-27T23:41:02.039256390-08:00 2024-02-28 07:41:02.039	I Synced 1001/3093926 (0%, 3092925 left)
2024-02-27T23:41:02.125859464-08:00 2024-02-28 07:41:02.125	I Synced 1101/3093926 (0%, 3092825 left)
2024-02-27T23:41:02.303082550-08:00 2024-02-28 07:41:02.302	I Synced 1201/3093926 (0%, 3092725 left)
2024-02-27T23:41:02.605310500-08:00 2024-02-28 07:41:02.605	I Synced 1301/3093926 (0%, 3092625 left)
2024-02-27T23:41:02.689265966-08:00 2024-02-28 07:41:02.687	I Synced 1401/3093926 (0%, 3092525 left)
2024-02-27T23:41:02.815945070-08:00 2024-02-28 07:41:02.815	I Synced 1501/3093926 (0%, 3092425 left)
2024-02-27T23:41:02.899519381-08:00 2024-02-28 07:41:02.899	I Synced 1601/3093926 (0%, 3092325 left)
2024-02-27T23:41:03.941338579-08:00 2024-02-28 07:41:03.941	I Synced 1701/3093926 (0%, 3092225 left)
2024-02-27T23:41:04.044335536-08:00 2024-02-28 07:41:04.043	I Synced 1801/3093926 (0%, 3092125 left)
2024-02-27T23:41:04.141436128-08:00 2024-02-28 07:41:04.141	I Synced 1901/3093926 (0%, 3092025 left)
2024-02-27T23:41:04.235216871-08:00 2024-02-28 07:41:04.235	I Synced 2001/3093926 (0%, 3091925 left)
2024-02-27T23:41:04.352567301-08:00 2024-02-28 07:41:04.351	I Synced 2101/3093926 (0%, 3091825 left)
2024-02-27T23:41:04.444570107-08:00 2024-02-28 07:41:04.444	I Synced 2201/3093926 (0%, 3091725 left)
2024-02-27T23:41:04.544649379-08:00 2024-02-28 07:41:04.544	I Synced 2301/3093926 (0%, 3091625 left)
2024-02-27T23:41:04.652189050-08:00 2024-02-28 07:41:04.649	I Synced 2401/3093926 (0%, 3091525 left)
2024-02-27T23:41:04.752131585-08:00 2024-02-28 07:41:04.752	I Synced 2501/3093926 (0%, 3091425 left)
2024-02-27T23:41:04.846306136-08:00 2024-02-28 07:41:04.845	I Synced 2601/3093926 (0%, 3091325 left)
2024-02-27T23:41:04.951703437-08:00 2024-02-28 07:41:04.951	I Synced 2701/3093926 (0%, 3091225 left)
2024-02-27T23:41:05.053748717-08:00 2024-02-28 07:41:05.053	I Synced 2801/3093926 (0%, 3091125 left)
2024-02-27T23:41:05.156228703-08:00 2024-02-28 07:41:05.155	I Synced 2901/3093926 (0%, 3091025 left)
2024-02-27T23:41:05.433976998-08:00 2024-02-28 07:41:05.433	I Synced 3001/3093926 (0%, 3090925 left)
2024-02-27T23:41:05.541911820-08:00 2024-02-28 07:41:05.541	I Synced 3101/3093926 (0%, 3090825 left)
2024-02-27T23:41:05.670332352-08:00 2024-02-28 07:41:05.670	I Synced 3201/3093926 (0%, 3090725 left)
2024-02-27T23:41:05.828774126-08:00 2024-02-28 07:41:05.828	I Synced 3301/3093926 (0%, 3090625 left)
2024-02-27T23:41:06.233637203-08:00 2024-02-28 07:41:06.233	I Synced 3401/3093926 (0%, 3090525 left)
2024-02-27T23:41:06.574960271-08:00 2024-02-28 07:41:06.574	I Synced 3501/3093926 (0%, 3090425 left)
2024-02-27T23:41:06.692207787-08:00 2024-02-28 07:41:06.690	I Synced 3601/3093926 (0%, 3090325 left)
2024-02-27T23:41:06.793974346-08:00 2024-02-28 07:41:06.793	I Synced 3701/3093926 (0%, 3090225 left)
2024-02-27T23:41:06.898013036-08:00 2024-02-28 07:41:06.897	I Synced 3801/3093926 (0%, 3090125 left)
2024-02-27T23:41:07.007622220-08:00 2024-02-28 07:41:07.007	I Synced 3901/3093926 (0%, 3090025 left)
2024-02-27T23:41:07.119911202-08:00 2024-02-28 07:41:07.119	I Synced 4001/3093926 (0%, 3089925 left)
2024-02-27T23:41:07.223060514-08:00 2024-02-28 07:41:07.222	I Synced 4101/3093926 (0%, 3089825 left)
2024-02-27T23:41:07.325659233-08:00 2024-02-28 07:41:07.325	I Synced 4201/3093926 (0%, 3089725 left)
2024-02-27T23:41:07.451453448-08:00 2024-02-28 07:41:07.451	I Synced 4301/3093926 (0%, 3089625 left)
2024-02-27T23:41:07.569552933-08:00 2024-02-28 07:41:07.569	I Synced 4401/3093926 (0%, 3089525 left)

2024-02-27 23:51:44.496994-08:002024-02-28 07:51:44.496	I Synced 185788/3093930 (6%, 2908142 left)
2024-02-27 23:51:44.737436-08:002024-02-28 07:51:44.733	I Synced 185888/3093930 (6%, 2908042 left)
2024-02-27 23:51:45.059368-08:002024-02-28 07:51:45.035	I Synced 185988/3093930 (6%, 2907942 left)
2024-02-27 23:51:45.280243-08:002024-02-28 07:51:45.277	I Synced 186088/3093930 (6%, 2907842 left)
2024-02-27 23:51:45.577486-08:002024-02-28 07:51:45.574	I Synced 186188/3093930 (6%, 2907742 left)
2024-02-27 23:51:45.577524-08:002024-02-28 07:51:45.575	I [batch] DB resize needed
2024-02-27 23:51:45.667148-08:002024-02-28 07:51:45.666	I LMDB Mapsize increased.  Old: 3296MiB, New: 3907MiB
2024-02-27 23:51:46.215326-08:002024-02-28 07:51:46.210	I Synced 186288/3093930 (6%, 2907642 left)
2024-02-27 23:51:51.659769-08:002024-02-28 07:51:51.659	I p2p net loop stopped
2024-02-27 23:51:51.673136-08:002024-02-28 07:51:51.672	I Stopping core RPC server...
2024-02-27 23:51:51.673375-08:002024-02-28 07:51:51.673	I Stopping restricted RPC server...
2024-02-27 23:51:51.673410-08:002024-02-28 07:51:51.673	I Node stopped.
2024-02-27 23:51:51.673455-08:002024-02-28 07:51:51.673	I Deinitializing core RPC server...
2024-02-27 23:51:51.675759-08:002024-02-28 07:51:51.675	I Deinitializing restricted RPC server...
2024-02-27 23:51:51.679233-08:002024-02-28 07:51:51.677	I Deinitializing p2p...
2024-02-27 23:51:51.690896-08:002024-02-28 07:51:51.690	I Deinitializing core...
2024-02-27 23:51:52.409958-08:002024-02-28 07:51:52.409	I Stopping cryptonote protocol...


After stopping & restarting app:

2024-02-27T23:54:04.131568408-08:00 2024-02-28 07:54:04.122	I Monero 'Fluorine Fermi' (v0.18.3.1-release)
2024-02-27T23:54:04.131640964-08:00 2024-02-28 07:54:04.122	I Initializing cryptonote protocol...
2024-02-27T23:54:04.131649170-08:00 2024-02-28 07:54:04.122	I Cryptonote protocol initialized OK
2024-02-27T23:54:04.131655842-08:00 2024-02-28 07:54:04.123	I Initializing core...
2024-02-27T23:54:04.131662254-08:00 2024-02-28 07:54:04.123	I Loading blockchain from folder /root/.bitmonero/lmdb ...
2024-02-27T23:54:04.199461286-08:00 2024-02-28 07:54:04.198	I Loading checkpoints
2024-02-27T23:54:04.199499087-08:00 2024-02-28 07:54:04.198	I Pruning blockchain...
2024-02-27T23:54:04.199506892-08:00 2024-02-28 07:54:04.198	I Core initialized OK
2024-02-27T23:54:04.199513975-08:00 2024-02-28 07:54:04.198	I Initializing p2p server...
2024-02-27T23:54:04.200165479-08:00 2024-02-28 07:54:04.199	I p2p server initialized OK
2024-02-27T23:54:04.200178042-08:00 2024-02-28 07:54:04.200	I Initializing core RPC server...
2024-02-27T23:54:04.200508022-08:00 2024-02-28 07:54:04.200	I Binding on 127.0.0.1 (IPv4):18081
2024-02-27T23:54:04.596221101-08:00 2024-02-28 07:54:04.590	I core RPC server initialized OK on port: 18081
2024-02-27T23:54:04.596268931-08:00 2024-02-28 07:54:04.590	I Initializing restricted RPC server...
2024-02-27T23:54:04.596276155-08:00 2024-02-28 07:54:04.590	I Binding on 0.0.0.0 (IPv4):18089
2024-02-27T23:54:05.259887547-08:00 2024-02-28 07:54:05.247	I restricted RPC server initialized OK on port: 18089
2024-02-27T23:54:05.259935136-08:00 2024-02-28 07:54:05.251	I Starting core RPC server...
2024-02-27T23:54:05.259942490-08:00 2024-02-28 07:54:05.251	I core RPC server started ok
2024-02-27T23:54:05.259949283-08:00 2024-02-28 07:54:05.251	I Starting restricted RPC server...
2024-02-27T23:54:05.259955394-08:00 2024-02-28 07:54:05.251	I restricted RPC server started ok
2024-02-27T23:54:05.259974961-08:00 2024-02-28 07:54:05.251	I Public RPC port 18089 will be advertised to other peers over P2P
2024-02-27T23:54:05.259984569-08:00 2024-02-28 07:54:05.251	I Starting p2p net loop...
2024-02-27T23:54:06.262351480-08:00 2024-02-28 07:54:06.262	I 
2024-02-27T23:54:06.262389973-08:00 2024-02-28 07:54:06.262	I **********************************************************************
2024-02-27T23:54:06.262401464-08:00 2024-02-28 07:54:06.262	I The daemon will start synchronizing with the network. This may take a long time to complete.
2024-02-27T23:54:06.262464883-08:00 2024-02-28 07:54:06.262	I 
2024-02-27T23:54:06.262473890-08:00 2024-02-28 07:54:06.262	I You can set the level of process detailization through "set_log <level|categories>" command,
2024-02-27T23:54:06.262480723-08:00 2024-02-28 07:54:06.262	I where <level> is between 0 (no details) and 4 (very verbose), or custom category based levels (eg, *:WARNING).
2024-02-27T23:54:06.262491874-08:00 2024-02-28 07:54:06.262	I 
2024-02-27T23:54:06.262497965-08:00 2024-02-28 07:54:06.262	I Use the "help" command to see the list of available commands.
2024-02-27T23:54:06.262504488-08:00 2024-02-28 07:54:06.262	I Use "help <command>" to see a command's documentation.
2024-02-27T23:54:06.262510178-08:00 2024-02-28 07:54:06.262	I **********************************************************************
2024-02-27T23:54:07.852221355-08:00 2024-02-28 07:54:07.850	I [141.95.72.61:18080 OUT] Sync data returned a new top block candidate: 1 -> 3093931 [Your node is 3093930 blocks (9.8 years) behind] 
2024-02-27T23:54:07.852275817-08:00 2024-02-28 07:54:07.850	I SYNCHRONIZATION started
2024-02-27T23:54:08.880665024-08:00 2024-02-28 07:54:08.878	I Synced 101/3093931 (0%, 3093830 left)
2024-02-27T23:54:09.073359187-08:00 2024-02-28 07:54:09.073	I Synced 201/3093931 (0%, 3093730 left)
2024-02-27T23:54:09.287907237-08:00 2024-02-28 07:54:09.287	I Synced 301/3093931 (0%, 3093630 left)
2024-02-27T23:54:09.510416159-08:00 2024-02-28 07:54:09.508	I Synced 401/3093931 (0%, 3093530 left)
2024-02-27T23:54:09.643565183-08:00 2024-02-28 07:54:09.635	I Synced 501/3093931 (0%, 3093430 left)
2024-02-27T23:54:09.723153448-08:00 2024-02-28 07:54:09.712	I Synced 601/3093931 (0%, 3093330 left)
2024-02-27T23:54:09.789351062-08:00 2024-02-28 07:54:09.789	I Synced 701/3093931 (0%, 3093230 left)
2024-02-27T23:54:09.983763372-08:00 2024-02-28 07:54:09.983	I Synced 801/3093931 (0%, 3093130 left)
2024-02-27T23:54:10.108236170-08:00 2024-02-28 07:54:10.105	I Synced 901/3093931 (0%, 3093030 left)
2024-02-27T23:54:10.210174864-08:00 2024-02-28 07:54:10.206	I Synced 1001/3093931 (0%, 3092930 left)
2024-02-27T23:54:10.312225508-08:00 2024-02-28 07:54:10.307	I Synced 1101/3093931 (0%, 3092830 left)
2024-02-27T23:54:10.407315509-08:00 2024-02-28 07:54:10.406	I Synced 1201/3093931 (0%, 3092730 left)
2024-02-27T23:54:10.507801053-08:00 2024-02-28 07:54:10.506	I Synced 1301/3093931 (0%, 3092630 left)
2024-02-27T23:54:10.616234556-08:00 2024-02-28 07:54:10.614	I Synced 1401/3093931 (0%, 3092530 left)
2024-02-27T23:54:10.716231693-08:00 2024-02-28 07:54:10.713	I Synced 1501/3093931 (0%, 3092430 left)
2024-02-27T23:54:10.813820406-08:00 2024-02-28 07:54:10.812	I Synced 1601/3093931 (0%, 3092330 left)
2024-02-27T23:54:10.924049843-08:00 2024-02-28 07:54:10.923	I Synced 1701/3093931 (0%, 3092230 left)
2024-02-27T23:54:11.038077402-08:00 2024-02-28 07:54:11.032	I Synced 1801/3093931 (0%, 3092130 left)
2024-02-27T23:54:11.137714333-08:00 2024-02-28 07:54:11.137	I Synced 1901/3093931 (0%, 3092030 left)
2024-02-27T23:54:11.259993561-08:00 2024-02-28 07:54:11.259	I Synced 2001/3093931 (0%, 3091930 left)
2024-02-27T23:54:11.408204574-08:00 2024-02-28 07:54:11.404	I Synced 2101/3093931 (0%, 3091830 left)
2024-02-27T23:54:11.512372325-08:00 2024-02-28 07:54:11.511	I Synced 2201/3093931 (0%, 3091730 left)
2024-02-27T23:54:11.617836210-08:00 2024-02-28 07:54:11.617	I Synced 2301/3093931 (0%, 3091630 left)
2024-02-27T23:54:11.730928142-08:00 2024-02-28 07:54:11.730	I Synced 2401/3093931 (0%, 3091530 left)
2024-02-27T23:54:11.845822890-08:00 2024-02-28 07:54:11.845	I Synced 2501/3093931 (0%, 3091430 left)
2024-02-27T23:54:11.960199485-08:00 2024-02-28 07:54:11.959	I Synced 2601/3093931 (0%, 3091330 left)
2024-02-27T23:54:12.068909197-08:00 2024-02-28 07:54:12.064	I Synced 2701/3093931 (0%, 3091230 left)
2024-02-27T23:54:12.192849094-08:00 2024-02-28 07:54:12.190	I Synced 2801/3093931 (0%, 3091130 left)
2024-02-27T23:54:12.304302078-08:00 2024-02-28 07:54:12.304	I Synced 2901/3093931 (0%, 3091030 left)
2024-02-27T23:54:12.423402425-08:00 2024-02-28 07:54:12.423	I Synced 3001/3093931 (0%, 3090930 left)
2024-02-27T23:54:12.527360091-08:00 2024-02-28 07:54:12.526	I Synced 3101/3093931 (0%, 3090830 left)
2024-02-27T23:54:12.640722020-08:00 2024-02-28 07:54:12.640	I Synced 3201/3093931 (0%, 3090730 left)
2024-02-27T23:54:12.754153801-08:00 2024-02-28 07:54:12.754	I Synced 3301/3093931 (0%, 3090630 left)
2024-02-27T23:54:12.868589708-08:00 2024-02-28 07:54:12.868	I Synced 3401/3093931 (0%, 3090530 left)
2024-02-27T23:54:12.977454791-08:00 2024-02-28 07:54:12.977	I Synced 3501/3093931 (0%, 3090430 left)

Application Configuration

Everything left to default settings. I also tried specifying host storage location, no different results

Describe the bug

When restarting the app after being stopped it starts downloading and syncing the blockchain from the beginning again.

To Reproduce

Start app, view logs to see blockchain sync progress
Stop app
Start app, view logs to see blockchain sync has started from 0 again.

Expected Behavior

Blockchain should be saved to lmdb and when node starts it continues syncing from where it left off last.

Screenshots

n/a

Additional Context

n/a

I've read and agree with the following

• I've checked all open and closed issues and my issue is not there.