Cached object is lost when node is power cycled

[ikovalev1]

Expected Behavior

objects nust not be lost any way

Current Behavior

if node is power cycled, it loses cached object

Steps to Reproduce (for bugs)

1. create a file whose size is less than writecache max_object_size: (e.g. 5M)

dd if=/dev/random of=smallfile bs=1M count=5

2. create container with REP 1 on node1

neofs-cli container create -r node1:8080 -w /etc/neofs/storage/wallet.json --policy "REP 1 IN X CBF 1 SELECT 1 FROM n1 AS X FILTER Node EQ node1 AS n1" --await --basic-acl public-read-write
Enter password >
container ID: FrQfJYeGyeaNyT3tdHzLDYpn4ZHXVdkxszDLacLRT2k8
awaiting...
container has been persisted on sidechain

3. put the file in the container

neofs-cli object put -r node1:8080 -w /etc/neofs/storage/wallet.json  --file smallfile --cid FrQfJYeGyeaNyT3tdHzLDYpn4ZHXVdkxszDLacLRT2k8
Enter password >
 5242880 / 5242880 [===================================================================================================================================================] 100.00% 0s
[smallfile] Object successfully stored
  OID: 6THEa9GpLgKR1hxvKwLFUAk3PMMcbVutPicCbwZMsQF1
  CID: FrQfJYeGyeaNyT3tdHzLDYpn4ZHXVdkxszDLacLRT2k8

4 power off the node1

echo 1 > /proc/sys/kernel/sysrq; echo o > /proc/sysrq-trigger 

5. power on the node (if it is sbercloudm use job Pipeline sbercloud_power_manage)
6. try to get the object -not found

neofs-cli object get -r node1:8080 -w /etc/neofs/storage/wallet.json --cid FrQfJYeGyeaNyT3tdHzLDYpn4ZHXVdkxszDLacLRT2k8 --oid 6THEa9GpLgKR1hxvKwLFUAk3PMMcbVutPicCbwZMsQF1 > /tmp/kia
Enter password >
rpc error: read object header: status: code = 2049 message = object not found

Your Environment

/home/service# neofs-node --version
NeoFS Storage node
Version: v0.33.0-78-g846ff515
GoVersion: go1.18.4

[fyrchik]

Let's make SYNC writes the default and allow to opt-out with no_sync option.