panic: division by zero in settlement code

[roman-khimov]

Expected Behavior

No panics.

Current Behavior

Feb 16 19:31:42 node2 neofs-ir[4736]: 2023/02/16 19:31:42 worker exits from a panic: division by zero
Feb 16 19:31:42 node2 neofs-ir[4736]: 2023/02/16 19:31:42 worker exits from panic: goroutine 148390 [running]:
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/panjf2000/ants/v2.(*goWorker).run.func1.1()
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/panjf2000/ants/v2@v2.4.0/worker.go:58 +0x10c
Feb 16 19:31:42 node2 neofs-ir[4736]: panic({0xd0c800, 0xfb16a0})
Feb 16 19:31:42 node2 neofs-ir[4736]:         runtime/panic.go:1038 +0x215
Feb 16 19:31:42 node2 neofs-ir[4736]: math/big.nat.div({0x0, 0xfe3c28, 0xc00009d080}, {0x0, 0xc0017827e0, 0xc001405da0}, {0x0, 0xc0017827a0, 0xc00041db10}, {0x0, ...})
Feb 16 19:31:42 node2 neofs-ir[4736]:         math/big/natdiv.go:507 +0x348
Feb 16 19:31:42 node2 neofs-ir[4736]: math/big.(*Int).QuoRem(0xc00026da40, 0xc00026da40, 0xc00026da20, 0xc00041db58)
Feb 16 19:31:42 node2 neofs-ir[4736]:         math/big/int.go:239 +0x9c
Feb 16 19:31:42 node2 neofs-ir[4736]: math/big.(*Int).Div(0xc00026da40, 0xc00026da40, 0xc00026d9e0)
Feb 16 19:31:42 node2 neofs-ir[4736]:         math/big/int.go:250 +0x45
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.normalizedValue(0xc0016e0b90, 0xfd2900, 0xc001405e48)
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/distribute.go:53 +0x7b
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.(*IncomeSettlementContext).Distribute.func1({0xc001754240, 0x21, 0x30}, 0x20)
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/distribute.go:40 +0x2f1
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.(*NodeSizeTable).Iterate(0xc000291b40, 0xc00041ddf0)
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/util.go:26 +0x8f
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.(*IncomeSettlementContext).Distribute(0xc0003ecdc0)
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/distribute.go:27 +0x2ee
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement.(*Processor).HandleIncomeDistributionEvent.func1()
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/calls.go:117 +0x1d
Feb 16 19:31:42 node2 neofs-ir[4736]: github.com/panjf2000/ants/v2.(*goWorker).run.func1()
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
Feb 16 19:31:42 node2 neofs-ir[4736]: created by github.com/panjf2000/ants/v2.(*goWorker).run
Feb 16 19:31:42 node2 neofs-ir[4736]:         github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x68

Steps to Reproduce (for bugs)

Unknown.

Context

T5 testnet.

Your Environment

NeoFS Inner Ring node
Version: v0.34.0 
GoVersion: go1.17.13

[vvarg229]

The bug is reproduced in the https://github.com/nspcc-dev/neofs-testcases/blob/master/pytest_tests/testsuites/services/http_gate/test_http_system_header.py#L140 test.

2023/03/17 11:36:42 worker exits from a panic: division by zero
2023/03/17 11:36:42 worker exits from panic: goroutine 10552 [running]:
github.com/panjf2000/ants/v2.(*goWorker).run.func1.1()
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:58 +0x10c
panic({0xd1a4c0, 0x10847c0})
	runtime/panic.go:1038 +0x215
math/big.nat.div({0x0, 0x10b7230, 0xc000051040}, {0x0, 0xc0018f0980, 0xc000330b40}, {0x0, 0xc0018f0940, 0xc001999b10}, {0x0, ...})
	math/big/natdiv.go:507 +0x348
math/big.(*Int).QuoRem(0xc0003321c0, 0xc0003321c0, 0xc0003321a0, 0xc001999b58)
	math/big/int.go:239 +0x9c
math/big.(*Int).Div(0xc0003321c0, 0xc0003321c0, 0xc000332160)
	math/big/int.go:250 +0x45
github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.normalizedValue(0xc00031a190, 0x10a5c98, 0xc00000cfc0)
	github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/distribute.go:53 +0x7b
github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.(*IncomeSettlementContext).Distribute.func1({0xc001f302d0, 0x21, 0x30}, 0x20)
	github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/distribute.go:40 +0x2f1
github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.(*NodeSizeTable).Iterate(0xc001634500, 0xc001999df0)
	github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/util.go:26 +0x8f
github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic.(*IncomeSettlementContext).Distribute(0xc00170c630)
	github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/basic/distribute.go:27 +0x2ee
github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement.(*Processor).HandleIncomeDistributionEvent.func1()
	github.com/nspcc-dev/neofs-node/pkg/innerring/processors/settlement/calls.go:117 +0x1d
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
created by github.com/panjf2000/ants/v2.(*goWorker).run
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x68

Logs:
b2b2ba8df1dffb27.zip

See more in allure report:
run_x.zip
(unzip and run allure serve run_x)

Test Environment:

# Basenet settings
LOCAL_DOMAIN=neofs.devenv
IPV4_PREFIX=192.168.100
CA_CERTS_TRUSTED_STORE=/etc/ssl/certs

# Bastion image
BASTION_VERSION=10
BASTION_IMAGE=debian

# NeoGo privnet
#CHAIN_PATH="/path/to/devenv.dump.gz"
CHAIN_URL="https://github.com/nspcc-dev/neofs-contract/releases/download/v0.16.0/devenv_mainchain_notary_disabled.gz"
NEOGO_VERSION=0.101.0
NEOGO_IMAGE=nspccdev/neo-go

# NeoFS InnerRing nodes
IR_VERSION=0.35.0
IR_IMAGE=nspccdev/neofs-ir

# NeoFS Storage nodes
NODE_VERSION=0.35.0
NODE_IMAGE=nspccdev/neofs-storage

# NATS Server
NATS_VERSION=2.7.2
NATS_IMAGE=nats

# HTTP Gate
HTTP_GW_VERSION=0.26.0
HTTP_GW_IMAGE=nspccdev/neofs-http-gw

# REST Gate
REST_GW_VERSION=0.5.0 
REST_GW_IMAGE=nspccdev/neofs-rest-gw

# S3 Gate
S3_GW_VERSION=0.26.1
S3_GW_IMAGE=nspccdev/neofs-s3-gw

# Coredns
#COREDNS_VERSION=v019
COREDNS_VERSION=v016
COREDNS_IMAGE=nspccdev/coredns

# NeoFS LOCODE database
LOCODE_DB_URL=https://github.com/nspcc-dev/neofs-locode-db/releases/download/v0.3.0/locode_db.gz
#LOCODE_DB_PATH=/path/to/locode_db

# NeoFS CLI binary
NEOFS_CLI_URL=https://github.com/nspcc-dev/neofs-node/releases/download/v0.35.0/neofs-cli-amd64.tar.gz
#NEOFS_CLI_PATH=/path/to/neofs-cli-binary

# NeoFS ADM tool binary
NEOFS_ADM_VERSION=v0.35.0
NEOFS_ADM_URL=https://github.com/nspcc-dev/neofs-node/releases/download/${NEOFS_ADM_VERSION}/neofs-adm-amd64.tar.gz
#NEOFS_ADM_PATH=/path/to/neofs-adm-binary

# Compiled NeoFS Smart Contracts
NEOFS_CONTRACTS_VERSION=v0.16.0
NEOFS_CONTRACTS_URL=https://github.com/nspcc-dev/neofs-contract/releases/download/${NEOFS_CONTRACTS_VERSION}/neofs-contract-${NEOFS_CONTRACTS_VERSION}.tar.gz
#NEOFS_CONTRACTS_PATH=/path/to/unpacked/neofs-contracts-dir

Linux burzum 5.19.0-31-generic #32-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 20 15:20:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[roman-khimov]

Hmm. BTW, @cthulhu-rider, maybe we've fixed this with #2283?

[cthulhu-rider]

@roman-khimov I don't think so, basic income mechanism distributes funds between storage nodes.