evm leaks memory doing 256-bit arithmetic

[gcolvin]

System information

evm version 1.6.1-stable-021c3c28

solc, the solidity compiler commandline interface
Version: 0.4.11-develop.2017.5.3+commit.34b28ed7.Darwin.appleclang

macOS 10.12.4

Expected behaviour

Program should run with stable memory usage until gas is exhausted.

Actual behaviour

Program leaks, using more memory the longer it runs, until system is thrashing.

Other programs with other operators do the same, making it impossible to benchmark them.

Steps to reproduce the behaviour

Compile appended code and run it with evm.

$ solc -o . --bin thrash.sol
$ evm --codefile thrash.bin run

pragma solidity ^0.4.0;
contract thrash {
	function thrash() {
		uint x = 3;
		while (true) x *= x;
	}
}

[obscuren]

Well it does contain an infinite loop and unfortunately 256bit arithmetic -- or rather Go's big num package -- more or less requires newly allocated objects for each operation; and by default the evm utility runs unmetered, i.e. it won't ever run OOG and trash your system :-P

Try running with --gas 1000000.

[holiman]

I'd expect the problem to be due to the intpool - it's not limited, is it ?

[obscuren]

No it isn't limited, but that doesn't matter. It wouldn't be "limited" without the pool either, i.e. it would have to allocate a new big.Int for each operation.

[gcolvin]

It still uses more memory until it runs out of gas. A leak in my book - the collector should recover the memory. I'm using long loops in my tests to get stable timings that overwhelm the startup costs.

[obscuren]

The problem is the while (true) and the fact that the evm runs unmetered.

Maybe I'm not understanding the problem, does it, or does it not go oog?

[gcolvin]

I have a 16 GB machine , and Sierra killed it when the memory image hit 62 GB.

[obscuren]

Sorry, my bad it does run metered. I was under the impression we were running unmetered which was causing your issues. I'll look in to this.

[gcolvin]

Thanks.

[holiman]

Here's what happens, a snippet of execution:

Intpool size 6993{"Pc":26,"Op":128,"Gas":9999890371,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6995{"Pc":27,"Op":129,"Gas":9999890368,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6995{"Pc":28,"Op":2,"Gas":9999890363,"GasCost":5,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6995{"Pc":29,"Op":144,"Gas":9999890360,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6996{"Pc":30,"Op":80,"Gas":9999890358,"GasCost":2,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6996{"Pc":31,"Op":96,"Gas":9999890355,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6997{"Pc":33,"Op":86,"Gas":9999890347,"GasCost":8,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,19],"Storage":null,"Depth":1,"Err":null}
Intpool size 6996{"Pc":19,"Op":91,"Gas":9999890346,"GasCost":1,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6997{"Pc":20,"Op":96,"Gas":9999890343,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6997{"Pc":22,"Op":21,"Gas":9999890340,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6996{"Pc":23,"Op":96,"Gas":9999890337,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,0],"Storage":null,"Depth":1,"Err":null}
Intpool size 6997{"Pc":25,"Op":87,"Gas":9999890327,"GasCost":10,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,0,34],"Storage":null,"Depth":1,"Err":null}
Intpool size 6996{"Pc":26,"Op":128,"Gas":9999890324,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6998{"Pc":27,"Op":129,"Gas":9999890321,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6998{"Pc":28,"Op":2,"Gas":9999890316,"GasCost":5,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6998{"Pc":29,"Op":144,"Gas":9999890313,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6999{"Pc":30,"Op":80,"Gas":9999890311,"GasCost":2,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6999{"Pc":31,"Op":96,"Gas":9999890308,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 7000{"Pc":33,"Op":86,"Gas":9999890300,"GasCost":8,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,19],"Storage":null,"Depth":1,"Err":null}
Intpool size 6999{"Pc":19,"Op":91,"Gas":9999890299,"GasCost":1,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 7000{"Pc":20,"Op":96,"Gas":9999890296,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size 7000{"Pc":22,"Op":21,"Gas":9999890293,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
Intpool size 6999{"Pc":23,"Op":96,"Gas":9999890290,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,0],"Storage":null,"Depth":1,"Err":null}
Intpool size 7000{"Pc":25,"Op":87,"Gas":9999890280,"GasCost":10,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,0,34],"Storage":null,"Depth":1,"Err":null}
Intpool size 6999{"Pc":26,"Op":128,"Gas":9999890277,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
Intpool size ...

The ops are skewed, they instantiate bigints and put them on the pool for reuse - but there are not enough reuse going on within these particular ops in the loop. These ops specifically:

Intpool size 68090
{"Pc":29,"Op":144,"Gas":9998933205,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
+1 Intpool size 68091
{"Pc":30,"Op":80,"Gas":9998933203,"GasCost":2,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,1],"Storage":null,"Depth":1,"Err":null}
+1 Intpool size 68092
{"Pc":33,"Op":86,"Gas":9998933192,"GasCost":8,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,19],"Storage":null,"Depth":1,"Err":null}
-1 Intpool size 68091
{"Pc":19,"Op":91,"Gas":9998933191,"GasCost":1,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
+1 Intpool size 68092
{"Pc":20,"Op":96,"Gas":9998933188,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
-1 Intpool size 68091
{"Pc":23,"Op":96,"Gas":9998933182,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,0],"Storage":null,"Depth":1,"Err":null}
+1 Intpool size 68092
{"Pc":25,"Op":87,"Gas":9998933172,"GasCost":10,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1,0,34],"Storage":null,"Depth":1,"Err":null}
-1 Intpool size 68091
{"Pc":26,"Op":128,"Gas":9998933169,"GasCost":3,"Memory":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg","Stack":[1],"Storage":null,"Depth":1,"Err":null}
+2 Intpool size 68093

So we should limit the intpool. Each loop adds 3 elements.

[obscuren]

This issue has been resolved in #14336

[gcolvin]

Giving the program 1E9 units of gas, and running on Xubuntu on an old Dell, I now get these numbers from top and time.

            res memory  user time
ethvm:      23388       0m37.216s
evm:        19520       1m49.632s
parity-evm:  8916       4m59.576s

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.