Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Look into using twmemproxy. Stats is a very expensive call, and there were bugs in the past. #24

Open
sigkill opened this issue May 7, 2022 · 0 comments

Comments

@sigkill
Copy link

sigkill commented May 7, 2022

https://github.com/twitter/twemproxy

Think about setting this up, in front of your cluster.

You can still gather stats, but calling it every 15 seconds isn't a good idea. As a caching layer, there have been issues with calling the stats command in other earlier versions of memcached. Googling doesn't yield great results, but I know because I know from experience. Watching a system that takes billions of transactions per second, running the stats collector, it would impact memcached.

By running twemproxy, you can get around this issue. You'll need to modify your code to get the stats from the port that outputs them via twemproxy, and just connecting to that port outputs them in json.

I've tested out a crappy temporary setup with five hosts just now, and a latest dump of the data.txt file.

Also - the range of output when a key is found is pretty bad - the ouput to plutus.txt dumps a hell of a lot of data, vs just dumping the key that you found. I didn't find anything useful, but I loaded privatekey 1 into memcache, and modified scroo.py to only look in a specific range. https://privatekeys.pw/key/0000000000000000000000000000000000000000000000000000000000000001

Other tweaks I tried, which are unfortunately Intel only at this time, is to grab the secp256k1 module, dll and library from here: https://github.com/iceland2k14/secp256k1
It is much faster using these libraries, and switching some of the random range settings.

With these tweaks in place, memcache is more stable. Have about a decade of experience with memcached. You'll want to change the stats check to something else to get the stability, I haven't tweaked the json collection from twemproxy, but that would be the most ideal.

Additionally, not sure if useful, redis has a bloom filter mode built in. Could be useful for low memory systems that want to do a bloomfilter search the way BSGS works, but I've got no idea how that would work ... yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant