Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MemoryError('dropping message over 1,000,000 bytes and re-synchronizing',) #713

Closed
sumBTC opened this issue Feb 2, 2019 · 8 comments

Comments

@sumBTC
Copy link

commented Feb 2, 2019

I'm running Electrumx since a long time now and continuously monitor the number of peers. Yesterday I suddenly got strange values. Turned out:

electrumx_rpc peers

gave me

MemoryError('dropping message over 1,000,000 bytes and re-synchronizing',)

A restart didn't help. I found out the solution was to add

MAX_SEND=10000000

(default MAX_SEND=1000000)

to

/etc/electrumx.conf

Electrumx is now running again without problems.

I decided to submit this issue as it might be useful to others.

Question is: shouldn't Electrumx exit with an error message telling what went wrong and how to solve it?

Anyway, a big thank you to all the developers for all the hard work you are doing. Awesome project!

System information:

Ubuntu: 16.04
Electrumx: 1.9.1
COIN: BitcoinSegwit

More info on my website.

@kyuupichan kyuupichan closed this in c9292b6 Feb 2, 2019

@kyuupichan

This comment has been minimized.

Copy link
Owner

commented Feb 2, 2019

I believe the typo just fixed above should fix this issue; let me know...

@sumBTC

This comment has been minimized.

Copy link
Author

commented Feb 2, 2019

I removed the MAX_SEND=10000000 and the problem is gone. The only thing is: I'm not able to reproduce the issue but if there is a typo that has to be fixed anyway and if you're sure it has something to do with the issue, it's probably solved by this.

Again, thanks for all your efforts.

@shsmith

This comment has been minimized.

Copy link
Contributor

commented Feb 2, 2019

I think this is related to the ongoing Sybil attack against ElectrumX servers. Recently there have been just under 100 legit servers, but in the last 24 hours we had over 248 servers, which probably exceeded the hardcoded limit. Enough of the malicious servers have gone stale that we are back below the limit. Malicious servers are in the majority now, especially for older vulnerable clients that don't understand protocol version 1.4.

@sumBTC

This comment has been minimized.

Copy link
Author

commented Feb 2, 2019

Indeed, recently I saw about 100 servers and suddenly there were a lot more. So would you advice me to set the MAX_SEND higher (like I did before)?

SomberNight added a commit to SomberNight/electrumx that referenced this issue Feb 2, 2019

@kyuupichan

This comment has been minimized.

Copy link
Owner

commented Feb 3, 2019

My change fixed this; there is not intended to be a limit on local RPC size

@SomberNight

This comment has been minimized.

Copy link
Contributor

commented Feb 6, 2019

Please reopen. The fix is insufficient. The issue still occurs.

@SomberNight

This comment has been minimized.

Copy link
Contributor

commented Feb 6, 2019

To test, patch:

diff --git a/electrumx/lib/text.py b/electrumx/lib/text.py
index 8546eb9..24ce76e 100644
--- a/electrumx/lib/text.py
+++ b/electrumx/lib/text.py
@@ -52,6 +52,7 @@ def peers_lines(data):
     '''A generator returning lines for a list of peers.
 
     data is the return value of rpc_peers().'''
+    return data
     def time_fmt(t):
         if not t:
             return 'Never'
@@ -80,3 +81,4 @@ def peers_lines(data):
                          item['try_count'],
                          item['source'][:20],
                          item['ip_addr'] or '')
+
diff --git a/electrumx/server/peers.py b/electrumx/server/peers.py
index c9ed073..c645689 100644
--- a/electrumx/server/peers.py
+++ b/electrumx/server/peers.py
@@ -518,4 +518,4 @@ class PeerManager(object):
         def peer_key(peer):
             return (peer.bad, -peer.last_good)
 
-        return [peer_data(peer) for peer in sorted(self.peers, key=peer_key)]
+        return [{'asd': "qqq"*100} for _ in range(10000)]

and run ./electrumx_rpc peers

kyuupichan added a commit that referenced this issue Feb 7, 2019

Sombernight's fix for #713
Require aiorpcx >= 0.10.3
@kyuupichan

This comment has been minimized.

Copy link
Owner

commented Feb 7, 2019

I believe this is fixed properly thanks to @SomberNight's work on it

cculianu added a commit to cculianu/electrumx that referenced this issue Feb 7, 2019

cculianu added a commit to cculianu/electrumx that referenced this issue Feb 7, 2019

Sombernight's fix for kyuupichan#713
Require aiorpcx >= 0.10.3

markblundeberg added a commit to Electron-Cash/electrumx that referenced this issue Feb 7, 2019

markblundeberg added a commit to Electron-Cash/electrumx that referenced this issue Feb 7, 2019

Sombernight's fix for kyuupichan#713
Require aiorpcx >= 0.10.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.