Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qa: fix p2p_invalid_messages on macOS #14812

Merged
merged 2 commits into from Nov 26, 2018

Conversation

@jamesob
Copy link
Member

@jamesob jamesob commented Nov 26, 2018

Infinite mea culpa for the number of problems with this test.

This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On @MarcoFalke's macOS test build we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.

@jamesob
Copy link
Member Author

@jamesob jamesob commented Nov 26, 2018

Fixes #14710

@MarcoFalke MarcoFalke merged commit 5a1f576 into bitcoin:master Nov 26, 2018
1 of 2 checks passed
MarcoFalke added a commit that referenced this issue Nov 26, 2018
5a1f576 qa: clean up assert_memory_usage_stable utility (James O'Beirne)
0cf1632 qa: fix p2p_invalid_messages on macOS (James O'Beirne)

Pull request description:

  Infinite mea culpa for the number of problems with this test.

  This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On [@MarcoFalke's macOS test build](https://travis-ci.org/MarcoFalke/btc_nightly) we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.

Tree-SHA512: 150a7b88080fd883c7a5d0b9ffa470f61a97c4885fccc1a06fde6260aaef15640a7c1de7e89c581b245df7807d617ec3d86775330386ec5149ad567492fc5d31
@fanquake fanquake added the Tests label Nov 26, 2018
@kallewoof
Copy link
Member

@kallewoof kallewoof commented Nov 27, 2018

Post-merge (that was quick btw, but I guess ok since it's only tests): I assume error is unrelated?

2018-11-27T07:29:34.685000Z TestFramework (INFO): Initializing test directory /var/folders/b8/znqr8hj918772gfmd875gzgdd3ypz1/T/testzl4oubll
2018-11-27T07:29:35.945000Z TestFramework (INFO): Sending a bunch of large, junk messages to test memory exhaustion. May take a bit...
2018-11-27T07:29:40.680000Z TestFramework (INFO): Waiting for node to drop junk messages.
Fatal write error on socket transport
protocol: <test_framework.mininode.P2PDataStore object at 0x10f38d240>
transport: <_SelectorSocketTransport fd=10 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 868, in write
    n = self._sock.send(data)
OSError: [Errno 41] Protocol wrong type for socket
2018-11-27T07:29:40.799000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:14387 due to [Errno 41] Protocol wrong type for socket
2018-11-27T07:29:40.899000Z TestFramework (INFO): Sending a message with incorrect size of 2
2018-11-27T07:29:40.956000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:14387 due to [Errno 54] Connection reset by peer
2018-11-27T07:29:41.061000Z TestFramework (INFO): Sending a message with incorrect size of 77
2018-11-27T07:29:41.222000Z TestFramework (INFO): Sending a message with incorrect size of 78
2018-11-27T07:29:41.384000Z TestFramework (INFO): Sending a message with incorrect size of 79
2018-11-27T07:29:41.761000Z TestFramework (INFO): Stopping nodes
2018-11-27T07:29:42.346000Z TestFramework (INFO): Cleaning up /var/folders/b8/znqr8hj918772gfmd875gzgdd3ypz1/T/testzl4oubll on exit
2018-11-27T07:29:42.346000Z TestFramework (INFO): Tests successful

@AkioNak
Copy link
Contributor

@AkioNak AkioNak commented Nov 27, 2018

I met the error that @kallewoof pointed out above, too.
(macbook pro/macos 10.13.6)

p2p_invalid_messages.py says Tests successful, but test_runner.py judges that it Failed.

$ ./test/functional/test_runner.py p2p_invalid_messages.py 
Temporary test directory at /var/folders/gy/zxy8j2k17yxfpdbs_td03mwjzvzqs2/T/test_runner_₿_🏃_20181127_164919
1/1 - p2p_invalid_messages.py failed, Duration: 6 s

stdout:
2018-11-27T07:49:20.102000Z TestFramework (INFO): Initializing test directory /var/folders/gy/zxy8j2k17yxfpdbs_td03mwjzvzqs2/T/test_runner_₿_🏃_20181127_164919/p2p_invalid_messages_0
2018-11-27T07:49:21.079000Z TestFramework (INFO): Sending a bunch of large, junk messages to test memory exhaustion. May take a bit...
2018-11-27T07:49:24.885000Z TestFramework (INFO): Waiting for node to drop junk messages.
2018-11-27T07:49:24.989000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:11000 due to [Errno 41] Protocol wrong type for socket
2018-11-27T07:49:25.098000Z TestFramework (INFO): Sending a message with incorrect size of 2
2018-11-27T07:49:25.153000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:11000 due to [Errno 54] Connection reset by peer
2018-11-27T07:49:25.258000Z TestFramework (INFO): Sending a message with incorrect size of 77
2018-11-27T07:49:25.314000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:11000 due to [Errno 54] Connection reset by peer
2018-11-27T07:49:25.420000Z TestFramework (INFO): Sending a message with incorrect size of 78
2018-11-27T07:49:25.579000Z TestFramework (INFO): Sending a message with incorrect size of 79
2018-11-27T07:49:25.960000Z TestFramework (INFO): Stopping nodes
2018-11-27T07:49:26.446000Z TestFramework (INFO): Cleaning up /var/folders/gy/zxy8j2k17yxfpdbs_td03mwjzvzqs2/T/test_runner_₿_🏃_20181127_164919/p2p_invalid_messages_0 on exit
2018-11-27T07:49:26.446000Z TestFramework (INFO): Tests successful


stderr:
Fatal write error on socket transport
protocol: <test_framework.mininode.P2PDataStore object at 0x10ae10a58>
transport: <_SelectorSocketTransport fd=10 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py", line 868, in write
    n = self._sock.send(data)
OSError: [Errno 41] Protocol wrong type for socket



TEST                    | STATUS    | DURATION

p2p_invalid_messages.py | ✖ Failed  | 6 s

ALL                     | ✖ Failed  | 6 s (accumulated) 
Runtime: 6 s

@MarcoFalke
Copy link
Member

@MarcoFalke MarcoFalke commented Nov 27, 2018

Is that a deterministic failure due to one of the commits in this pull request? I doubt it, since I have seen it before, but please open a new issue about this.

@MarcoFalke
Copy link
Member

@MarcoFalke MarcoFalke commented Nov 27, 2018

According to some other reports asyncio has issues with large buffers on macOS...

@kallewoof
Copy link
Member

@kallewoof kallewoof commented Nov 28, 2018

Yeah this could be an error that would happen if the original error did not (i.e. the situation post-merge).

random-zebra added a commit to PIVX-Project/PIVX that referenced this issue Jun 28, 2021
…t framework update)

7b04c0a [Test] Clean duplicate connections creation in mining_pos_coldStaking.py (furszy)
15a799e [Test] MAX_PROTOCOL_MESSAGE_LENGTH PIVXified. (furszy)
0aedf35 [tests] Don't import asyncio to test magic bytes (John Newbery)
9bb5309 Refactor resource exhaustion test (Troy Giorshev)
589a780 Fix "invalid message size" test (Troy Giorshev)
8a0c12b Move size limits to module-global (Troy Giorshev)
b3391c5 Remove two unneeded tests (Troy Giorshev)
1404e68 test: Add various low-level p2p tests (furszy)
8aaf7e1 test: Remove fragile assert_memory_usage_stable (MarcoFalke)
f68e22c [Test] p2p_invalid_messages.py do not change msg.command in test_command and raise sync_with_ping timeout (furszy)
c851c0b Test framework: Wait for verack by default on every new connection (furszy)
c02e9a0 [Test] move framework subversion to string (furszy)
3472a39 Replace coroutine with async def in p2p_invalid_messages.py (Elichai Turkel)
33c7b19 net_processing: align msg checksum check properly. (furszy)
7f71b1b Hash P2P messages as they are received instead of at process-time (furszy)
215a638 test: Skip flaky p2p_invalid_messages test on macOS (Fabian Jahr)
c11f565 qa: Make swap_magic_bytes in p2p_invalid_messages atomic (MarcoFalke)
be979ad test: Fix race in p2p_invalid_messages (MarcoFalke)
6a72f0c qa: Add tests for invalid message headers (MarcoFalke)
51ddd3d Introduce p2p_invalid_messages.py test (furszy)
55a37b5 net: fix missing jump line in "Oversized message from peer.." error. (furszy)
0edfce5 test_node: get_mem_rss fixups (MarcoFalke)
6f21213 tests: add P2PConnection.send_raw_message (James O'Beirne)
ae68c6e tests: add utility to assert node memory usage hasn't increased (James O'Beirne)
8469afe test: forward timeouts properly in send_blocks_and_test (James O'Beirne)
db28a53 Skip is_closing() check when not available. (Daniel Kraft)
be9dacb tests: fixes mininode's P2PConnection sending messages on closing transport (marcoagner)
53599c8 qa: Avoid start/stop of the network thread mid-test (furszy)
688190c [qa] mininode: Expose connection state through is_connected (MarcoFalke)

Pull request description:

  Part of the deep and long net and ser work that I'm doing (and Tor v3 onion addresses support). Friend of #2359.

  Focused on the end goal of implementing the `p2p_invalid_messages` functional test which validates that invalid msg headers, msg types, oversized payloads and inventory msgs aren't accepted nor can cause a resource exhaustion. And an extra covered scenario, in `p2p_invalid_tx.py`, for the orphan pool overflow.

  Plus, to get up to the goal, had to work over the functional test framework as well.

  So.. adapted list:

  * bitcoin#9045.
  * bitcoin#13512.
  * bitcoin#13517.
  * bitcoin#13715.
  * bitcoin#13747.
  * bitcoin#14456.
  * bitcoin#14522.
  * bitcoin#14672.
  * bitcoin#14693.
  * bitcoin#14812.
  * bitcoin#15246.
  * bitcoin#15330.
  * bitcoin#15697.
  * bitcoin#16445.
  * bitcoin#17931.
  * bitcoin#17469.
  * bitcoin#18628 (only `p2p_invalid_tx.py` and `p2p_invalid_messages.py`. We don't support the other tests yet).
  * bitcoin#19177.
  * bitcoin#19264.

ACKs for top commit:
  random-zebra:
    utACK 7b04c0a and merging...

Tree-SHA512: 48d1f1a6acc24a9abce033fbf1f281ba4392147da39d118a694a09d63c3e0610cc1a29d711b434b16cc0d0e030dd9f1a89843870091b6999b862b9ab18a20679
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
knst added a commit to knst/dash that referenced this issue Aug 10, 2021
5a1f576 qa: clean up assert_memory_usage_stable utility (James O'Beirne)
0cf1632 qa: fix p2p_invalid_messages on macOS (James O'Beirne)

Pull request description:

  Infinite mea culpa for the number of problems with this test.

  This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On [@MarcoFalke's macOS test build](https://travis-ci.org/MarcoFalke/btc_nightly) we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.

Tree-SHA512: 150a7b88080fd883c7a5d0b9ffa470f61a97c4885fccc1a06fde6260aaef15640a7c1de7e89c581b245df7807d617ec3d86775330386ec5149ad567492fc5d31
knst added a commit to knst/dash that referenced this issue Aug 10, 2021
5a1f576 qa: clean up assert_memory_usage_stable utility (James O'Beirne)
0cf1632 qa: fix p2p_invalid_messages on macOS (James O'Beirne)

Pull request description:

  Infinite mea culpa for the number of problems with this test.

  This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On [@MarcoFalke's macOS test build](https://travis-ci.org/MarcoFalke/btc_nightly) we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.

Tree-SHA512: 150a7b88080fd883c7a5d0b9ffa470f61a97c4885fccc1a06fde6260aaef15640a7c1de7e89c581b245df7807d617ec3d86775330386ec5149ad567492fc5d31
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 12, 2021
knst added a commit to knst/dash that referenced this issue Aug 16, 2021
5a1f576 qa: clean up assert_memory_usage_stable utility (James O'Beirne)
0cf1632 qa: fix p2p_invalid_messages on macOS (James O'Beirne)

Pull request description:

  Infinite mea culpa for the number of problems with this test.

  This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On [@MarcoFalke's macOS test build](https://travis-ci.org/MarcoFalke/btc_nightly) we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.

Tree-SHA512: 150a7b88080fd883c7a5d0b9ffa470f61a97c4885fccc1a06fde6260aaef15640a7c1de7e89c581b245df7807d617ec3d86775330386ec5149ad567492fc5d31
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 22, 2021
PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this issue Aug 23, 2021
5a1f576 qa: clean up assert_memory_usage_stable utility (James O'Beirne)
0cf1632 qa: fix p2p_invalid_messages on macOS (James O'Beirne)

Pull request description:

  Infinite mea culpa for the number of problems with this test.

  This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On [@MarcoFalke's macOS test build](https://travis-ci.org/MarcoFalke/btc_nightly) we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.

Tree-SHA512: 150a7b88080fd883c7a5d0b9ffa470f61a97c4885fccc1a06fde6260aaef15640a7c1de7e89c581b245df7807d617ec3d86775330386ec5149ad567492fc5d31
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants