Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flaky test test_send_remote_failover_sync_non_transport_exception_error #1988

Open
xrmx opened this issue Mar 6, 2024 · 1 comment
Open
Labels
agent-python community Issues opened by the community triage Issues awaiting triage

Comments

@xrmx
Copy link
Member

xrmx commented Mar 6, 2024

Describe the bug:

Cannot reproduce reliably still may be useful to fix:

=================================== FAILURES ===================================
_________ test_send_remote_failover_sync_non_transport_exception_error _________

should_try = <MagicMock name='should_try' id='140274647588368'>
http_send = <MagicMock name='send' id='140274647596096'>
caplog = <_pytest.logging.LogCaptureFixture object at 0x7f943c8a5c90>

    @mock.patch("elasticapm.transport.http.Transport.send")
    @mock.patch("elasticapm.transport.base.TransportState.should_try")
    def test_send_remote_failover_sync_non_transport_exception_error(should_try, http_send, caplog):
        should_try.return_value = True

        client = Client(
            server_url="http://example.com",
            service_name="app_name",
            secret_token="secret",
            transport_class="elasticapm.transport.http.Transport",
            metrics_interval="0ms",
            metrics_sets=[],
        )
        # test error
        http_send.side_effect = ValueError("oopsie")
        with caplog.at_level("ERROR", "elasticapm.transport"):
            client.capture_message("foo", handled=False)
        client._transport.flush()
        assert client._transport.state.did_fail()
        assert_any_record_contains(caplog.records, "oopsie", "elasticapm.transport")

        # test recovery
        http_send.side_effect = None
        client.capture_message("foo", handled=False)
        client.close()
>       assert not client._transport.state.did_fail()
E       assert not True
E        +  where True = <bound method TransportState.did_fail of <elasticapm.transport.base.TransportState object at 0x7f943c8ddb40>>()
E        +    where <bound method TransportState.did_fail of <elasticapm.transport.base.TransportState object at 0x7f943c8ddb40>> = <elasticapm.transport.base.TransportState object at 0x7f943c8ddb40>.did_fail
E        +      where <elasticapm.transport.base.TransportState object at 0x7f943c8ddb40> = <elasticapm.transport.http.Transport object at 0x7f943c8ddae0>.state
E        +        where <elasticapm.transport.http.Transport object at 0x7f943c8ddae0> = <elasticapm.base.Client object at 0x7f943c8a5c30>._transport

tests/client/client_tests.py:288: AssertionError
----------------------------- Captured stderr call -----------------------------
{"@timestamp":"2024-03-06T16:09:14.917Z","log.level":"error","message":"Failed to submit message: 'oopsie'","ecs":{"version":"1.6.0"},"log":{"logger":"elasticapm.transport","origin":{"file":{"line":318,"name":"base.py"},"function":"handle_transport_fail"},"original":"Failed to submit message: 'oopsie'"},"process":{"name":"MainProcess","pid":905083,"thread":{"id":140274711856704,"name":"eapm event processor thread"}},"service":{"name":"app_name"}}
{"@timestamp":"2024-03-06T16:09:14.917Z","log.level":"error","message":"Failed to submit message: 'oopsie'","ecs":{"version":"1.6.0"},"log":{"logger":"elasticapm.transport","origin":{"file":{"line":318,"name":"base.py"},"function":"handle_transport_fail"},"original":"Failed to submit message: 'oopsie'"},"process":{"name":"MainProcess","pid":905083,"thread":{"id":140274711856704,"name":"eapm event processor thread"}},"service":{"name":"app_name"}}
------------------------------ Captured log call -------------------------------
{"@timestamp":"2024-03-06T16:09:14.917Z","log.level":"error","message":"Failed to submit message: 'oopsie'","ecs":{"version":"1.6.0"},"log":{"logger":"elasticapm.transport","origin":{"file":{"line":318,"name":"base.py"},"function":"handle_transport_fail"},"original":"Failed to submit message: 'oopsie'"},"process":{"name":"MainProcess","pid":905083,"thread":{"id":140274711856704,"name":"eapm event processor thread"}},"service":{"name":"app_name"}}
=============================== warnings summary ===============================

To Reproduce

  1. pytest

Environment (please complete the following information)

  • OS: [e.g. Linux] Linux
  • Python version: 3.10.12
  • Framework and version [e.g. Django 2.1]:
  • APM Server version:
  • Agent version:

Additional context

None

@github-actions github-actions bot added agent-python community Issues opened by the community triage Issues awaiting triage labels Mar 6, 2024
@basepi
Copy link
Contributor

basepi commented Mar 20, 2024

This is definitely one of the most flakey tests in the suite. Luckily, it was mostly only flakey on windows in CI, so I haven't spent a ton of time digging into the internals. I've followed the code paths in the past trying to track this down and nothing seems amiss. I can't figure out how you can call client._transport.flush() and not have the transport correctly set its state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-python community Issues opened by the community triage Issues awaiting triage
Projects
None yet
Development

No branches or pull requests

2 participants