Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to fix flaky test by waiting for setup to complete #74734

Merged
merged 1 commit into from Jul 8, 2022

Conversation

allenporter
Copy link
Contributor

Proposed change

Wait for setup to complete before trying to assert on log messages. The idea is that the integration setup should finish (and any associated log messages) so that when the test looks at warnings it should only find warnings that it created itself during the test.

Test failure is:

_____________________________ test_warning[pyloop] _____________________________
[gw1] linux -- Python 3.9.13 /home/runner/work/core/core/venv/bin/python3
hass = <homeassistant.core.HomeAssistant object at 0x7f0efc785790>
simple_queue = <_queue.SimpleQueue object at 0x7f0f100aaea0>
hass_ws_client = <function hass_ws_client.<locals>.create_client at 0x7f0f123fb040>
    async def test_warning(hass, simple_queue, hass_ws_client):
        """Test that warning are logged and retrieved correctly."""
        await async_setup_component(hass, system_log.DOMAIN, BASIC_CONFIG)
        _LOGGER.warning("warning message")
        await _async_block_until_queue_empty(hass, simple_queue)
        log = find_log(await get_error_log(hass_ws_client), "WARNING")
>       assert_log(log, "", "warning message", "WARNING")
tests/components/system_log/test_init.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
log = {'count': 1, 'exception': '', 'first_occurred': 16572[90](https://github.com/home-assistant/core/runs/7253180094?check_suite_focus=true#step:10:91)[92](https://github.com/home-assistant/core/runs/7253180094?check_suite_focus=true#step:10:93)8.7554243, 'level': 'WARNING', ...}
exception = '', message = ['warning message'], level = 'WARNING'
    def assert_log(log, exception, message, level):
        """Assert that specified values are in a specific log entry."""
        if not isinstance(message, list):
            message = [message]
>       assert log["name"] == "test_logger"
E       AssertionError: assert 'asyncio' == 'test_logger'
E         - test_logger
E         + asyncio

With these log messages:

WARNING:test_logger:warning message
WARNING:asyncio:Executing <Task pending name='Task-13849' coro=<test_warning() running at /home/runner/work/core/core/tests/components/system_log/test_init.py:116> cb=[_run_until_complete_cb() at /opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/asyncio/base_events.py:184] created at /opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/asyncio/base_events.py:626> took 0.160 seconds

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 馃 Silver
  • 馃 Gold
  • 馃弳 Platinum

To help with the load of incoming pull requests:

@epenet epenet added the ci-full-run Marks a PR to trigger a full CI run (instead of partial) label Jul 8, 2022
@epenet epenet closed this Jul 8, 2022
@epenet epenet reopened this Jul 8, 2022
@epenet
Copy link
Contributor

epenet commented Jul 8, 2022

Closed/Reopened to trigger full CI run instead of partial run.

@balloob balloob merged commit ff324ab into home-assistant:dev Jul 8, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Jul 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
ci-full-run Marks a PR to trigger a full CI run (instead of partial) cla-signed core has-tests integration: system_log small-pr PRs with less than 30 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants