Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OpenStack dependencies to Yoga #7625

Merged
merged 9 commits into from May 5, 2023

Conversation

nelljerram
Copy link
Member

@nelljerram nelljerram commented May 3, 2023

Description

Related issues/PRs

Todos

  • Tests
  • Documentation
  • Release note

Release Note

Calico's integration code for OpenStack has been updated for OpenStack Yoga.

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

  • docs-pr-required: This change requires a change to the documentation that has not been completed yet.
  • docs-completed: This change has all necessary documentation completed.
  • docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

  • release-note-required: This PR has user-facing changes. Most PRs should have this label.
  • release-note-not-required: This PR has no user-facing changes.

Other optional labels:

  • cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
  • needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

@nelljerram nelljerram requested a review from a team as a code owner May 3, 2023 23:15
@marvin-tigera marvin-tigera added this to the Calico v3.27.0 milestone May 3, 2023
@marvin-tigera marvin-tigera added release-note-required Change has user-facing impact (no matter how small) docs-pr-required Change is not yet documented labels May 3, 2023
Fixes this traceback that we were getting in neutron-server.log, and that manifests as an "interval
server error" when making pretty much any request to the Neutron server (and hence causing total
failure of Calico's networking):

    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico [req-51d35ac0-b723-4925-a309-71107e576706 cd08451f0f57426eb5072dce1488e1af 83794660beb94e5b8c0107195933b7b5 - default default] Exception in function %s: oslo_config.cfg.NoSuchOptError: no such option report_interval in group [AGENT]
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico Traceback (most recent call last):
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3.10/dist-packages/networking_calico/logutils.py", line 21, in wrapped
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     return fn(*args, **kwargs)
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3.10/dist-packages/networking_calico/plugins/ml2/drivers/calico/mech_calico.py", line 365, in _post_fork_init
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     self.state_report_rpc = agent_rpc.PluginReportStateAPI(
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3/dist-packages/neutron/agent/rpc.py", line 86, in __init__
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     self.timeout = cfg.CONF.AGENT.report_interval
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 3161, in __getattr__
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     return self._conf._get(name, self._group)
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2653, in _get
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     value, loc = self._do_get(name, group, namespace)
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2671, in _do_get
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     info = self._get_opt_info(name, group)
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico   File "/usr/lib/python3/dist-packages/oslo_config/cfg.py", line 2876, in _get_opt_info
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico     raise NoSuchOptError(opt_name, group)
    2023-05-03 21:57:49.103 48436 ERROR networking_calico.plugins.ml2.drivers.calico.mech_calico oslo_config.cfg.NoSuchOptError: no such option report_interval in group [AGENT]

Explanation:

- Our Neutron driver uses a Neutron module (agent_rpc.PluginReportStateAPI) that is more typically
used by agents running on compute nodes, to report the health of Felix (which is our 'agent', in
OpenStack terminology) into the Neutron DB.

- From Ussuri to Yoga, it seems the code inside agent_rpc.PluginReportStateAPI has been updated to
assume registration of the [AGENT] report_interval option.

- Our driver code wasn't registering the group of options that includes that one.

Simple fix: register those options.  We don't _configure_ that option anywhere in our setup, to the
agent_rpc.PluginReportStateAPI code will get the default value for it, which is 30s.
Feels like a good idea, to check that the code fixes we're making for Yoga are not breaking Ussuri.
PIP_CONSTRAINT is needed as part of the ability to continue testing against both Ussuri and Yoga.

Not sure why HOME is now needed, but probably because we're picking up a newer version of some
library, as a result of removing constraints from test-requirements.txt.
To get past https://bugs.launchpad.net/tempest/+bug/1946321, with signature:

    Traceback (most recent call last):
      File "/opt/stack/tempest/tempest/cmd/verify_tempest_config.py", line 476, in take_action
        main(parsed_args)
      File "/opt/stack/tempest/tempest/cmd/verify_tempest_config.py", line 448, in main
        results = verify_extensions(os, service, results)
      File "/opt/stack/tempest/tempest/cmd/verify_tempest_config.py", line 248, in verify_extensions
        resp = extensions_client.list_extensions()
      File "/opt/stack/tempest/tempest/lib/services/compute/extensions_client.py", line 29, in list_extensions
        self.validate_response(schema.list_extensions, resp, body)
      File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 980, in validate_response
        jsonschema.validate(body, body_schema,
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 1119, in validate
        error = exceptions.best_match(validator.iter_errors(instance))
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/exceptions.py", line 384, in best_match
        best = next(errors, None)
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 288, in iter_errors
        for error in errors:
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/_validators.py", line 332, in properties
        yield from validator.descend(
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 305, in descend
        for error in self.evolve(schema=schema).iter_errors(instance):
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 288, in iter_errors
        for error in errors:
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/_legacy_validators.py", line 107, in items_draft3_draft4
        yield from validator.descend(item, items, path=index)
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 305, in descend
        for error in self.evolve(schema=schema).iter_errors(instance):
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 288, in iter_errors
        for error in errors:
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/_validators.py", line 332, in properties
        yield from validator.descend(
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 305, in descend
        for error in self.evolve(schema=schema).iter_errors(instance):
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/validators.py", line 288, in iter_errors
        for error in errors:
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/_validators.py", line 238, in format
        validator.format_checker.check(instance, format)
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/_format.py", line 135, in check
        result = func(instance)
      File "/opt/stack/tempest/tempest/lib/common/jsonschema_validator.py", line 34, in _validate_datetime_format
        if isinstance(instance, jsonschema.compat.str_types):
      File "/opt/stack/tempest/.tox/venv/lib/python3.8/site-packages/jsonschema/__init__.py", line 71, in __getattr__
        raise AttributeError(f"module {__name__} has no attribute {name}")
    AttributeError: module jsonschema has no attribute compat
    ERROR: InvocationError for command /opt/stack/tempest/.tox/venv/bin/tempest verify-config -uro /tmp/tmp.wPp8dRxbOW (exited with code 1)
For some reason python3-wrapt is pre-installed on a Semaphore ubuntu2004 node, but with
a version (1.11.2) that is different from the version that OpenStack needs (1.13.3), and
this was causing the DevStack setup to fail, because pip doesn't know how to uninstall
or replace the existing version.  Happily we do know that, so let's do it upfront here.
@nelljerram nelljerram added release-note-not-required Change has no user-facing impact release-note-required Change has user-facing impact (no matter how small) and removed release-note-required Change has user-facing impact (no matter how small) release-note-not-required Change has no user-facing impact labels May 5, 2023
@nelljerram nelljerram merged commit 286ff5d into projectcalico:master May 5, 2023
1 of 2 checks passed
@nelljerram nelljerram deleted the openstack-yoga branch May 5, 2023 14:55
@nelljerram
Copy link
Member Author

Docs PR is at tigera/docs#633

@comay
Copy link

comay commented Jun 5, 2023

While testing 3.26.0, I noticed that this PR did not appear to be presented in the Debian packages nor does it seem to be in the release-v3.26 branch. Could someone please look into this?

nelljerram added a commit to nelljerram/calico that referenced this pull request Jun 6, 2023
This was referenced Jun 6, 2023
@mgleung mgleung added the docs-not-required Docs not required for this change label Jun 12, 2023
@marvin-tigera marvin-tigera removed the docs-pr-required Change is not yet documented label Jun 12, 2023
@mgleung mgleung added docs-pr-required Change is not yet documented and removed docs-not-required Docs not required for this change labels Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-pr-required Change is not yet documented release-note-required Change has user-facing impact (no matter how small)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants