Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1873556: [on-prem] inject the proxy into the env for NetworkManager.service #2266

Merged
merged 1 commit into from Dec 2, 2020

Conversation

EmilienM
Copy link
Member

@EmilienM EmilienM commented Nov 30, 2020

In order to have the proxy variables (HTTP_PROXY, HTTPS_PROXY and
NO_PROXY), we need to export them otherwise then don't end up being
loaded in the environment and it causes issues if a proxy is used,
when pulling an image from a registry for example.

@openshift-ci-robot
Copy link
Contributor

@EmilienM: An error was encountered adding this pull request to the external tracker bugs for bug 1873556 on the Bugzilla server at https://bugzilla.redhat.com:

JSONRPC error 32000: There was an error reported for the RPC call to Jira: There was an error reported for a GitHub REST call. URL: https://api.github.com/repos/openshift/machine-config-operator/pulls/2266 Error: 403 Forbidden at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Type/GitHub.pm line 111. at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Type/GitHub.pm line 111. eval {...} called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Type/GitHub.pm line 98 Bugzilla::Extension::ExternalBugs::Type::GitHub::_do_rest_call('Bugzilla::Extension::ExternalBugs::Type::GitHub=HASH(0x556e28...', 'https://api.github.com/repos/openshift/machine-config-operato...', 'GET') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Type/GitHub.pm line 62 Bugzilla::Extension::ExternalBugs::Type::GitHub::get_data('Bugzilla::Extension::ExternalBugs::Type::GitHub=HASH(0x556e28...', 'Bugzilla::Extension::ExternalBugs::Bug=HASH(0x556e28c51390)') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Bug.pm line 302 eval {...} called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Bug.pm line 302 Bugzilla::Extension::ExternalBugs::Bug::update_ext_info('Bugzilla::Extension::ExternalBugs::Bug=HASH(0x556e28c51390)', 1) called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Bug.pm line 125 Bugzilla::Extension::ExternalBugs::Bug::create('Bugzilla::Extension::ExternalBugs::Bug', 'HASH(0x556e28c3c8c8)') called at /var/www/html/bugzilla/extensions/ExternalBugs/Extension.pm line 940 Bugzilla::Extension::ExternalBugs::bug_start_of_update('Bugzilla::Extension::ExternalBugs=HASH(0x556e27e46760)', 'HASH(0x556e27b3ef00)') called at /var/www/html/bugzilla/Bugzilla/Hook.pm line 21 Bugzilla::Hook::process('bug_start_of_update', 'HASH(0x556e27b3ef00)') called at /var/www/html/bugzilla/Bugzilla/Bug.pm line 1173 Bugzilla::Bug::update('Bugzilla::Bug=HASH(0x556e284f3378)') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/WebService.pm line 88 Bugzilla::Extension::ExternalBugs::WebService::add_external_bug('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...', 'HASH(0x556e28636cc0)') called at (eval 2269) line 1 eval ' $procedure->{code}->($self, @params) ;' called at /usr/share/perl5/vendor_perl/JSON/RPC/Legacy/Server.pm line 220 JSON::RPC::Legacy::Server::_handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...', 'HASH(0x556e28c35d48)') called at /var/www/html/bugzilla/Bugzilla/WebService/Server/JSONRPC.pm line 297 Bugzilla::WebService::Server::JSONRPC::_handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...', 'HASH(0x556e28c35d48)') called at /usr/share/perl5/vendor_perl/JSON/RPC/Legacy/Server.pm line 126 JSON::RPC::Legacy::Server::handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...') called at /var/www/html/bugzilla/Bugzilla/WebService/Server/JSONRPC.pm line 70 Bugzilla::WebService::Server::JSONRPC::handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...') called at /var/www/html/bugzilla/jsonrpc.cgi line 31 ModPerl::ROOT::Bugzilla::ModPerl::ResponseHandler::var_www_html_bugzilla_jsonrpc_2ecgi::handler('Apache2::RequestRec=SCALAR(0x556e284fd658)') called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 207 eval {...} called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 207 ModPerl::RegistryCooker::run('Bugzilla::ModPerl::ResponseHandler=HASH(0x556e27e80498)') called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 173 ModPerl::RegistryCooker::default_handler('Bugzilla::ModPerl::ResponseHandler=HASH(0x556e27e80498)') called at /usr/lib64/perl5/vendor_perl/ModPerl/Registry.pm line 32 ModPerl::Registry::handler('Bugzilla::ModPerl::ResponseHandler', 'Apache2::RequestRec=SCALAR(0x556e284fd658)') called at /var/www/html/bugzilla/mod_perl.pl line 139 Bugzilla::ModPerl::ResponseHandler::handler('Bugzilla::ModPerl::ResponseHandler', 'Apache2::RequestRec=SCALAR(0x556e284fd658)') called at (eval 2269) line 0 eval {...} called at (eval 2269) line 0 at /var/www/html/bugzilla/Bugzilla/Error.pm line 130. Bugzilla::Error::_throw_error('global/user-error.html.tmpl', 'ext_bz_rest_error', 'HASH(0x556e28c30b90)') called at /var/www/html/bugzilla/Bugzilla/Error.pm line 193 Bugzilla::Error::ThrowUserError('ext_bz_rest_error', 'HASH(0x556e28c30b90)') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Type/GitHub.pm line 120 Bugzilla::Extension::ExternalBugs::Type::GitHub::_do_rest_call('Bugzilla::Extension::ExternalBugs::Type::GitHub=HASH(0x556e28...', 'https://api.github.com/repos/openshift/machine-config-operato...', 'GET') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Type/GitHub.pm line 62 Bugzilla::Extension::ExternalBugs::Type::GitHub::get_data('Bugzilla::Extension::ExternalBugs::Type::GitHub=HASH(0x556e28...', 'Bugzilla::Extension::ExternalBugs::Bug=HASH(0x556e28c51390)') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Bug.pm line 302 eval {...} called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Bug.pm line 302 Bugzilla::Extension::ExternalBugs::Bug::update_ext_info('Bugzilla::Extension::ExternalBugs::Bug=HASH(0x556e28c51390)', 1) called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/Bug.pm line 125 Bugzilla::Extension::ExternalBugs::Bug::create('Bugzilla::Extension::ExternalBugs::Bug', 'HASH(0x556e28c3c8c8)') called at /var/www/html/bugzilla/extensions/ExternalBugs/Extension.pm line 940 Bugzilla::Extension::ExternalBugs::bug_start_of_update('Bugzilla::Extension::ExternalBugs=HASH(0x556e27e46760)', 'HASH(0x556e27b3ef00)') called at /var/www/html/bugzilla/Bugzilla/Hook.pm line 21 Bugzilla::Hook::process('bug_start_of_update', 'HASH(0x556e27b3ef00)') called at /var/www/html/bugzilla/Bugzilla/Bug.pm line 1173 Bugzilla::Bug::update('Bugzilla::Bug=HASH(0x556e284f3378)') called at /loader/0x556e20ddc390/Bugzilla/Extension/ExternalBugs/WebService.pm line 88 Bugzilla::Extension::ExternalBugs::WebService::add_external_bug('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...', 'HASH(0x556e28636cc0)') called at (eval 2269) line 1 eval ' $procedure->{code}->($self, @params) ;' called at /usr/share/perl5/vendor_perl/JSON/RPC/Legacy/Server.pm line 220 JSON::RPC::Legacy::Server::_handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...', 'HASH(0x556e28c35d48)') called at /var/www/html/bugzilla/Bugzilla/WebService/Server/JSONRPC.pm line 297 Bugzilla::WebService::Server::JSONRPC::_handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...', 'HASH(0x556e28c35d48)') called at /usr/share/perl5/vendor_perl/JSON/RPC/Legacy/Server.pm line 126 JSON::RPC::Legacy::Server::handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...') called at /var/www/html/bugzilla/Bugzilla/WebService/Server/JSONRPC.pm line 70 Bugzilla::WebService::Server::JSONRPC::handle('Bugzilla::WebService::Server::JSONRPC::Bugzilla::Extension::E...') called at /var/www/html/bugzilla/jsonrpc.cgi line 31 ModPerl::ROOT::Bugzilla::ModPerl::ResponseHandler::var_www_html_bugzilla_jsonrpc_2ecgi::handler('Apache2::RequestRec=SCALAR(0x556e284fd658)') called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 207 eval {...} called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 207 ModPerl::RegistryCooker::run('Bugzilla::ModPerl::ResponseHandler=HASH(0x556e27e80498)') called at /usr/lib64/perl5/vendor_perl/ModPerl/RegistryCooker.pm line 173 ModPerl::RegistryCooker::default_handler('Bugzilla::ModPerl::ResponseHandler=HASH(0x556e27e80498)') called at /usr/lib64/perl5/vendor_perl/ModPerl/Registry.pm line 32 ModPerl::Registry::handler('Bugzilla::ModPerl::ResponseHandler', 'Apache2::RequestRec=SCALAR(0x556e284fd658)') called at /var/www/html/bugzilla/mod_perl.pl line 139 Bugzilla::ModPerl::ResponseHandler::handler('Bugzilla::ModPerl::ResponseHandler', 'Apache2::RequestRec=SCALAR(0x556e284fd658)') called at (eval 2269) line 0 eval {...} called at (eval 2269) line 0
Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

Bug 1873556: export proxy variables to be taken in account

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@cgwalters
Copy link
Member

Oh heh, that's an embarrassing bug.
/approve

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 30, 2020
@cgwalters
Copy link
Member

(Unrelated to this PR's code, I love the stack trace from our Github automation trying to talk to Bugzilla which is trying to talk to JIRA...)

@kikisdeliveryservice kikisdeliveryservice changed the title Bug 1873556: export proxy variables to be taken in account Bug 1873556: [on-prem] export proxy variables to be taken in account Nov 30, 2020
@mandre
Copy link
Member

mandre commented Dec 1, 2020

Perhaps it's a good time to start setting the proxy configuration system-wide rather than per script. Not sure if this has been considered previously, but is there any reason not to do it?

For our use, it should be enough to drop relevant files to /etc/systemd/system.conf.d/ and /etc/profile.d/. Then we could clean up the many places where we set the proxy in MCO.

See example at https://coreos.com/os/docs/latest/using-environment-variables-in-systemd-units.html#other-examples

@EmilienM
Copy link
Member Author

EmilienM commented Dec 1, 2020

Perhaps it's a good time to start setting the proxy configuration system-wide rather than per script. Not sure if this has been considered previously, but is there any reason not to do it?

For our use, it should be enough to drop relevant files to /etc/systemd/system.conf.d/ and /etc/profile.d/. Then we could clean up the many places where we set the proxy in MCO.

We could even inject them via ignition, as in the example at https://coreos.com/os/docs/latest/using-environment-variables-in-systemd-units.html#other-examples

+1, if maintainers think it's a good idea I would be happy to take a look in this direction.

@mandre
Copy link
Member

mandre commented Dec 1, 2020

Perhaps it's a good time to start setting the proxy configuration system-wide rather than per script. Not sure if this has been considered previously, but is there any reason not to do it?

+1, if maintainers think it's a good idea I would be happy to take a look in this direction.

Sorry I should have been a bit more specific. IMO making the setting system-wide is the next logical step but should not necessarily be handled in this PR as it would extend the scope quite a bit.

Thanks for the bug fix.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Dec 1, 2020
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@cgwalters
Copy link
Member

Yeah, I argued to do system wide but...I forget why we didn't.

Hmm. One thing we could probably do is render an /etc/mco/proxy.env file and change our units to source it via EnvironmentFile=/etc/mco/proxy.env instead.

Rather than patching our dispatcher script we could also inject the proxy into the env for NetworkManager.service instead (so all dispatcher scripts would inherit).

@EmilienM
Copy link
Member Author

EmilienM commented Dec 1, 2020

Yeah, I argued to do system wide but...I forget why we didn't.

Hmm. One thing we could probably do is render an /etc/mco/proxy.env file and change our units to source it via EnvironmentFile=/etc/mco/proxy.env instead.

Rather than patching our dispatcher script we could also inject the proxy into the env for NetworkManager.service instead (so all dispatcher scripts would inherit).

sounds like a plan!

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Dec 1, 2020
@EmilienM EmilienM changed the title Bug 1873556: [on-prem] export proxy variables to be taken in account Bug 1873556: [on-prem] inject the proxy into the env for NetworkManager.service Dec 1, 2020
@openshift-ci-robot openshift-ci-robot added bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Dec 1, 2020
@openshift-ci-robot
Copy link
Contributor

@EmilienM: This pull request references Bugzilla bug 1873556, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1873556: [on-prem] inject the proxy into the env for NetworkManager.service

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@cgwalters
Copy link
Member

Did you test this? It looks correct...but it's a larger change. I agree with this comment - let's merge the obviously-correct original code and do this as a separate followup.

(But if you've tested it I'm OK to merge as is too)

@EmilienM
Copy link
Member Author

EmilienM commented Dec 1, 2020

Did you test this? It looks correct...but it's a larger change. I agree with this comment - let's merge the obviously-correct original code and do this as a separate followup.

I agree it would be safer to proceed with the initial change request and move that refactor into another PR.
I'm still in in the process of testing your proposal, so I'll split PRs now so bug can be closed soon.

In order to have the proxy variables (HTTP_PROXY, HTTPS_PROXY and
NO_PROXY), we need to `export` them otherwise then don't end up being
loaded in the environment and it causes issues if a proxy is used,
when pulling an image from a registry for example.

Signed-off-by: Emilien Macchi <emilien@redhat.com>
@openshift-ci-robot
Copy link
Contributor

@EmilienM: This pull request references Bugzilla bug 1873556, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1873556: [on-prem] inject the proxy into the env for NetworkManager.service

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

19 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot
Copy link
Contributor

openshift-merge-robot commented Dec 2, 2020

@EmilienM: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-metal-ipi f41b1d2 link /test e2e-metal-ipi
ci/prow/e2e-aws-workers-rhel7 f41b1d2 link /test e2e-aws-workers-rhel7
ci/prow/okd-e2e-aws f41b1d2 link /test okd-e2e-aws

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-robot openshift-merge-robot merged commit 824ddee into openshift:master Dec 2, 2020
@openshift-ci-robot
Copy link
Contributor

@EmilienM: All pull requests linked via external trackers have merged:

Bugzilla bug 1873556 has been moved to the MODIFIED state.

In response to this:

Bug 1873556: [on-prem] inject the proxy into the env for NetworkManager.service

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@EmilienM EmilienM deleted the bz/1873556 branch December 2, 2020 18:52
@EmilienM
Copy link
Member Author

EmilienM commented Dec 2, 2020

/cherry-pick release-4.6

@openshift-cherrypick-robot

@EmilienM: #2266 failed to apply on top of branch "release-4.6":

Applying: Bug 1873556: export proxy variables to be taken in account
Using index info to reconstruct a base tree...
A	templates/common/on-prem/files/NetworkManager-resolv-prepender.yaml
Falling back to patching base and 3-way merge...
Auto-merging templates/common/baremetal/files/NetworkManager-resolv-prepender.yaml
CONFLICT (content): Merge conflict in templates/common/baremetal/files/NetworkManager-resolv-prepender.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Bug 1873556: export proxy variables to be taken in account
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-4.6

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mandre
Copy link
Member

mandre commented Dec 3, 2020

/cherry-pick release-4.6

The patch doesn't apply cleanly because we de-duped the on prem platforms templates into the on-prem directory in 4.7. They were previously in platform specific directories, openstack, baremetal, ovirt, and vsphere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet