fix LD_LIBRARY_PATH for opentelemetry

[esigo]

What this PR does / why we need it:

Currently opentelemetry library files cannot be loaded because the LD_LIBRARY_PATH of the nginx image point to wrong path.
This PR fixes the path used for opentelemetry.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• CVE Report (Scanner found CVE and adding report)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation only

Which issue/s this PR fixes

#9016 step 1

How Has This Been Tested?

This is e2e tested in my dev branch.

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added unit and/or e2e tests to cover my changes.
• All new and existing tests passed.
• Added Release Notes.

Does my pull request need a release note?

Any user-visible or operator-visible change qualifies for a release note. This could be a:

• CLI change
• API change
• UI change
• configuration schema change
• behavioral change
• change in non-functional attributes such as efficiency or availability, availability of a new platform
• a warning about a deprecation
• fix of a previous Known Issue
• fix of a vulnerability (CVE)

No release notes are required for changes to the following:

• Tests
• Build infrastructure
• Fixes for unreleased bugs

For more tips on writing good release notes, check out the Release Notes Handbook

PLACE RELEASE NOTES HERE 

[k8s-ci-robot]

@esigo: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

Hi @esigo. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[longwuyuan]

@esigo , can you add copy/paste and logs and other such info, from your local brnach build, that this path fixes the "current" ld_lib_path. I was under the impression that the current code you checked in puts the modules in a path that has "modules/modules" (2 times modules). But since you are changing both eh makefile and the path, I guess you are changing the whole scene, so please confirm just for posterity sake, that this is good to go. Because merging this will fire cloudbuild for the otel image and then we need to promote it etc etc.

[longwuyuan]

cc @strongjz

Building k8s.io/ingress-nginx/cmd/nginx
+ git config --global --add safe.directory /go/src/k8s.io/ingress-nginx
error: could not lock config file //.gitconfig: Permission denied
make: *** [Makefile:120: build] Error 255
Error: Process completed with exit code 2.

[esigo]

@esigo , can you add copy/paste and logs and other such info, from your local brnach build, that this path fixes the "current" ld_lib_path. I was under the impression that the current code you checked in puts the modules in a path that has "modules/modules" (2 times modules). But since you are changing both eh makefile and the path, I guess you are changing the whole scene, so please confirm just for posterity sake, that this is good to go. Because merging this will fire cloudbuild for the otel image and then we need to promote it etc etc.

@longwuyuan yes there was an extra modules in the path. I think it would be better to move everything under /modules_mount/etc/nginx/modules/otel/

I can't get a bash to the controller container, so I just post log and a trace I have for test.
This is how I load the module:

{{ if (shouldLoadOpentelemetryModule $cfg $servers) }}
load_module /modules_mount/etc/nginx/modules/otel/otel_ngx_module.so;
{{ end }}

I have these logs (with so many debug messages ;)), and a trace going into Jaeger.

[longwuyuan]

/ok-to-test

[longwuyuan]

@esigo soem form/content of docs also will be part of the final delivery on this. Including stating that the project does not use opentracing etc. anymore and all traces are only through otel

[esigo]

@esigo soem form/content of docs also will be part of the final delivery on this. Including stating that the project does not use opentracing etc. anymore and all traces are only through otel

@longwuyuan I've planned to bring the documentation in the 3rd step (user-interface). I can segregate that into another step if that's desired.

[longwuyuan]

not desired. just getting a few words out on the aspect.
Can you please rebase. The git safedir related --global flag was removed so need to check if the failing CI step passes, after rebase.

[longwuyuan]

/lgtm

cc @tao12345666333 @rikatz @strongjz we discussed this in the last meeting. This is ready to go to next step. Please check

[tao12345666333]

/lgtm

thanks

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: esigo, longwuyuan, tao12345666333

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tao12345666333]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[longwuyuan]

@esigo , please lets stay in sync to promote and configure values.yaml, with the new otel image that will get built in gcp-cloudbuild, because of this PR. We can sync on slack chat tomorrow.

[esigo]

@esigo , please lets stay in sync to promote and configure values.yaml, with the new otel image that will get built in gcp-cloudbuild, because of this PR. We can sync on slack chat tomorrow.

@longwuyuan sure, thanks :)