Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DocDB] RPATH is not being set correctly on some OpenLDAP libraries #21236

Open
1 task done
mbautin opened this issue Feb 28, 2024 · 0 comments
Open
1 task done

[DocDB] RPATH is not being set correctly on some OpenLDAP libraries #21236

mbautin opened this issue Feb 28, 2024 · 0 comments
Assignees
@mbautin
Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug priority/medium Medium priority issue

Comments

@mbautin
Copy link
Collaborator

mbautin commented Feb 28, 2024 •
edited by jira bot

Jira Link: DB-10161

Description

We are not setting RPATH correctly for a few OpenLDAP libraries because the executable bit is not set on them. As a result, either system versions of those libraries would be picked up, or they would not be found at all.

This currently affects the 2.20 branch.

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@mbautin mbautin added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Feb 28, 2024
@mbautin mbautin self-assigned this Feb 28, 2024
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue labels Feb 28, 2024
mbautin added a commit to mbautin/yugabyte-db that referenced this issue Feb 28, 2024
@mbautin
[BACKPORT 2.20] [yugabyte#21236] DocDB: Fix RPATH issue with OpenLDAP…
0f642dd 
… libraries for native glibc x86_64 build

Summary:
We were not setting RPATH correctly for a few OpenLDAP libraries because the executable bit was not set on them.  As a result, either system versions of those libraries would be picked up, or they would not be found at all.

Also refactoring library_packager.py:
- Use an enum for dependency categories.
- Reduce verbose logging when setting RPATHs.

Test Plan:
Jenkins: compile only

Manual testing:

```
./yb_build.sh release --no-tests --sj
./yb_release --skip_yugabyted_ui --force --no_reinitdb --skip_build --verbose |& tee ~/yb_release.log
cd build
tar xzf yugabyte-2.21.0.0-*-release-clang16-centos-x86_64.tar.gz
cd yugabyte-2.21.0.0

ldd postgres/bin/postgres
ldd postgres/bin/initdb
```

Make sure that ldd output of the above only references system glibc libraries and libldap/liblber libraries are linked from the yb-thirdparty directory.

Repeat the test in an AlmaLinux 8 Docker container.

---

Test with Linuxbrew to make sure it still works (even though we are deprecating it):

```
./yb_build.sh release --linuxbrew --no-tests
./yb_release --build_args="--linuxbrew" --build_root=build/release-clang16-linuxbrew-dynamic-ninja --skip_yugabyted_ui --force --no_reinitdb --skip_build --verbose
cd build
rm -rf yugabyte-2.21.0.0
tar xzf yugabyte-2.21.0.0-*-release-clang16-centos-x86_64.tar.gz
docker run -it --mount "type=bind,source=$PWD,target=/mnt/dir_with_package" almalinux:8
cd /mnt/dir_with_package/yugabyte-2.21.0.0
bin/post_install.sh
bin/yb-ctl create
bin/ysqlsh
```
and then try to create a table and insert/select some rows.

Reviewers: steve.varnau

Reviewed By: steve.varnau

Subscribers: ybase
mbautin added a commit that referenced this issue Feb 29, 2024
@mbautin
[BACKPORT 2.20] [#21236] DocDB: Fix RPATH issue with OpenLDAP librari…
f4464de 
…es for native glibc x86_64 build

Summary:
We were not setting RPATH correctly for a few OpenLDAP libraries because the executable bit was not set on them.  As a result, either system versions of those libraries would be picked up, or they would not be found at all.

Also refactoring library_packager.py:
- Use an enum for dependency categories.
- Reduce verbose logging when setting RPATHs.

Original commit: d893bb7  / D29140
Jira: DB-10161

Test Plan:
Jenkins: compile only

Manual testing:

```
./yb_build.sh release --no-tests --sj
./yb_release --skip_yugabyted_ui --force --no_reinitdb --skip_build --verbose |& tee ~/yb_release.log
cd build
tar xzf yugabyte-2.20.3.0-*-release-clang16-centos-x86_64.tar.gz
cd yugabyte-2.20.3.0
bin/post_install.sh  # Only on x86_64.
ldd postgres/bin/postgres
ldd postgres/bin/initdb
```

Make sure that ldd output of the above only references Linuxbrew glibc libraries, and the libldap/liblber libraries are linked from the yb-thirdparty directory.

Repeat the test in an AlmaLinux 8 Docker container on both x86_64 (with Linuxbrew) and aarch64 (without Linuxbrew).

Reviewers: steve.varnau

Reviewed By: steve.varnau

Subscribers: ybase

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D32731
mbautin added a commit to mbautin/yugabyte-db that referenced this issue Feb 29, 2024
@mbautin
[BACKPORT 2.20.2] [yugabyte#21236] DocDB: Fix RPATH issue with OpenLD…
5a4a93a 
…AP libraries for native glibc x86_64 build

Summary:
We were not setting RPATH correctly for a few OpenLDAP libraries because the executable bit was not set on them.  As a result, either system versions of those libraries would be picked up, or they would not be found at all.

Also refactoring library_packager.py:
- Use an enum for dependency categories.
- Reduce verbose logging when setting RPATHs.

Original commit: d893bb7  / D29140
Jira: DB-10161

Test Plan:
Jenkins: compile only

Manual testing:

```
./yb_build.sh release --no-tests --sj
./yb_release --skip_yugabyted_ui --force --no_reinitdb --skip_build --verbose |& tee ~/yb_release.log
cd build
tar xzf yugabyte-2.20.3.0-*-release-clang16-centos-x86_64.tar.gz
cd yugabyte-2.20.3.0
bin/post_install.sh  # Only on x86_64.
ldd postgres/bin/postgres
ldd postgres/bin/initdb
```

Make sure that ldd output of the above only references Linuxbrew glibc libraries, and the libldap/liblber libraries are linked from the yb-thirdparty directory.

Repeat the test in an AlmaLinux 8 Docker container on both x86_64 (with Linuxbrew) and aarch64 (without Linuxbrew).

Reviewers: steve.varnau

Subscribers: ybase

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D32740
@yugabyte-ci yugabyte-ci removed the status/awaiting-triage Issue awaiting triage label Feb 29, 2024
mbautin added a commit that referenced this issue Feb 29, 2024
@mbautin
[BACKPORT 2.20.2] [#21236] DocDB: Fix RPATH issue with OpenLDAP libra…
e3718ec 
…ries for native glibc x86_64 build

Summary:
We were not setting RPATH correctly for a few OpenLDAP libraries because the executable bit was not set on them.  As a result, either system versions of those libraries would be picked up, or they would not be found at all.

Also refactoring library_packager.py:
- Use an enum for dependency categories.
- Reduce verbose logging when setting RPATHs.

Original commit: d893bb7  / D29140
Jira: DB-10161

Test Plan:
Jenkins: compile only

Manual testing:

```
./yb_build.sh release --no-tests --sj
./yb_release --skip_yugabyted_ui --force --no_reinitdb --skip_build --verbose |& tee ~/yb_release.log
cd build
tar xzf yugabyte-2.20.2.0-*-release-clang16-centos-*.tar.gz
cd yugabyte-2.20.2.0
bin/post_install.sh  # Only on x86_64.
ldd postgres/bin/postgres | grep ldap
ldd postgres/bin/initdb | grep ldap
```

Make sure that ldd output of the above only references Linuxbrew glibc libraries, and the libldap/liblber libraries are linked from the yb-thirdparty directory.

Repeat the test in an AlmaLinux 8 Docker container on both x86_64 (with Linuxbrew) and aarch64 (without Linuxbrew).

bin/run_codecheck

Reviewers: steve.varnau

Reviewed By: steve.varnau

Subscribers: ybase

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D32740
mbautin added a commit that referenced this issue Mar 9, 2024
@mbautin
[BACKPORT 2.18] [#21236] Fix RPATH issue with OpenLDAP libraries for …
c7e4c74 
…native glibc x86_64 build

Summary:
We were not setting RPATH correctly for a few OpenLDAP libraries because the executable bit was not set on them.  As a result, either system versions of those libraries would be picked up, or they would not be found at all.

This backport does not include some of the refactoring from the original diff.

Original commit: d893bb7  / D29140

Test Plan:
bin/run_codecheck

Jenkins: compile only

Manual testing:

```
#!/usr/bin/env bash
set -euo pipefail -x
./yb_build.sh release --no-tests --sj
./yb_release --skip_yugabyted_ui --force --no_reinitdb --skip_build --verbose |& tee ~/yb_release.log
version=$( cat version.txt | cut -d- -f1 )
cd build
rm -rf yugabyte-$version
tar xzf yugabyte-$version-*-release-clang*-centos-x86_64.tar.gz
cd "yugabyte-$version"
bin/post_install.sh

ldd postgres/bin/postgres
ldd postgres/bin/initdb
```

Make sure that ldd output of the above only references system glibc libraries and libldap/liblber libraries are linked from the yb-thirdparty directory.

Repeat the test in an AlmaLinux 8 Docker container:

```
docker run -it --mount type=bind,src=$PWD/build,dst=/build docker.io/yugabyteci/yb_build_infra_almalinux8_x86_64:v2023-10-25T03_33_01
# In container:
cd /tmp
tar -xzf /build/yugabyte-*-release-clang*-centos-x86_64.tar.gz
cd yugabyte-*
bin/post_install.sh
ldd postgres/bin/postgres
ldd postgres/bin/initdb
```

Reviewers: steve.varnau

Reviewed By: steve.varnau

Subscribers: ybase

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D33005
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mbautin mbautin

Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mbautin @yugabyte-ci