Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try building py39 wheels to see if that helps with reinitialization errors #695

Merged
merged 1 commit into from
Dec 15, 2023

Conversation

alex
Copy link
Member

@alex alex commented Dec 12, 2023

No description provided.

@alex alex marked this pull request as draft December 12, 2023 15:48
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Dec 13, 2023
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-bcrypt that referenced this pull request Dec 13, 2023
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/_bcrypt/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-bcrypt_4.1.1.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-bcrypt.spec ../python-bcrypt_4.1.1-1.spec
    cp ../python*-bcrypt*4.1.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-bcrypt-4.1.1/
    rm -rf ../python*-bcrypt*4.1.1*.*

See pyca#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
@hswong3i
Copy link

hswong3i commented Dec 13, 2023

Original error message before applying patch from this PR:

Dec 12 03:41:14 node12 ceph-mgr[3265527]: 2023-12-12T03:41:14.178+0000 7fc8359f5280 -1 mgr[py] Module status has missing NOTIFY_TYPES member
Dec 12 03:41:14 node12 ceph-mgr[3265527]: 2023-12-12T03:41:14.314+0000 7fc8359f5280 -1 mgr[py] Module not found: 'mgr_module'
Dec 12 03:41:14 node12 ceph-mgr[3265527]: 2023-12-12T03:41:14.314+0000 7fc8359f5280 -1 mgr[py] Traceback (most recent call last):
Dec 12 03:41:14 node12 ceph-mgr[3265527]:   File "/usr/share/ceph/mgr/mgr_module.py", line 28, in <module>
Dec 12 03:41:14 node12 ceph-mgr[3265527]:     from mgr_util import profile_method
Dec 12 03:41:14 node12 ceph-mgr[3265527]:   File "/usr/share/ceph/mgr/mgr_util.py", line 6, in <module>
Dec 12 03:41:14 node12 ceph-mgr[3265527]:     import bcrypt
Dec 12 03:41:14 node12 ceph-mgr[3265527]:   File "/lib/python3/dist-packages/bcrypt/__init__.py", line 13, in <module>
Dec 12 03:41:14 node12 ceph-mgr[3265527]:     from ._bcrypt import (
Dec 12 03:41:14 node12 ceph-mgr[3265527]: ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

Now the error message after applying patch from this PR become:

Dec 13 06:41:15 node12 ceph-mgr[17264]: 2023-12-13T06:41:15.339+0000 7f3d7e793280 -1 mgr[py] Module status has missing NOTIFY_TYPES member
Dec 13 06:41:15 node12 ceph-mgr[17264]: 2023-12-13T06:41:15.479+0000 7f3d7e793280 -1 mgr[py] Module not found: 'mgr_module'
Dec 13 06:41:15 node12 ceph-mgr[17264]: 2023-12-13T06:41:15.479+0000 7f3d7e793280 -1 mgr[py] Traceback (most recent call last):
Dec 13 06:41:15 node12 ceph-mgr[17264]:   File "/usr/share/ceph/mgr/mgr_module.py", line 28, in <module>
Dec 13 06:41:15 node12 ceph-mgr[17264]:     from mgr_util import profile_method
Dec 13 06:41:15 node12 ceph-mgr[17264]:   File "/usr/share/ceph/mgr/mgr_util.py", line 6, in <module>
Dec 13 06:41:15 node12 ceph-mgr[17264]:     import bcrypt
Dec 13 06:41:15 node12 ceph-mgr[17264]:   File "/lib/python3/dist-packages/bcrypt/__init__.py", line 13, in <module>
Dec 13 06:41:15 node12 ceph-mgr[17264]:     from ._bcrypt import (
Dec 13 06:41:15 node12 ceph-mgr[17264]: ImportError: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576

See PyO3/pyo3#576

@hswong3i
Copy link

Since this PR already solve the "initialized once per interpreter process" issue, I would vote up for merge it + another patch release, so we could push Ceph side to fix their remaining subinterpreter issue.

Any idea?

@alex alex marked this pull request as ready for review December 14, 2023 00:42
@alex
Copy link
Member Author

alex commented Dec 14, 2023

Yes, I think it makes sense to do a release with this.

A quick review of the ceph source code suggests that using sub-interpreters is quite intentional by them (see src/mgr/PyModule.cc PyModule::load).

@reaperhulk reaperhulk merged commit b9223e6 into pyca:main Dec 15, 2023
51 checks passed
@alex alex deleted the try-py39 branch December 15, 2023 14:38
hswong3i added a commit to alvistack/pyca-bcrypt that referenced this pull request Dec 25, 2023
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/_bcrypt/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-bcrypt_4.1.1.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-bcrypt.spec ../python-bcrypt_4.1.1-1.spec
    cp ../python*-bcrypt*4.1.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-bcrypt-4.1.1/
    rm -rf ../python*-bcrypt*4.1.1*.*

See pyca#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Dec 25, 2023
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-bcrypt that referenced this pull request Jan 1, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/_bcrypt/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-bcrypt_4.1.1.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-bcrypt.spec ../python-bcrypt_4.1.1-1.spec
    cp ../python*-bcrypt*4.1.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-bcrypt-4.1.1/
    rm -rf ../python*-bcrypt*4.1.1*.*

See pyca#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Jan 1, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-bcrypt that referenced this pull request Jan 3, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/_bcrypt/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-bcrypt_4.1.1.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-bcrypt.spec ../python-bcrypt_4.1.1-1.spec
    cp ../python*-bcrypt*4.1.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-bcrypt-4.1.1/
    rm -rf ../python*-bcrypt*4.1.1*.*

See pyca#695
See pyca#714

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-bcrypt that referenced this pull request Jan 3, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/_bcrypt/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-bcrypt_4.1.1.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-bcrypt.spec ../python-bcrypt_4.1.1-1.spec
    cp ../python*-bcrypt*4.1.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-bcrypt-4.1.1/
    rm -rf ../python*-bcrypt*4.1.1*.*

See pyca#695
See pyca#714

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Feb 6, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Feb 16, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Feb 24, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-bcrypt that referenced this pull request Feb 24, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/_bcrypt/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-bcrypt_4.1.1.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-bcrypt.spec ../python-bcrypt_4.1.1-1.spec
    cp ../python*-bcrypt*4.1.1*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-bcrypt-4.1.1/
    rm -rf ../python*-bcrypt*4.1.1*.*

See pyca#695
See pyca#714

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
hswong3i added a commit to alvistack/pyca-cryptography that referenced this pull request Apr 6, 2024
    git clean -xdf
    mkdir -p .cargo
    cargo vendor --manifest-path ./src/rust/Cargo.toml > .cargo/config.toml
    tar zcvf ../python-cryptography_41.0.7.orig.tar.gz --exclude=.git .
    debuild -uc -us
    cp python-cryptography.spec ../python-cryptography_41.0.7-1.spec
    cp ../python*-cryptography*41.0.7*.{gz,xz,spec,dsc} /osc/home\:alvistack/pyca-cryptography-41.0.7/
    rm -rf ../python*-cryptography*41.0.7*.*

See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/pyo3-0.19.patch
See https://src.fedoraproject.org/rpms/python-cryptography/blob/rawhide/f/ouroboros-0.17.patch
See pyca/bcrypt#695

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants