Micromamba appears to fail at unlinking on Anaconda3 Docker #2193
Comments
|
Lots of packages from Anaconda channels are being replaced with conda-forge ones in your command. That could cause problems. |
|
@jonashaag I've thought about channel switching, but I can't get that to stick. Maybe you can elaborate if I missed something. I'm aware that mixing channels can lead to runtime issues, such as missing symbol references, but that's not the problem here, rather the transactions are warning/failing during file copying/link creation. Perhaps if the failure was in some pre-link or post-link script. Another possibility is clobbering, say if one channel's build includes files deployed by a differently-named package in another build. But that's not what fails in the Moreover, in the follow-up test, the exact same transaction pair that emits a
works without any error. So, I don't see anything inherent to the channel switching that can explain the behavior. I'm more thinking there is something about how the Anaconda base is installed, e.g., the Docker image runs these commands after installation: find /opt/conda/ -follow -type f -name '*.a' -delete
find /opt/conda/ -follow -type f -name '*.js.map' -delete
/opt/conda/bin/conda clean -afyDoes Micromamba somehow depend on something being deleted there? Does it short-circuit if a single file listed in the |
|
Maybe you can try to do the package removals only and have a look at the filesystem state after that? Are some files left over? Maybe the json files are incomplete? |
|
Another wild guess, maybe Mamba doesn’t correctly handle packages being switched from one channel to another (my idea would be that it doesn’t correctly remove the old package). I guess this could also be checked by doing the uninstalls explicitly and inspecting the filesystem. |
JSONs appear complete(base) root@f653a4a3b93e:/# sed -n '/"files":/,/]/p' /opt/conda/conda-meta/openssl-1.1.1q-h7f8727e_0.json
"files": [
"bin/c_rehash",
"bin/openssl",
"include/openssl/aes.h",
"include/openssl/asn1.h",
"include/openssl/asn1_mac.h",
"include/openssl/asn1err.h",
"include/openssl/asn1t.h",
"include/openssl/async.h",
"include/openssl/asyncerr.h",
"include/openssl/bio.h",
"include/openssl/bioerr.h",
"include/openssl/blowfish.h",
"include/openssl/bn.h",
"include/openssl/bnerr.h",
"include/openssl/buffer.h",
"include/openssl/buffererr.h",
"include/openssl/camellia.h",
"include/openssl/cast.h",
"include/openssl/cmac.h",
"include/openssl/cms.h",
"include/openssl/cmserr.h",
"include/openssl/comp.h",
"include/openssl/comperr.h",
"include/openssl/conf.h",
"include/openssl/conf_api.h",
"include/openssl/conferr.h",
"include/openssl/crypto.h",
"include/openssl/cryptoerr.h",
"include/openssl/ct.h",
"include/openssl/cterr.h",
"include/openssl/des.h",
"include/openssl/dh.h",
"include/openssl/dherr.h",
"include/openssl/dsa.h",
"include/openssl/dsaerr.h",
"include/openssl/dtls1.h",
"include/openssl/e_os2.h",
"include/openssl/ebcdic.h",
"include/openssl/ec.h",
"include/openssl/ecdh.h",
"include/openssl/ecdsa.h",
"include/openssl/ecerr.h",
"include/openssl/engine.h",
"include/openssl/engineerr.h",
"include/openssl/err.h",
"include/openssl/evp.h",
"include/openssl/evperr.h",
"include/openssl/hmac.h",
"include/openssl/idea.h",
"include/openssl/kdf.h",
"include/openssl/kdferr.h",
"include/openssl/lhash.h",
"include/openssl/md2.h",
"include/openssl/md4.h",
"include/openssl/md5.h",
"include/openssl/mdc2.h",
"include/openssl/modes.h",
"include/openssl/obj_mac.h",
"include/openssl/objects.h",
"include/openssl/objectserr.h",
"include/openssl/ocsp.h",
"include/openssl/ocsperr.h",
"include/openssl/opensslconf.h",
"include/openssl/opensslv.h",
"include/openssl/ossl_typ.h",
"include/openssl/pem.h",
"include/openssl/pem2.h",
"include/openssl/pemerr.h",
"include/openssl/pkcs12.h",
"include/openssl/pkcs12err.h",
"include/openssl/pkcs7.h",
"include/openssl/pkcs7err.h",
"include/openssl/rand.h",
"include/openssl/rand_drbg.h",
"include/openssl/randerr.h",
"include/openssl/rc2.h",
"include/openssl/rc4.h",
"include/openssl/rc5.h",
"include/openssl/ripemd.h",
"include/openssl/rsa.h",
"include/openssl/rsaerr.h",
"include/openssl/safestack.h",
"include/openssl/seed.h",
"include/openssl/sha.h",
"include/openssl/srp.h",
"include/openssl/srtp.h",
"include/openssl/ssl.h",
"include/openssl/ssl2.h",
"include/openssl/ssl3.h",
"include/openssl/sslerr.h",
"include/openssl/stack.h",
"include/openssl/store.h",
"include/openssl/storeerr.h",
"include/openssl/symhacks.h",
"include/openssl/tls1.h",
"include/openssl/ts.h",
"include/openssl/tserr.h",
"include/openssl/txt_db.h",
"include/openssl/ui.h",
"include/openssl/uierr.h",
"include/openssl/whrlpool.h",
"include/openssl/x509.h",
"include/openssl/x509_vfy.h",
"include/openssl/x509err.h",
"include/openssl/x509v3.h",
"include/openssl/x509v3err.h",
"lib/engines-1.1/afalg.so",
"lib/engines-1.1/capi.so",
"lib/engines-1.1/padlock.so",
"lib/libcrypto.a",
"lib/libcrypto.so",
"lib/libcrypto.so.1.1",
"lib/libssl.a",
"lib/libssl.so",
"lib/libssl.so.1.1",
"lib/pkgconfig/libcrypto.pc",
"lib/pkgconfig/libssl.pc",
"lib/pkgconfig/openssl.pc",
"ssl/ct_log_list.cnf",
"ssl/ct_log_list.cnf.dist",
"ssl/misc/CA.pl",
"ssl/misc/tsget",
"ssl/misc/tsget.pl",
"ssl/openssl.cnf",
"ssl/openssl.cnf.dist"
],Dry-Run TransactionLet's try force-removing (base) root@f653a4a3b93e:/# micromamba remove --dry-run -fn base --json openssl
{
"actions": {
"PREFIX": "/opt/conda",
"UNLINK": [
{
"build": "h7f8727e_0",
"build_number": 0,
"build_string": "h7f8727e_0",
"channel": "pkgs/main",
"constrains": null,
"depends": [
"ca-certificates",
"libgcc-ng >=7.5.0"
],
"fn": "openssl-1.1.1q-h7f8727e_0.conda",
"license": "",
"md5": "2ac47797afee2ece8d339c18b095b8d8",
"name": "openssl",
"sha256": "49804293b87141523b2606836ece8e2aaa5202983698fd91e7c36bdb8c8a8de5",
"size": 2649280,
"subdir": "linux-64",
"timestamp": 0,
"track_features": "",
"url": "https://repo.anaconda.com/pkgs/main/linux-64/openssl-1.1.1q-h7f8727e_0.conda",
"version": "1.1.1q"
}
]
},
"dry_run": true,
"prefix": "/opt/conda",
"success": true
}Package RemovalHere it reports the error in missing the (base) root@f653a4a3b93e:/# micromamba remove -fn base -vv openssl
__
__ ______ ___ ____ _____ ___ / /_ ____ _
/ / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
/ /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
/ .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
/_/
debug libmamba Loading configuration
debug libmamba 107 configurables computed
info libmamba Loading single package record: "/opt/conda/conda-meta/mkl_random-1.2.2-py39h51133e4_0.json"
info libmamba Loading single package record: "/opt/conda/conda-meta/textdistance-4.2.1-pyhd3eb1b0_0.json"
info libmamba Loading single package record: "/opt/conda/conda-meta/python-fastjsonschema-2.16.2-py39h06a4308_0.json"
[ omitting additional packages ]
info libmamba Loading single package record: "/opt/conda/conda-meta/openssl-1.1.1q-h7f8727e_0.json"
[ omitting additional packages ]
info libmamba Loading single package record: "/opt/conda/conda-meta/libev-4.33-h7f8727e_1.json"
info libmamba Loading single package record: "/opt/conda/conda-meta/zope-1.0-py39h06a4308_1.json"
info libmamba Loading single package record: "/opt/conda/conda-meta/conda-package-handling-1.9.0-py39h5eee18b_0.json"
info libmamba Adding package record to repo _ipyw_jlab_nb_ext_conf
info libmamba Adding package record to repo _libgcc_mutex
info libmamba Adding package record to repo _openmp_mutex
[ omitting additional packages ]
info libmamba Adding package record to repo openssl
[ omitting additional packages ]
info libmamba Adding package record to repo zope.interface
info libmamba Adding package record to repo zstd
info libmamba Parsing MatchSpec openssl
info libmamba Found python in installed packages 3.9.13
Transaction
Prefix: /opt/conda
Removing specs:
- openssl
debug libmamba Verify cache '/opt/conda/pkgs' for package extracted directory 'openssl-1.1.1q-h7f8727e_0'
debug libmamba Extracted package cache '/opt/conda/pkgs/openssl-1.1.1q-h7f8727e_0' not found
debug libmamba 'openssl-1.1.1q-h7f8727e_0' extracted directory cache is invalid
debug libmamba Verify cache '/opt/conda/pkgs' for package tarball 'openssl-1.1.1q-h7f8727e_0'
debug libmamba 'openssl-1.1.1q-h7f8727e_0' tarball cache is invalid
Package Version Build Channel Size
──────────────────────────────────────────────────────
Remove:
──────────────────────────────────────────────────────
- openssl 1.1.1q h7f8727e_0 pkgs/main 3MB
Summary:
Remove: 1 packages
Total download: 0 B
──────────────────────────────────────────────────────
Confirm changes: [Y/n] y
debug libmamba Locking directory '/opt/conda/conda-meta'
debug libmamba Successfully locked
info libmamba Cleaned 0 .mamba_trash files. 0 remaining.
Transaction starting
info libmamba All targets to download are cached
Unlinking openssl-1.1.1q-h7f8727e_0
info libmamba Unlinking package 'openssl-1.1.1q-h7f8727e_0'
debug libmamba Use metadata found at '/opt/conda/conda-meta/openssl-1.1.1q-h7f8727e_0.json'
debug libmamba Error when removing file '/opt/conda/lib/libcrypto.a' will be ignored
info libmamba Waiting for pyc compilation to finish
Transaction finished
info libmamba Opening history file: "/opt/conda/conda-meta/history"
debug libmamba Unlocking '/opt/conda/conda-meta'
info libmamba Freeing transaction.
info libmamba Freeing pool.Inspecting Post-Remove ResultsInspecting the filesystem with (base) root@f653a4a3b93e:/# cd /opt/conda
(base) root@f653a4a3b93e:/opt/conda# ls bin/{c_rehash,openssl}
ls: cannot access 'bin/c_rehash': No such file or directory
bin/openssl
(base) root@f653a4a3b93e:/opt/conda# ls include/openssl
aes.h bnerr.h conf_api.h dtls1.h evperr.h obj_mac.h pkcs12.h rsa.h stack.h x509.h
asn1.h buffer.h conferr.h e_os2.h hmac.h objects.h pkcs12err.h rsaerr.h store.h x509_vfy.h
asn1_mac.h buffererr.h crypto.h ebcdic.h idea.h objectserr.h pkcs7.h safestack.h storeerr.h x509err.h
asn1err.h camellia.h cryptoerr.h ec.h kdf.h ocsp.h pkcs7err.h seed.h symhacks.h x509v3.h
asn1t.h cast.h ct.h ecdh.h kdferr.h ocsperr.h rand.h sha.h tls1.h x509v3err.h
async.h cmac.h cterr.h ecdsa.h lhash.h opensslconf.h rand_drbg.h srp.h ts.h
asyncerr.h cms.h des.h ecerr.h md2.h opensslv.h randerr.h srtp.h tserr.h
bio.h cmserr.h dh.h engine.h md4.h ossl_typ.h rc2.h ssl.h txt_db.h
bioerr.h comp.h dherr.h engineerr.h md5.h pem.h rc4.h ssl2.h ui.h
blowfish.h comperr.h dsa.h err.h mdc2.h pem2.h rc5.h ssl3.h uierr.h
bn.h conf.h dsaerr.h evp.h modes.h pemerr.h ripemd.h sslerr.h whrlpool.h
(base) root@f653a4a3b93e:/opt/conda# ls lib/engines-1.1/
afalg.so capi.so padlock.so
(base) root@f653a4a3b93e:/opt/conda# ls lib/libcrypto*
lib/libcrypto.so
(base) root@f653a4a3b93e:/opt/conda# ls lib/libssl*
lib/libssl.so lib/libssl.so.1.1 lib/libssl3.so
(base) root@f653a4a3b93e:/opt/conda# ls lib/pkgconfig/{libcrypto,libssl,openssl}.pc
ls: cannot access 'lib/pkgconfig/libcrypto.pc': No such file or directory
ls: cannot access 'lib/pkgconfig/libssl.pc': No such file or directory
ls: cannot access 'lib/pkgconfig/openssl.pc': No such file or directory
(base) root@f653a4a3b93e:/opt/conda# ls ssl
cacert.pem cert.pem ct_log_list.cnf ct_log_list.cnf.dist misc openssl.cnf openssl.cnf.dist
(base) root@f653a4a3b93e:/opt/conda# ls ssl/misc/
CA.pl tsget tsget.plThe fact that some files are deleted seems to imply that |
|
Can you please run with another |
Yes, seems like it continues to unlink some files. Here's the last part with TRACE-level logging: debug libmamba Locking directory '/opt/conda/conda-meta'
trace libmamba Lockfile created at '/opt/conda/conda-meta/conda-meta.lock'
debug libmamba Successfully locked
info libmamba Cleaned 0 .mamba_trash files. 0 remaining.
Transaction starting
info libmamba All targets to download are cached
Unlinking openssl-1.1.1q-h7f8727e_0
info libmamba Unlinking package 'openssl-1.1.1q-h7f8727e_0'
debug libmamba Use metadata found at '/opt/conda/conda-meta/openssl-1.1.1q-h7f8727e_0.json'
trace libmamba Unlinking '/opt/conda/bin/c_rehash'
trace libmamba Unlinking '/opt/conda/lib/libcrypto.a'
debug libmamba Error when removing file '/opt/conda/lib/libcrypto.a' will be ignored
trace libmamba Unlinking '/opt/conda/lib/libcrypto.so.1.1'
trace libmamba Unlinking '/opt/conda/lib/pkgconfig/libcrypto.pc'
trace libmamba Unlinking '/opt/conda/lib/pkgconfig/libssl.pc'
trace libmamba Unlinking '/opt/conda/lib/pkgconfig/openssl.pc'
info libmamba Waiting for pyc compilation to finish
Transaction finished
info libmamba Opening history file: "/opt/conda/conda-meta/history"
debug libmamba Unlocking '/opt/conda/conda-meta'
trace libmamba Removing file '/opt/conda/conda-meta/conda-meta.lock'
info libmamba Freeing transaction.
info libmamba Freeing pool.Files versus PathsLooking at the full I'll have to check on the specification, but is this something malformed with this JSON shipping from Anaconda? Checking the one installed in my own system's base ( |
|
Maybe this is a recent extension to the file format that we don’t implement yet |
|
Just another note that I've also tested the Miniconda3 image ( In summary, the finding is that Anaconda and Miniconda ship with JSON files in the The reason why my original "follow-up" test passed was that installing the same package from Anaconda Cloud does result in a full version of the JSON, i.e., includes |
Not sure where to report this, but here's some curious behavior: Attempting to use Micromamba to install
mambainto the Anaconda installation in the official Docker image (continuumio/anaconda3) results in warnings and critical errors fromlibmamba. These seem to indicate cryptic failures in unlinking existing packages.I was hoping to give users a recommendation for how to quickly install Mamba into Anaconda base by using Micromamba, but this is worrisome behavior. Please let me know if something more needs to be configured besides the
MAMBA_ROOT_PREFIX.Reproducible Results
Additional Notes
This only appears to impact Micromamba when operating on the base.
Non-base works fine
Mimicking the same transaction in a new environment does not lead to an issue:
The text was updated successfully, but these errors were encountered: