-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
allow xarray > 2023.2 #542
Comments
A minor update: I set the xarray-constraint on the Feedstock because it's an issue from ska-sdp-func-python and not Karabo. In addition, I removed the build-nr fixings to enable constaint-updates. Otherwise, each legacy karabo installation would be corrupt if a feedstock-wheel needs an update. The hotfix is finished, as soon as there is an updated build of ska-sdp-func-python available with a larger build-nr. However, I'll try to create a reproducible test-case to catch this error. |
* added sarus doc 🍔 * enhanced doc 🎏 * added conda-prefix in dockerfile to have correct set env-var in sarus container 👨 * bugfix gpu-testing 📒 * skip pinocchio-test because it seems not to work properly on each platform 😖 * integrated user-build testing in workflow 8️⃣ * updated dockerfile-dev 🎅 * removed test-user-package because testing is now happening before pushing the image 👠 * adapted docker-dev 🍹 * Revert "removed test-user-package because testing is now happening before pushing the image 👠" This reverts commit a828dfd. * updated dockerfile-dev 🔌 * changed dev-workflow for testing 💀 * removed workflow-run trigger of dev-image because out of memory 🚈 * added mpich std-loc in Dockerfile.user 🍞 * adapted docker-user ⛎ * minor bugfix dockerfile-user 📥 * updated environments-files 🎇 * updated mpi-version in dockerfile 🆎 * updated Dockerfile-user structure 👾 * added mpi-tests to karabo 🌱 * updated requirements.txt with new versions 🎼 * integrated mpi-tests into github workflow and codecov-report 🆖 * minor fix using pytest-cov instead of coverage in github workflow 🌊 * fixed coverage-files discovery (I think) 🏧 * adapted codecov workflow according to github.com/codecov/codecov-action 🎲 * adapted container-doc 🍗 * bugfix Dockerfile mpi-compilation needs python 🍚 * minor adjustments in Dockerfile.user 😉 * added mpi-supported h5py wheel as karabo dependency 🍟 * removed pinocchio from environment.yaml 🔘 * updated environment.yaml with mpich-deps 🈴 * updated conda-build files 🚟 * replaced np.object0 with according non-deprecated alias 🐳 * minor bugfix in pyproject.toml 🙇 * deleted test-pinocchio (wrongly merged) 🍭 * updated environment files 🐎 * imporved dev-setup 🔉 * removed requirements.txt * added ipykernel to dev-deps 🚙 * bugfix filter-sources for xarray>2023.2 🔦 * updated build-procedure ⛪ * implemented dynamic versioneering 😔 * removed remaining requirements.txt stuff 👜 * updated Dockerfile-user mpi-installation to be dependent on karabo-user installation version 🛀 * refactored dockerfiles 🐭 * added versioneer to meta.yaml build-stage 📚 * added versioneer to meta.yaml build-stage 🚕 * bugfix conda-build meta.yaml for versioneer 💩 * adapted conda-build to custom pkg-version 😤 * adapted Dockerfile.user to a testable (before release) setup 👸 * bugfix build-user-image.yml ➿ * bugfix build-user-image.yml 🚧 * updated Dockerfile-user 👏 * bugfix user-dockerfile 🕘 * bugfix Dockerfile-dev 🔖 * minor doc-update in dockerfile-dev 🚸 * updated description of build-user-image workflow inputs 🔫 * introduced venv in dockerfile-user to not f*** up base env 😌 * updated dev-img with venv 😓 * updated build-user-image workflow to be able to run on workflow-dispatch 🐬 * added file to build-and-export-docker action 📌 * bugfix build-args in build-user-image workflow 👨 * bugfix get-shallow git-repo from git-rev (not only branches or tags) 📷 * bugfix remote from https and not from ssh 🎮 * updated workflow-dispatch descriptions‼️ * bugfix build-args passing to docker-build-push action 🆘 * throw exit-code in dockerfile-user if build not set correctly 🎵 * added dev-flag to conda-build 🌹 * added security to dev-builds in conda-build workflow 🛄 * bugfix dev-evaluation in conda-build.yml 🈂️ * outcommented mpich-compilation in Dockerfile 😥 * bugfix conda-build export of env-vars in same step ⛺ * added failing tests if PR is draft 🆖 * setup build-user-img, conda-build & test-user-package for dev-testing 🔍 * added df-h to build-user-image workflow 🎃 * made docker-build by myself ✂️ * made docker-build by myself entirely 🚥 * ensured to push on ghcr.io✈️ * bugfix docker-push user-image workflow 💘 * bugfix docker-push user-image workflow 📢 * bugfix added img-name to push image 🚸 * bugfix registry docker push 🍎 * adapted docker-img address accordingly to ghcr.io 🎩 * adapted build-user-image to standard 🐗 * added pytest installation to docker-user image testing workflow 🎢 * bugfix pytest-call in user-image 💴 * added bash shell in docker-run in build-test-user-image 👥 * adapted ld-library-path to base-image 2️⃣ * made site-package-location identification in build-user-image more robust 📵 * defined entrypoint to Dockerfile-user ➡️ * bugfix calling tests in build-user-image 🐻 * readded conda-activate-karabo to .bashrc for interactive mode 💓 * bugfix removed unnecessary " at the end of docker-run 🚅 * set env-vars in test-user-image ♓ * minor changes 🎵 * adapted test-workflow to main-setup 💡 * adapted docker-img dev to karabo-venv 📄 * adapted mpi-doc ❇️ * addressed mypy-issues 6️⃣ * added type-ignore to __init__.py because mypy can't handle that 🍥 * adapted exclude-option in setup.cfg to hopefully ignore __init__.py on the runners 🐽 * adapted mypy-exclude-regex to exclude all __init__.py ⏰ * hopefully bugfix to ignore __init__.py by mypy 🍟 * trying editable install to avoid duplicate modules 😓 * added verbose-flag to pytest-testing of docker-image 📫 * removed pytest-mpi as dep and added --only-mpi flag handling in conftest.py 👪 * minor changes in pytest test-discovery 🎁 * removed mpi-pytest from codecov because of race-condition issue which doesn't seem to be solvable atm 👬 * bugfix compute filter-mask in filter-sky 👫 * ugly hotfix to initialize dask-mpi inside docker-container 🏯 * added mpi-tests to docker-user tests 💀 * bugfix only enter dask-initialize with mpi if mpirun 🐚 * bugfix conda-build inherited env-vars [skip ci] 👚 * bugfix conda-build inherited env-vars [skip ci] 🚑 * bugfix conda-build set output-vars [skip ci] 🌈 * removed notebook-test from user-img test because data-dirs are not part of package and therefore would cause an error [skip ci] 🕠 * bugfix casting to str-to-boolean for reproducable github-workflows 🔩 * added debugging logs to github workflows [skip ci] ☎️ * adapted meta.yaml to pin compatible numpy ❓ * updated conda-build to be closer to a best-practise build 😿 * fixed bluebild-version 🌅 * removed echos and improved build-user-image workflow 🔻 * removed build-string default to ensure that workflow-user knows what he/she does 🐟 * minor update in container.md ⏩ * bugfix check leading v in tag in conda-build workflow 🍙 * bugfix set dev-string check correclty in conda-build.yml 🚷 * added versioneer to dev-dep ⚪ * improved version definition security for conda-build workflows ♒ * minor imporvements in conda-build⤵️ * hopefully bugfix of github boolean passing in reusable workflow see #1483 of github runners 🎈 * improved version definition security for conda-build workflows ↪️ * added little verbosity to conda-build workflow 📍 * hopefully bugfix to trigger build-docker by taking boolean values directly from input 👂 * bugfix build-user-image.yml 🛅 * minor improvements in build-user-image.yml 🎫 * replaced exit 2 with exit 1 in all bash scripts 3️⃣ * bugfix install environment in user Dockerfile correctly 💙 * removed weird leading v to github-workflows version-args 🍚 * updated codecov.yml to not fail if below target 🎰 * bugfix: added python interpreter and versioneer through conda for checking conda-build worklow inputs 🇷🇺 * added conda-prefix to python interpreter to hopefully get viable binaryⓂ️ * added conda-prefix to python interpreter to hopefully get viable binary * removed dev-deps in environment.yaml and meta.yaml ⌛ * bugfix: removed bdsf dev-dep 💒 * addressed mypy-issues 🚝 * updated mypy-complaints chunks-dict issue in sky-model 🚞 * fixed build-nr of feedstock-deps 🐹 * addresses mypy attr-defined for matplotlib BLUE 😾 * renamed build-user-image do build-docker-image 🐚 * adapted readme-badges 🍰 * addressed pr-request to install mpich via apt 🐒 * adapted documentation and dockerfile-steps 🍶 * bugfix added -y to apt install in Dockerfile [skip ci] ↩️ * improved docs ♈ * changed Dockerfile setup to use andromeda user instead of root 😉 * minor update in dockerfile 🔛 * changed run-stages of dockerfile 🍲 * removed user-changeing because of singularity uid issues 🔫 * moved Dockerfile to root & removed docker-dir 🚌 * added bash-env to env-vars for singularity noninteractive-shell [skip ci] 🕣 * bugfix: correctly activate venv in docker & singularity container for interactive and non-interactive shells 💛 * fixed conda activate functionality for docker & singularity interactive and non-interactive shells 💉 * removed unnecessary sourcing in dockerfile [skip ci] 🔣 * added ldconfig after mpich-installation in dockerfile 👤 * minor changes in dockerfile [skip ci] 🐟 * outcommented mpi-hook from dockerfile because it's still error-prone 📢 * adapted container-doc ↩️ * bugfix: in test docker-image to enable --only-mpi custom flag for pytest 📓 * added karabo shared lib to ldconfig cache to enable native cscs-mpi-hook 🕡 * replaced weird file-handler root-dir setup with /tmp with honor of TMP, TMPDIR & TEMP 🌐 * changed tmp-dir-name setup to avoid collisions 🐂 * loosened mpich-version constraints because we no longer rely on apt to install mpich ◻️ * updated mpi-doc 😡 * minor bugfix for mpi-tests in ci * added scratch as a possible tmpdir in FileHandler ⏰ * bugfix get-tmp-dir 🐲 * added cachetools as dep 😇 * redesigned FileHandler for short- and long-term-memory caching 😡 * improved file-handler by getting unique tmp-dir per unique object 💥 * refactored image and imager to new filehandler-setup 😔 * adapted interferometer.py and telescope.py to new FileHandler setup ↪️ * adapted visibility and sourcedetection to new FileHandler setup 🙌 * enhanced FileHandler get-tmp-dir with subdir & mkdir option 🚘 * adapted Karabo to new FileHandler setup 🐍 * bugfix accessing .ltm and .stm through FileHandler 👅 * bugfixes FileHandler & according tests 📝 * changed Downloadobject-storage cache from site-packages to tmp 🌁 * removed weird KaraboResource pseudo-interface from repo♥️ * removed FileHandler get-tmp-dir subdir option because seems unnecessary ⛽ * implemented seed-option to FileHandler.get-tmp-dir 🍢 * changed ltm & stm of FileHandler from static functions to lazy class-attributes 📄 * intermediate commit separating Dask & Slurm concerns in dask.py 🕔 * adapted other karabo-files to new DaskHandler setup 🚨 * readded accidentally removed plotting-util.py 👡 * bugfix dask-usage 🍀 * minor bugfix in test-dask 🚬 * refactored create_baseline_cut_telelescope to improved disk-caching & be less error-prone 😷 * bugfixes in Telescope.create-baseline-cut-telescope 🔑 * addressed mypy-issues 😐 * updated singularity-doc 🍓 * addressed PR-requests 🎍 * addressed PR526 requests 🍀 * bugfix removed set libmamba-solver globally in installation-user.md 📦 * made plot-function api more consistent 😰 * refactored DaskHandler to use just a single DaskHandler class for function-calling purpose 🎼 * bugfix removed fetch-dask-handler function from karabo 🎁 * addressed PR540 requests 🐻 * bugfix seeding ltm-memory 🎵 * improved api-signatures & bugfix in imager and result 🆖 * constrained xarray <=2023.2 (see issue #542) 💒 * bugfix cls to classmethod 🐐 * bugfix kwargs quiet passing in detect-sources-in-images 🌱 * removed guess-beam-parameters from result.py because it doesn't work anymore 🌀 * removed fixing of build-nr to enable dependency-constraint updates for past releases 💉 * reverted xarray-constraint because it's an issue of ska-sdp-func-python and not of Karabo⚠️ * added imaging-rascil test to test-image 💯 * enhanced environment.yaml description 🈴 * added guess-beam-parameters again after removal with improvements ♐ * relocated and enhanced guess-beam-parameters and it's usage 📔
For the imaging part of Karabo while running
source_detection.ipynb
, I've encountered the following error withxarray
2023.11 (see traceback). When I downgradedxarray
to 2023.2 (release before blame commit to raise ValueError), there was no such error and everything worked fine. Because this error is caused throughRASCIL
andska_sdp_func_python
(see traceback), I think fixing this error is out of my control. If my assessment is correct, when should fix it as follows:First, just constrain
xarray
to be <= 2023.2. However, this also constrainspandas
to be < 2 (conda-resolver said so). And because these two libraries are widely used libraries, we should address this issue further. However, this might require some effort. Firstly, becauseRASCIL
hasn't contrainedska_sdp_func_python
&ska_sdp_func
versions, I'm concerned that the latest release ofRASCIL
1.1.0 (April 2023) might not be compatible with the (hopefully fixed) major release of ska-sdp-func 1.0 (November 2023). In addition, we also need to build the according conda-wheels after testing.Long story short: As a hotfix, I'll constrain
xarray
<= 2023.2 (and accept thepandas
downgrade for the moment). The best-case scenario would be thatRASCIL
soon does a new release, where we could fix the accordingska-sdp
deps. I don't think that building a dirty conda-wheel (10 months after the last release) ofRASCIL
is a good solution. If there's a better suggestion, I'd like to hear it.The text was updated successfully, but these errors were encountered: