-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support conda (python) package manager #2213
Comments
Hi, does Renovate now support conda? |
Hi, just wondering whether there are plans to add conda support soon? Alternatively, shall I try adding it using the instructions here: https://github.com/renovatebot/renovate/blob/master/docs/development/adding-a-package-manager.md? |
No plans, and a PR would be very welcome! I updated the doc just now to make sure it's current. |
Thank you for updating the docs - I'll give this a go when I have some time :) |
#6969 duplicated this, thus closed it. The relevant comments from there: To also verify python package versions in conda environment files ( Did you already have any implementation ideas? Are there any workarounds or alternative ideas you've tried to avoid needing this feature? Conda environments can also include pip requirememnts, a workaround is to put those in a separate txt file, and have renovatebot check those.
This workaround however does not work for the conda packages (like the python=3.7 here, and any conda packages installed, like jupyter in this case) Is this a feature you'd be interested in implementing yourself? ** Related features** |
It would be helpful if anyone can provide some public repo examples that can be tested against, as well as clarifications on file naming / file syntax. For example should we match against every |
From their docs https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-from-an-environment-yml-file they reference it against environment.yml files. I know internally we use different names for our different projects (IE we name our env the same codename as the project so titan.yml ect) but if that is an issue we can look at changing them back to the conda default name. |
I'm also interested in this as the PyTorch images (and the jupyter-stack ones) use conda. |
We could give it a shot, starting from the datasource. I am not very familiar with typescript but I could give it a go. The docs in the datasource require a function called ``getReleases` with input:
If I understood it correctly, for something like
Am I missing something? |
I would just like to make a note that the conda-forge channel would be important to have support for (also in terms of ToS of the regular conda channel). |
I agree that starting with the datasource first makes best sense. Importantly, note that Renovate doesn't yet have the concept of "platform" for datasources but it looks like that might be necessary for conda packages? |
Jupyter docker images would also greatly benefit from this feature. Right now, we're updating our dependencies manually, and it would be great to get rid of this maintenance burden. |
Hey everyone, Anaconda engineer here. We've assembled a small group of engineers that is looking into adding functionality for conda over time. Please note that none of us is working on this full time right now, but work will be done over time. I've started implementing a datasource for conda in #14257, any help with my testing issue and feedback in general is very welcome! |
Thanks for starting the work on this! I am happy to start testing, but I am not sure how since I am new to renovate. I gave it a try with adding a
What is the best way to test and configure this? Or I am too early 😺 |
@mdehollander You're not too early, but there's no manager implemented yet, just a datasource. I'll take a first shot at a manager next Friday. You can check the datasource documentation at https://docs.renovatebot.com/modules/datasource/#conda-datasource. If you want to use it right now, here's an example for how to do so. In your
This annotates the {
"reviewersFromCodeOwners": "true",
"regexManagers": [
{
"description": "Upgrade conde dependencies",
"fileMatch": [
"(^|/)environment.yml$"
],
"matchStrings": [
"# renovate datasource=conda\\sdepName=(?<depName>.*?)\\s+- [a-z0-9]+==\"?(?<currentValue>.*)\"?"
],
"datasourceTemplate": "conda"
}
]
} The job of the package manager is to do the discovery/annotation that I show above automatically so that you don’t need any configuration in the default case. |
Thanks for the extra information and the example config. With this I managed to get a PR triggered for an update of a conda environment.
And 2 PRs for 2 packages that I enabled: https://github.com/mdehollander/orochi/pulls To get the environment files in subfolders recognized I changed the regular expression in the config to:
Looks very promising! Thanks! Looking forward for a manager :) |
@morremeyer I am wondering if you managed to work on automatic discovery of conda packages via the package manager. That would make the use on existing installation easier, because you don't need to annotate the env files :) |
Hi, any progress on conda support via the package manager? I would like to mention/promote this in a presentation and if there have been any updates it would be nice to know. Thanks. |
we have the Versioning and Datasource for python(also for conda datasource now), In fact we can do this using regex managers for now i think, if not we will need to start with the datasource as Rarkins suggested |
Hey everyone, sorry for not updating this sooner. I looked into the conda versioning spec a while ago and have a short write-up of what my plans would be, but somehow forgot to post that here. I'm out of office right now and set a reminder to update this issue once I'm back, which should be by the end of this week. |
@PhilipAbed I'm not sure if the python datasource supports the conda versioning spec. Everyone, here's what I thought I had posted back in May when I last got to work on this (time really flies currently): We worked on this over the last few days, so here is a summary and the plan for how I’m thinking about proceeding here. There is no working manager yet that could be tested as the focus was on two things:
As the conda MatchSpec enables complex definitions of package versions, it’s vital to parse them correctly according to the specification so that we don’t create wrong updates. A MatchSpec can match on practically every attribute of a package, so we need to focus on the common cases that can be correctly implemented without hacking the renovate core. I discussed a lot of ideas with and got input by @rarkins, big thank you for that! Here’s what is planned to be supported in the first iteration of a conda manager:
The first iteration will have at least the following limitations:
This means that specifications like the following should be supported:
More complex cases can then be added over time. I will be working on this the week after next (likely 2022-08-10 and 2022-08-11), I'm happy about any feedback here! EDIT for additional resources: |
Thanks for the update, looks promising! good work |
What is the status of the PR? I see that the last update was a couple of months ago. Would be happy to help get it to the finish line! |
there's no more work from PR author after suggestions. so he's maybe no longer working on it. feel free to take it over by create a new PR with requested changes. |
Sounds good, thanks! |
Hey everyone! Sorry for the silence over the last months. We still have this on our agenda, but didn't get to work on it in recent weeks. Working on implementing the manager, we realized that we will first need to implement conda versioning to implement a manager that works reliably and consistently. We definitely have not abandoned this, but as stated initially we cannot give guarantees for progress here. If anyone else wants to contribute to this, we're more than happy to put in our feedback for PRs. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Hi @morremeyer 👋 Do you want to try (again) to get You tried getting We closed your PR because it was getting stale. You also said that you now have a better plan for the second attempt at the PR. I don't know if you still know about the better plan, and still want to try again. 😉 |
I‘m still planning to do this, but also still can not make any promises as to when. But this is definitely not forgotten! |
I have been looking at this for https://pixi.sh . @morremeyer You mentioned that you "realized that we will first need to implement conda versioning to implement a manager that works reliably and consistently". I was wondering if it would help if we package |
@baszalmstra That depends on what the maintainer's opinion on packaging other things. Our past discussions on that topic have resulted in "prefer an implementation in renovate". I don't think that would be overly complex, I just did not get to work on it yet. |
Are you planning on implementing the entire version specification with all edge cases? From my experience this can get pretty hairy. And are you planning to implement both version spec and match spec? @rarkins What is your stance on depending on external packages for these things? |
I'm fine with using any npm package if it's cross-platform |
@baszalmstra I would start implementing the most common cases and then extending it. However, if we don't have to do that, it makes the versioning a lot more simple. Then, what would be left would be possible improvements to the datasource and the implementation of a manager. |
I'm fine with a wasm module which is distributed as npm package, so we only need a slim JavaScript wrapper for the versioning interface implementation. |
https://conda.io/docs/
The text was updated successfully, but these errors were encountered: