Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review of model registry #321

Open
ablaom opened this issue Oct 14, 2020 · 1 comment
Open

Review of model registry #321

ablaom opened this issue Oct 14, 2020 · 1 comment
Labels
help wanted Extra attention is needed
Projects

Comments

@ablaom
Copy link
Member

ablaom commented Oct 14, 2020

I think it would be useful for someone with appropriate expertise to review how the model registry is currently implemented. (I think the user interface is basically fine .) This is part of the code that was written very early when there were many competing priorities and I feel there is room for improvement.

The main complaints I have are:

  • tedious to update. For example, when adding a new model-providing package, one must artificially force precompilation of MLJModels (see Find a way to load metadata from __init__ #74). And with so many packages, this takes a long while, although this is not too big a deal, at least for now.

  • A bit fragile: In principle anyone can follow the instructions but in practice small issues arise, which I can fix but which a newcomer might scratch their head over. The process would be hard to automate (using CI, for example) without some improvements.

  • lacking some flexibility in terms of the kind of metadata that can be serialised. Currently we basically do our own serialization into a toml file, which has worked so far, but for maintenance and extra generality, it might make better sense to use an external package or julia serialiaztion for this. Maybe.

Some other feedback about the system: #169

I believe @vollmersj had the very good idea to borrow the approach used in the julia package registry. I did not and still do not have enough familiarity with this system to take this approach but could indeed be best approach. In any case, someone with experience with that code base would presumably be in a very good position to propose a better design.

Maybe Pkg Artifacts could play a role??

@ablaom ablaom added the help wanted Extra attention is needed label Oct 14, 2020
@ablaom ablaom added this to priority low / difficulty high in General Jul 14, 2021
@ablaom
Copy link
Member Author

ablaom commented Oct 27, 2022

Other issues:

  • all packages have to be loadable at the same time; so if one is behind in a compat update, this can preclude updating some other package's metadata

  • currently models are found by looking for subtypes of Model. In the future, this will be a trait ismodel and this approach won't work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
General
priority low / involved
Development

No branches or pull requests

1 participant