RDKit minimal lib (JavaScript Lib) next steps & more

[MichelML]

Hi @greglandrum , in the spirit of improving on #4086 , here are some next steps I've been meditating on to improve JS usage and stronger presence in the RDKit ecosystem (and more).

Is your feature request related to a problem? Please describe.
RDKit minimal lib (JS Lib) build could be moved from the main repo for separate releases - to gradually improve and modernize usage and examples, while always remaining bound to an RDKit release (think of having versions of the rdkit minimallib js on npm looking like 2021.3.1-release1 2021.3.1-release2 etc). Modernizing usage and examples could mean:

• Gradually add better examples with React and other modern web frameworks
• Add examples on how to deploy as aws lambda (see very small example here https://github.com/MichelML/aws-sam-rdkitjs )
• Have React components ready to use out of the box (already have some that I could open source here with minimal refactoring)
• Improving/optimizing the wasm build .

Describe the solution you'd like

• See section above, but the starting idea would be to have a dedicated repository responsible for building the RDKit minimal lib js distribution, so that there can be JS releases deployed on top of RDKit main releases, and that those releases could be tracked with its own CI/CD.
• Another repository could be added to add aws lambdas templates:
  1. One for the JS minimal lib https://github.com/MichelML/aws-sam-rdkitjs
  2. Potentially another one for the main RDKit project see example here https://github.com/MichelML/sam-rdkitpython

Describe alternatives you've considered
Keeping everything in main repository.?

Additional context
None.

---

All thoughts welcome 🙏

[MichelML]

@greglandrum should I open specific issues for this or it should start here as a discussion like this?

Let me know what's best, have a good day!

[greglandrum]

I think the general discussion is a good place to start.

[greglandrum]

I agree that having a separate repo for doing builds/distributions of the MinimalLib JS wrappers makes sense. That's the same model that we were using with the rdkit conda builds. It might be worth adding a build of the JS wrappers to the main repo CI setup, but that would only be to make sure that code changes don't break them (like the rest of the CI builds). So that's sticking with the model that the main repo is really only doing CI, not CD.

I've already created a repo for doing CI/CD builds of the RDKit CFFI wrappers (also done on top of MinimalLib) here: https://github.com/greglandrum/rdkit-minimallib-build. I haven't set that up in the RDKit org yet because I wanted to try it out first, but I'll be transferring it in the near future. It needs to be reorganized a bit first - the current setup is more or less completely focused on getting the CI/CD stuff working, but the repo could also theoretically includes space for examples of things like using the CFFI wrappers in other languages.

So, yeah, I think it'd be great to have a repo which we can use for CI/CD around the JS wrappers and which includes examples and documentation. I'd be happy to coordinate with you on that if you want to set something up and transfer it into the RDKit org.

@eloyfelix has also set up something similar to handle CI/CD of Julia wrappers around the CFFI code, so he may also be interested in thinking about this topic.

[MichelML]

Thank you @greglandrum

I will start something on my side and tag you here with a link to the repo when it's done

[eloyfelix]

Hi. Looks like we're giving you a lot of work related to the MinimalLib @greglandrum!

I think that we should approach it in a way that the users of each programming language find what they expect to find. In my case with Julia I tried to do everything the "Julian" way and this implied sending a PR with a recipe to build RDKit (RDKit_jll) to their repository of recipies + setting a different repo with the wrappers to that library. I also tried to use the tools I saw the most pouplar repos are using (Documenter.jl, Test.jl, Codecov, GitHub Actions for CI...). For the examples I set a jupyter notebook using binder because I think that it works well for this kind of minimal examples.

The good thing about this approach (which is possibly the only one!) is that if someone, for example, decides to build their own cheminformatics package using the MinimalLib only to parse the molecules (as you suggested in your post) they will have enough installing RDKit_jll.

I understand that this might work completely different in the JS case. In any case, I agree that having a different repo for it is what makes more sense.

For the versioning my plan was to stick to anything appearing in Releases keepeing the same exact RDKit released version so every time there is an RDKit release I can send a PR to the recipe repository to update it with the latest archive (e.g. https://github.com/rdkit/rdkit/archive/refs/tags/Release_2021_03_1.tar.gz). I'm currently using Git as a source because there is no public release with the CFFI yet.

Regarding the organization I'm happy to move the wrappers to any organisation it can make the package look better. RDKit seems the most natural one.

Finally, I'm also happy to change the license of the wrappers package. I don't think should be any issue with MIT (I think that both are almost identical) but I could change it to RDKit's BSD-3 if preferred.

[greglandrum]

@eloyfelix : I have no strong preference about where the projects live. If you think it'd help to have your Julia wrappers in the RDKit org, I'm happy to host them there.

Finally, I'm also happy to change the license of the wrappers package. I don't think should be any issue with MIT (I think that both are almost identical) but I could change it to RDKit's BSD-3 if preferred.

MIT is completely fine.

[eloyfelix]

@greglandrum yes, I think that it may make them look more serious under the RDKit org.

The wraps are complete after your lastest commit improving MolStandardize aspects btw (made a new build and added Standardizer tests). I basically only need to improve a bit the docs and write a couple of more examples and that will be it for now. Happy to hear of any suggestion you may have.

[MichelML]

Hi @greglandrum ,

To continue on #4124 (comment) .

I have started a small repo https://github.com/MichelML/rdkit-js to explore how we could organize this (and it builds the same way as the current npm build in the main rdkit project (see beta version built from the new repo here.

The way I see how we could organise things is this:

1. Keep what is building and testing the distribution artifacts in the main RDKit repository.
2. Have the docs, the npm build and release process, and modern web examples in the rdkit-js repository.

I have two good reasons and one bad reason to do it this way:

---

The first good reason is that people coming from the JavaScript world will be more comfortable contributing to a repository looking like a regular npm package/project, without the artifacts related to the C++/WASM build.

The second good reason is I think it makes sense to keep what is closely knitted to the main RDKit project, at pre-build time, inside the main project itself. As a start, the new rdkit-js repository would only be responsible to handle 1) the npm build and release with git tags -- still following the main rdkit releases while being able to release multiple versions for the same rdkit release as new examples come along, 2) good documentation for JS usage, and 3) examples on how to integrate rdkit-js in modern web application and frameworks (see issues https://github.com/MichelML/rdkit-js/issues ).

The bad reason is I am a beginner at C++ (currently learning it in my spare time), and this vision reflects my reticence to move everything related to the minimallib under a new repository.

---

Let me know if you have other ideas. If you agree with the seperation of concerns, here would be my next moves:

1. Make a PR to remove what has been moved to https://github.com/MichelML/rdkit-js (which will be moved under the rdkit org)
2. Edit the Minimal lib readme in the main rdkit project to redirect to https://github.com/MichelML/rdkit-js (which will be moved under the rdkit org)
3. Move https://github.com/MichelML/rdkit-js under the rdkit organization
4. Start writing (ready to use) examples for modern web frameworks (see open issues here https://github.com/MichelML/rdkit-js/issues )
5. Add rdkit-like CI/CD pipeline in the repo (like you did for CFFI https://github.com/greglandrum/rdkit-minimallib-build )

Looking forward to hearing your thoughts 👍

[MichelML]

Thanks a lot guys, will continue progress and let you know when I think I have a first version ready to be transferred to the main RDKit github organization. ETA in maximum two weeks.

[MichelML]

Quick update on this, still work in progress but you can see the most up to date demo here https://michelml.github.io/rdkit-js/examples/react-example/build/index.html

[greglandrum]

@MichelML : that demo looks soooo much better than the things I put together! Thanks for sharing :-)

[MichelML]

Glad you like it @greglandrum , more to come hehe

[MichelML]

Minor update to the website structure -> https://michelml.github.io/rdkit-js/examples/react-example/build/#

still not done with the examples, but should be complete soon

[MichelML]

@greglandrum @DavidACosgrove anyway we can do multi-color, multi-substruct highlighting with rdkit minimallib for javascript at the moment? It seems we can't but I might miss something

Also, have you ever tried to do client-side similarity search with the minimal lib (not just substructure search)?

[DavidACosgrove]

Hi @MichelML. I have done multi-substructure searching. I’ll dig the code out later and post it. I’m less sure about colours.

[MichelML]

Thanks David, the question was mostly for multiple colours , the multi substruct search is actually implemented in the examples now https://michelml.github.io/rdkit-js/examples/react-example/build/

[greglandrum]

@MichelML : the current interface in MinimalLib doesn't expose the options needed to do multi-color highlighting.
That is certainly a possible future extension. Would you mind creating an issue for that enhancement so that it doesn't get lost?

[MichelML]

Will do @greglandrum

[MichelML]

here it is @greglandrum #4247

[MichelML]

@greglandrum Just an update that this is still in progress https://michelml.github.io/rdkit-js/examples/react-example/build/

I think I'm mostly done with example for the React example besides improving the documentation along side the examples. If you check out the additional options example, i think it's pretty cool :D

Will post further progress soon

[MichelML]

New progress:

A V1 of your (restyled) initial vanilla JS getting started is available here @greglandrum https://michelml.github.io/rdkit-js/examples/vanilla-javascript/

A V1 of React example is available here https://michelml.github.io/rdkit-js/examples/react-example/build/

A V1 of React-Redux is available here https://michelml.github.io/rdkit-js/examples/react-redux-example/build/

---

My goal from there is to consolidate the above efforts plus address issues related to packaging utilities with the vanilla RDKit.js in the npm package (see issues here https://github.com/MichelML/rdkit-js/issues )

Hopefully, after that and a revision of the whole README, the repository will be transferable to the RDKit GitHub org for better visibility

Thoughts welcome!

[MichelML]

Small update, still working on the v1, it has fallen low in my priorities but will get back to it very soon

[MichelML]

Still in progress, and sorry for the long delays, it is still my intention to finish a v1 of a dedicated repository for rdkit-js

will get back to this https://github.com/MichelML/rdkit-js soon enough

[MichelML]

Hi there @greglandrum @ptosco (sorry for the long message) !

I think I now feel comfortable enough with the state of https://github.com/MichelML/rdkit-js to consider transferring it to the rdkit github organization 🎉

Obviously, it also depends on your vision and approval, so here are my suggestions and discussion points for next steps:

1. First of all, should we find another name for it as there already is a legacy rdkit.js repository in the rdkit org? It's the name that makes the most sense to me, but curious to have your thoughts.

2. Any suggestions, questions or comments on https://github.com/MichelML/rdkit-js would be greatly appreciated from you two. If you're able to go through the READMEs and websites and find it a coherent/comprehensible good start to what's meant to be a maintainable added value to the main RDKit repo and current minimallib initiative , then I will be happy. Some particular things to check for:

• License
• All READMEs, see Contributing section for sub-readmes.
• Potential new npm versioning scheme explained here
• Azure pipeline for semi-automated releases here
• All deployed examples on the website rdkitjs.com and react.rdkitjs.com

3. In terms of licensing, I was wondering if you would be open to add Valence Discovery (the company I work for) to the names displayed in the License file. I think the file might be outdated at the moment, and I'm not an expert in License files, but here would be a suggestion:

current state

BSD 3-Clause License

Copyright (c) 2006-2015, Rational Discovery LLC, Greg Landrum, and Julie Penzotti and others
All rights reserved.

....

suggested change

BSD 3-Clause License

Copyright (c) 2006-2022, Rational Discovery LLC, Greg Landrum, and Julie Penzotti and others.
Copyright (c) 2022, Valence Discovery Inc
All rights reserved.

....

Or something like that. It's not a hard request, I don't mind not adding Valence, but that would be nice 👍 .

4. This was already discussed with @ptosco a bit here, but I think in the mid/long-term, this repository should be the mono-repo for all js efforts, leveraging the concept of yarn workspaces to make all javascript efforts and npm packages centralized at a single place. This would include @ptosco rdkit-structure-renderer for example, where we would only need to move this project almost as-is to a subfolder (yarn workspace) of the rdkit-js repository, and we could deploy the amazing interactive examples @ptosco as implemented live under rdkitjs.com (or renderer.rdkitjs.com , or any other subdomain). We could eventually have various packages for react, vue, angular components, etc as well (again in this same repository in dedicated yarn workspaces).

5. Here are a list of remaining things to do, but in my view we wouldn't have to wait for them to be completed for moving the repo to the rdkit org:

   • Adjust azure pipeline to use the new versioning scheme suggested
   • Renaming github repo path reference from MichelML/rdkit-js to rdkit/rdkit-js once the repo is transferred
   • Adjust azure pipeline, vercel deployments, and scripts folder to make sure there's no references to old MichelML repository anymore
   • Writing a rescoped readme in the minimallib source code explaining how to contribute to the wasm core module/minimallib, not how to manage the npm releases anymore.
   • The examples currently serve as informative/interactive documentation, but mid-term we should think about a system to have automated API documentation, but based on some research it doesn't seem super easy to have automated code documentation in a wasm module context like this. Issue logged here already. Think D3 docs here: a gallery of amazing examples + core API documentation available (rdkit-js is currently missing the latter).
   • Question: should the Dockerfile to build the wasm module go in rdkit-js or it should be kept in the minimallib source code?
   • Question: How should the repo be transferred? And, what will be my "access-level" to the repo once the transfer is done?

---

Waiting for your thoughts on next steps here. I'll be in vacation and unreachable next week, but looking forward to hearing from you both!

[greglandrum]

I will also need permission to manage hooks for rdkit/rdkit-js to be able to setup the Azure Pipeline it seems

Please check if you can do this now that you are a maintainer. I would assume so

[MichelML]

@greglandrum sadly it doesn't seem like I can, I still get the same error for Azure pipeline.

Vercel works and is now setup properly however.

[MichelML]

could you make me or the rdkit-js team an admin of the rdkit-js repository only (this is probably the role needed for hooks)?

[greglandrum]

@MichelML you are now an admin on that repo

[MichelML]

Thanks Greg, it works! I logged a few more things to do before an "official v1" here, as a FYI I'll be in vacation from today to July 3rd.

Happy to collect your feedback, discuss, and make necessary changes to the repo when I'm back.

Thanks for everything again 👍