[Project]: Incorporate the REINVENT generative model in the Ersilia Model Hub

[miquelduranfrigola]

Summary

REINVENT is one of the most successful reinforcement learning models in chemistry. Version 4 of REINVENT has been released recently: MolecularAI/REINVENT4

As we populate our generative chemistry tool (ChemSampler), it is becoming increasingly clear that REINVENT should be amongst our collections of generative tools.

We have used REINVENT (Version 2) in the past, with great success in the context of Open Source Malaria. See, for example, this repository. In our experience, the best way to use REINVENT is through Jupyter Notebooks, including tinkering with parameters and a lot of visual exploration. Unfortunately, ChemSampler is more geared towards automation and, therefore, we should come up with new settings to run REINVENT robustly without much intervention from the user.

This is a highly interesting project for many reasons:

1. It tackles the task that has the highest demand at Ersilia, namely, generative AI for chemistry.
2. It uses reinforcement learning, which is underrepresented in our hub currently.
3. If successful, this would be a publishable project in standalone form.

Scope

Initiative 🐋

Objective(s)

1. Incorporate a generic REINVENT4 model in the Ersilia Model Hub.
2. Incorporate at least 5 fine-tuned REINVENT4 models in the Ersilia Model Hub (to be determined, for example: malaria, tuberculosis, hiv, synthetic accessibility...)
3. Sufficient usage documentation.
4. Write a scientific publication.

Team

Role & Responsibility	Username(s)
DRI / Lead Developer	@ankitskvmdam
Supervisor	@miquelduranfrigola

Timeline

TBD

Documentation

• REINVENT: https://github.com/MolecularAI/REINVENT4
• ChemSampler: https://github.com/ersilia-os/chem-sampler
• ChemSampler GitBook: https://ersilia.gitbook.io/ersilia-book/chemistry-tools/sampling-the-chemical-space
• Other generative models at Ersilia: eos9taz, eos4qda

[miquelduranfrigola]

Hi @ankitskvmdam,

Let's use this issue to update on progress related to this project.

Some takeaways today's meeting (29/01/2024):

• We will meet every Monday at 1pm CET.
• I will add you in Ersilia's Slack workspace.
• Let's work on REINVENT 4, if possible.
• Let's not get distracted by the OSM project, previously conducted by Ersilia using REINVENT 2.
• Overall objective for the first week is to get familiar with REINVENT and test at least one of their notebooks. Please pay especial attention to computational demands (including the need for GPU).

[miquelduranfrigola]

Hi @ankitskvmdam,

Thanks for a useful meeting today (05/02/2024). Some actionables:

• As agreed, let's focus on one REINVENT 4 model, I would suggest the Mol2Mol (medium similarity) model. The advantage of this model is two fold: first, it takes one molecule as input and it generates multiple molecules as output; second, it has a very simple default config file with reasonable defaults (e.g. temperature = 1).
• Let's focus on incorporating this model following the model incorporation workflow that we designed a while ago. In brief, please just open a new model request issue, fill in the form and I will /approve the model. This will create a new model repo based on the template, which you can then fork, etc. Please let me know if you have any questions!.
• Importantly, let's start with CPU-only runs. Later on, we will come up with a way of having Docker containers on GPU as well. But let's go step by step.

Thanks a lot!

[ankitskvmdam]

@miquelduranfrigola Here are some molecules generated by the ‘reinvent 4 mol2mol_medium_similarity.prior’.

To view more you can visit:

1. https://github.com/ankitskvmdam/eos694w/blob/main/notebook/Playground.ipynb and scroll at the bottom, to see the output of the notebook
2. https://github.com/ankitskvmdam/eos694w/tree/main/notebook/output to see all the png files of the generated molecules.

[miquelduranfrigola]

Hello @ankitskvmdam this is going the right direction!

Generally speaking, output molecules are very similar to the input ones. This is OK, but it means we need to generate more candidates.

May I ask how many unique molecules are you able to generate, on average? Can we obtain ~100 molecules or so? If so, what is the computational cost?

Also, in the notebook I have noticed one instance where the output is equal to the input (second row of the image). In my opinion, we should discard these cases. So, we need to be sure that:

• All output molecules are different (similarity < 1) than the input.
• All output molecules are uniqut.
  I hope this makes sense?

[ankitskvmdam]

Hi @miquelduranfrigola ,

Yes it makes sense.

I have updated the code, now we are only saving unique molecules.

I have also created a notebook to test the performance: https://github.com/ankitskvmdam/eos694w/blob/main/notebook/TestPerformance.ipynb
https://github.com/ankitskvmdam/eos694w/blob/main/notebook/TestPerformance.ipynb

On average it generated 97 unique molecules (if we ask for 100). Sometimes it generates invalid molecule and sometimes molecule similar to the input, in both cases we are discarding them.

Also, on average it takes 33 seconds to generate 100 molecules.

Update the format in which we are writing the output. Visit https://github.com/ankitskvmdam/eos694w/blob/main/notebook/output/meph.smi https://github.com/ankitskvmdam/eos694w/blob/notebook/notebook/output/meph.smi to check. The format is as follows:

1. The first row is the header.
2. The second row is for generated smiles.
3. The third row is for input smiles.

[miquelduranfrigola]

Thanks @ankitskvmdam,

The TestPerformance.ipynb notebook is perfect; very useful.

In the ouput file, we do not need the third row (input smiles repeated). We can talk about it later today. In the case of <100 unique molecules being generated, let's add empty values at the end until reaching the 100 columns.

In today's meeting, I suggest we do the following:

1. Make sure we have everything ready to incorporate model eos694w in the hub. I know you had issues with versioning, but @DhanshreeA and I will take care of it if necessary.
2. Evaluate critically the molecules generated.
3. Decide the next REINVENT model to include, hopefully adding more nuance to the task 😃

[ankitskvmdam]

Hi, @miquelduranfrigola,

Other Reinvent4 Models:

1. LibInvent: This model helps you find R-groups that can be attached to specific scaffolds. It requires one scaffold per line and each scaffold marked with two asterisks ("*") to indicate the attachment points. Example scaffold smile [*:1]Cc2ccc1cncc(C[*:2])c1c2
2. LinkInvent: This model helps you in identifying a linker or scaffold that can connect two different fragments. It requires each line to specify a pair of "warheads" separated by a pipe symbol ("|"). Each warhead needs to be marked with an asterisk ("*") to indicate the connection points. Example warhead smile: Oc1cncc(*)c1|*c1ccoc1
3. Molecular optimization: Mol2mol mmp, Mol2mol scaffold, Mol2mol scaffold_generic. All these models will generate new molecules similar to the provided molecules.

[miquelduranfrigola]

Thanks @ankitskvmdam - I would start by exploring mol2mol scaffold and/or mol2mol scaffold_generic. Could we give it a quick go to see how they work in practice?

[ankitskvmdam]

Sure, I will explore them.

[miquelduranfrigola]

Perfect. The second most interesting one would be LibInvent. But for this, we'll have to:

1. Given a molecule, get its scaffold (using for example RDKit or Datamol)
2. Given a scaffold, identify its attachment points (i.e. the asterisk). @Inyrkz has done some interesting work on this, so I am looping him in.