Models and Metrics for Molecular Generation

[HFooladi]

I am going to implement some of the generative models (VAE, AAE, ...) for molecular generation in deepchem. For this purpose, I think moses library can be a good starting point. They have benchmarked different generative models and provided a set of metrics to evaluate the quality of generated molecules.

I think at first, I will try to add one or two models to the deepchem library. Moreover, I am going to add most of the metrics for evaluating quality.

I have started this issue to track my progress along this path.

[rbharath]

@HFooladi Awesome! Feel free to ask questions or raise discussions here :)

[HFooladi]

Current metrics for evaluation of generated molecules are:

• Validity (if the output is SMILES, check whether it is valid or not, both syntactically and semantically)
• Frechet ChemNet Distance (FCD)
• Fragment similarity
• Scaffold similarity
• Similarity to nearest neighbor (SNN)
• Diversity (which is based on Tanimoto similarity)
• Distribution difference for logP, SA, QED, weight
• Novelty (molecules not present in the train)

You can find the precise definition of all these metrics in this paper.

I will begin with the simple ones and gradually add more difficult ones.
@rbharath @ncfrey If you are aware of other metrics, please add them to this list.

[ncfrey]

Great list! You should be able to check validity for any output by transforming it into a SMILES.

The only two metrics I'd add are from the GuacaMol paper:

• Uniqueness: standard metric for penalizing models that generate duplicate molecules
• KL divergence: this might be the same as the "Distribution difference" you mention. It measures the difference between distributions of some physicochemical descriptors for the training and generated sets.

As you implement these, feel free to try them out using the generative model in the Normalizing flow tutorial. It would be great to update the tutorial with the new metrics to show how they can be used.

[PARODBE]

Hi @HFooladi, How was the smiles generations with MOSES? Did you obtain good results?Do you have some code for practice? Thanks!

[HFooladi]

Hi @PARODBE. Thanks for your comment. I have implemented locally some of these metrics for evaluating the performance of generative models. I have a plan to refactor it a little bit to make it compatible with deepchem style and will open a PR hopefully next week. But, I have implemented mostly the metrics, not the generative part.

If you are looking for a tutorial on the generative part, you can look at this tutorial by @ncfrey which has used normalizing flow for the molecule generation.
https://github.com/deepchem/deepchem/blob/master/examples/tutorials/Training_a_Normalizing_Flow_on_QM9.ipynb