How to train a custom prior model from scratch in REINVENT 4.7 to support unallowed tokens?

[ZevChan]

I'm new to the REINVENT project and currently encountering an issue where my SMILES sequences contain specific tokens (particularly the phosphorus atom "P") that are not supported by the pre-provided priors in REINVENT. I have an extensive database of over 3 million molecules, which fully covers all the tokens required for my research.

I have attempted to standardize the SMILES using RDKit and even converted them to SELFIES/DeepSMILES formats, but none of these approaches resolved the problem. The system consistently returns the error message:"Allowed tokens are: ({'c', '[n+]', '[nH]', '7', '5', '%10', '$', 'N', '-', 'n', 'O', '6', '=', '4', 'o', '3', '9', 'Br', '^', '8', 'Cl', '[N+]', 'F', '[S+]', '(', ')', 'C', 'S', '1', '[N-]', '2', '#', '[O-]', 's'}, set())"

I'm now using the provided transfer_learning.toml as a template, and my TOML configuration is as follows:
transfer_learning.txt

My questions are:
Is it possible to train a prior model from scratch in REINVENT using my custom dataset to support phosphorus-containing molecules?
What is the recommended format for preparing my 3-million-molecule database, and are there any specific preprocessing steps required?
Are there any modifications needed to my current TOML configuration to enable scratch training with custom tokens (e.g., "P")?
I noticed there was a previous discussion about methods for training a new prior to handle phosphorus atoms (#126). Has there been any progress or updated guidance on this topic since then?

REINVENT Version: 4.7

Any documentation, examples, or pointers to relevant code would be greatly appreciated!

[halx]

Hi,

if it is only about supported elements, I could provide you with an (experimental) PubChem based prior which may solve your problem.

Principally, REINVENT only supports SMILES. You can use our data pipeline through reinvent_datapre to prepare the SMILES. If that interests you, I can provide further details.

#126 referes to LinkInvent. I guess you are planning to train a new classical, de novo Reinvent prior.

Many thanks,
Hannes.

[ZevChan]

Hi Hannes,

Thank you so much for your prompt and helpful response!

To clarify, I’m not entirely certain if the issue is solely with unsupported elements (like phosphorus "P") or if most of the tokens in my custom dataset are not compatible with the default prior. Currently, I’m stuck at the initial stage where the system rejects my SMILES due to unrecognized tokens, so testing with your PubChem-based prior would be incredibly valuable to pinpoint the root cause.
I would be extremely grateful if you could share the (experimental) PubChem-based prior with me.

Thank you again for your generosity and patience—I really appreciate your support!

Best regards,
Zhongwei

[halx]

You would need to provide me with an online storage where I can drop a 100MB binary file.

[ZevChan]

I would like to extend my sincere gratitude for your generous assistance and for providing the experimental PubChem-based prior model. I'm pleased to inform you that it has successfully resolved my issue.

The prior you shared covers most of the tokens in my custom dataset. After filtering out the unsupported tokens, I still retained over 80% of my original data. Thanks to your help, I am now able to generate molecules successfully using REINVENT.

Your support has been invaluable in moving my research forward. I truly appreciate your expertise and willingness to help.

Thank you once again for your time and guidance.

[16:32:25 Using generator Reinvent
16:32:25 Writing sampled SMILES to CSV file FR_input_molecules_sampling.csv
16:32:25 Sampling 394000 SMILES from model priors/pubchem20250312.prior
16:32:26 Sampled SMILES C(C)Oc1c2c(cccc2)c(c2CC(=Cc2c1OCC)CC)OCC failed to sanitize.
16:32:26 Sampled SMILES c12c3c4c(cc2)cccc4ccc3c(n(Cc2ccc(Cl)cc2)c1=N)=O failed to sanitize.
16:32:26 Sampled SMILES c1ccc2c(c(c(C(NN=Cc3ccc(cc3)C)=O)[nH]c2c1)=O)O failed to sanitize.
16:32:26 Sampled SMILES C(CCc1cc2c(N=Cc3c4n(c[nH]c4ccc23)CCCC(O)=O)c(F)c1)C failed to sanitize.
16:32:26 Sampled SMILES c1(-c2c(cccc2)N(c2c(C3=CCN(CC=3)c3c(Cl)cccc3)nc3ccccc3n2)C)ccccc1 failed to sanitize.
16:32:26 Removed 11 invalid SMILES
16:32:26 Finished REINVENT on 2025-11-24]

[bryanyeo2108]

Hi Hannes,

I'm currently facing a similar problem as Zhongwei regarding the phosphorus atom. Would it be possible to also receive the experimental PubChem-based prior through my email: bryanyeo2108@gmail.com as it would greatly help in my research.

Thank you for your time and understanding!

Best regards,
Bryan

[halx]

https://drive.google.com/file/d/1rOwWruKtWHo5jyJ7s1-4guOSg_IiX_OK/view?usp=sharing

[aalexpersson]

https://drive.google.com/file/d/1rOwWruKtWHo5jyJ7s1-4guOSg_IiX_OK/view?usp=sharing

Hi Hannes,

Do you know how large and which version of the PubChem dataset you used for training this prior?

Thanks,
Alexander

[halx]

Hi,

You can see this in the metadata which is printed to the log file.

Cheers,
Hannes

[GuidoKirsten]

I'm facing the same problem. I tried to create a model using REINVENT4/reinvent/runmodes/create_model/reinvent.toml but got a couple of error messages. Would it be possible to either
replace the toml file by a working version
or get a toml file that can be used to create an empty model from an arbitrary dataset/SMILES flavour?
Can that model be used for re/lib/linkinvent and mol2mol? If not which additional steps are required?
Best
Guido

[halx]

Hi,

many thanks for your enquiry.

You would need to be more specific as to what the actual error messages are. Models are specific to the generator i.e. you can not mix, say, a Reinvent prior with a Mol2Mol prior. After you have create and "empty" model you would need to carry out transfer learning (TL) with a dataset suitable for the respective generator.

Many thanks,
Hannes.

[GuidoKirsten]

Hi,
I attached a screenshot of the errors I got. The toml contains:

[io]
smiles_file = "gmp.smi"
model_file = "empty.prior"
[network]
num_layers = 3
layer_siye = 512
dropout = 0.0
layer_normalization = false
standardize = true
[metadata]
data_source = "gmp_inhibitors"
comment = "this is a test"

the gmp.smi file contains one SMILES per line.
The purpose of my exercise is to create empty models with different network options based on a SMILES generator of a non-RDKit sorftware.
Best
Guido

[halx]

Hi,

model creation is not supported by the reinvent tool. You'll have to use reinvent/runmodes/create_model/create_reinvent.py.

Cheers,
Hannes.

[GuidoKirsten]

Ok, got it. I'll give it a try

[GuidoKirsten]

Sorry but I ran into the next problem. Creating the empty prior with:
python create_reinvent.py create_empty.toml gave me a prior file. But when I try to use it in transfer learning I get:
13:39:51 C:\Users\guido\empty.prior does not contain a hash ID
and the TL falls back to the default prior. So the question is how do I get a prior that I can use?

[halx]

That's just a warning. For an empty model it doesn't make much sense to compute a hash ID. There is no fall-back mechanism in place and REINVENT will use your empty model.

[GuidoKirsten]

Sorry for being such a pain :-) I created the empty model with standardize = false to allow elements that are otherwise filtered out by the default filter.
When I run TL with that model I see:
14:44:41 C:\Users\guido\empty_nonstandard.prior does not contain a hash ID
14:44:41 Number of network parameters: 5,805,602
14:44:41 Network architecture:
RNN(
(_embedding): Embedding(34, 256)
(_rnn): LSTM(256, 512, num_layers=3, batch_first=True)
(_linear): Linear(in_features=512, out_features=34, bias=True)
)
14:44:41 Using generator Reinvent
14:44:41 Applying filter default to input SMILES
14:44:42 "default" filter: Ic1c(N)cccc1 is invalid

So Reinvent is using the default filter in transfer learning and thus ignores that the prior was created with standardize set to false. That's why I lose the compound containing iodine. It would be nice if I could disable the filter in TL/RL.

Best
Guido

[halx]

Creating an empty model with standardize = false is ok provided you have preprocessed the SMILES and standardized with RDKit. In TL you can then run with standardize_smiles = false and randomize_smiles = true (recommended!) to by-pass the old chemistry fitler.