RDKit 2021.09 compatibility

[adam-of-barot]

Upon updating to RDKit 2021.09.02 from 2021.03.05 I can no longer perform structure based lookups.
I tried performing .filter() queries both from the frontend and the backend, but both resulted in the same error.
The cartridge itself still works, as I was able to do smiles based raw SQL queries.

Django gives the following error:
"DataError: problem generating molecule from blob data"

The top level traceback:

~/venv/lib/python3.8/site-packages/django/db/backends/utils.py, line 85, in _execute
78    def _execute(self, sql, params, *ignored_wrapper_args):
79        self.db.validate_no_broken_transaction()
80        with self.db.wrap_database_errors:
81            if params is None:
82                # params default might be backend specific.
83                return self.cursor.execute(sql)
84            else:
85                return self.cursor.execute(sql, params)

For the time being I reverted back to RDKit 2021.03.05.

[rvianello]

@adam-of-barot the problem sounds like could be related to a mismatching RDKit version (for example trying to unpickle a molecule with an older RDKit version), but I have the impression I am missing some information. Where did you update RDKit, in the django application environment, on the postgres server, or on both? And (sorry in case it should be obvious) with frontend and backend, what are you more exactly referring to, in this context? A more detailed description of the query you are performing I think would be also helpful to me.

[adam-of-barot]

@rvianello I updated RDKit in the conda environment that the Django application uses. I did not upgrade anything related to the cartridge. Running the SQL command SELECT rdkit_version(); gives me 0.73.0.

By frontend I meant that I tried performing the searches through a browser with the web pages that I've built, and by backend I meant performing the searches through the command line with python manage.py shell.

Here is a query that just tried to run through the aforementioned shell as a test:

In [1]: from app.models import ReagentStructure

In [2]: r = ReagentStructure.objects.last()

In [3]: print(r.molecule, r.smiles)

Out[3]: <rdkit.Chem.rdchem.Mol object at 0x00000212855768B0> NNC(=O)OCc1ccccc1

In [4]: ReagentStructure.objects.filter(molecule=r.smiles)  # search for an already existing molecule

Out[4]: ---------------------------------------------------------------------------
DataException                             Traceback (most recent call last)
~\Miniconda3\envs\app\lib\site-packages\django\db\backends\utils.py in _execute(self, sql, params, *ignored_wrapper_args)
     83             else:
---> 84                 return self.cursor.execute(sql, params)
     85

DataException: problem generating molecule from blob data

I also attached the full traceback here: traceback.txt

[rvianello]

Thank you for the additional details. I browsed the rdkit repository for some information, and I think version 0.73.0 refers to the 2018.03 RDKit release, implying that the postgres database is running on a version of RDKit that is now quite older than the one available to the django application.
I think the problem is probably that, because django-rdkit sends the molecule argument in binary form, the database cartridge may be unable to deserialize the data sent by the application and reconstruct a molecule instance (this is what the "problem generating molecule from blob data" error message refers to).

[adam-of-barot]

I did some testing on my home linux system to check things with the out-of-date and up-to-date versions.

I tried out a fresh conda environment installation of RDKit (v2021.09.03), django-rdkit (v0.3.1), and in another fresh conda environment installed rdkit-postgresql (v2021.09.02) and made the necessary database setups.
I also made a quick test Django model with a single MolField.

SELECT rdkit_version(); gives 0.76.0.
A quick query of TestModel.objects.filter(mol__hassubstruct='CCCCC') on an empty database gives a proper empty QuerySet.

I then made another environment, this time having rdkit-postgresql v2018.03.03. After setting up the database the same way, running the same query above gave the "problem generating molecule from blob data" error.

So yeah, the problem is definitely the out-of-date RDKit version on the database side.

Thank you for looking into it @rvianello !