Experimental support for the Decimal type

[RobertCraigie]

Why is this experimental?

Currently Prisma Client Python does not have access to the field metadata containing the precision of Decimal fields at the database level. This means that we cannot:

• Raise an error if you attempt to pass a Decimal value with a greater precision than the database supports, leading to implicit truncation which may cause confusing errors
• Set the precision level on the returned decimal.Decimal objects to match the database level, potentially leading to even more confusing errors.

To try and mitigate the effects of these errors you must be explicit that you understand that the support for the Decimal type is not up to the standard of the other types. You do this by setting the enable_experimental_decimal config flag, e.g.

generator py {
  provider                    = "prisma-client-py"
  enable_experimental_decimal = true
}

[RobertCraigie]

This is more difficult to do right than I initially anticipated.

If we just naively serialise decimal.Decimal objects then there can be a loss of precision at the database level due to a mismatch between decimal.Decimals precision and the precision at the database level.

Due to how our query builder works we actually cannot warn or error when a decimal.Decimal with a precision greater than what the database supports is encountered as precision is configurable on a per-field basis.

I do not know the best solution for this issue at this point in time.

[RobertCraigie]

Blocked by: prisma/prisma#10252

At this point in time I do not believe the workaround mentioned is a good enough solution.

I also do not want to introduce any footguns.

[RobertCraigie]

A solution I hadn't considered before for the aforementioned issue would be to simply process the data that prisma sends us and set the precision appropriate to that. However I do not know how decimal objects interact with each when the precision differs. This could also be confusing for consumers as the precision would be different for every object.

[danielweil]

How should we do currently if the Decimal type is not supported?

[RobertCraigie]

@danielweil you just can't use it unfortunately.

An alternative would be to use the String type at the database level and convert it to a decimal.Decimal yourself.

[tday]

Would you be willing to implement one of the workaround solutions while you work out the better long term fix? As is, I can't use this client at all since I cannot change the data type :(

[RobertCraigie]

@tday Considering that two people have asked for this feature I am willing to implement a workaround solution, however I am quite busy at the moment so while a complete solution might not be available for a little while I have made the necessary changes required for minimal support.

You can try it out by installing from the work in progress branch:

pip install -U git+https://github.com/RobertCraigie/prisma-client-py@wip/decimal

[tday]

@RobertCraigie Thanks for the quick turnaround! I think you need to add a quick line to the _get_sample_data function for the Decimal type as well.

Error: 
    sampled = self._get_sample_data()
  File "/Users/ayang/embark/genetic-predictor/beadchip/service/data-service/test/lib/python3.9/site-packages/prisma/generator/models.py", line 824, in _get_sample_data
    raise RuntimeError(f'Sample data not supported for {typ} yet')
RuntimeError: Sample data not supported for Decimal yet

[RobertCraigie]

@tday Sorry about that, I've pushed the fix.

[tday]

Thanks! Confirmed this works for Decimal on the WIP branch.

Given its sitting in a WIP feature branch, I was able to reduce my data type constraint down to double precision which maps to Float type in prisma/pydantic. This is cleaner in the meantime, but I'll keep an eye on this for the future.

[gakonst]

Hey folks - wonder how we can help push this forward? @RobertCraigie are there any outstanding items in your branch that need to get done? Happy to help.

[RobertCraigie]

I've spoken to the Prisma Team and while there is an open PR (prisma/prisma-engines#2660) to implement the internal feature required for us to be able to support Decimal properly they are wary about adding more data to the DMMF (essentially the Prisma AST) due to size concerns.

However, as this does seem to be a popular feature request I'll talk to them again and in the meantime I'm willing to implement a workaround until a proper solution is agreed upon.

In terms of what needs to be done before I can merge the in progress branch:

• Decide on an appropriate workaround
  • One solution is setting a very high precision in the decimal module, however, I do not know if there are performance concerns with this and it would lead to situations where the Decimal object contains a reference to a number with a higher precision than what is stored in the database which could be very confusing.
  • Another solution would be to parse the Prisma Schema ourselves to resolve the decimal precision for each field, this would work but would be very hacky.
  • Another solution would be to simply mark this feature as experimental, indicating to users that support for Decimal is not fully complete yet (I'm leaning towards this solution for now). This would mean that the client would have to be generated like this

  generator client {
    provider            = "prisma-client-py"
    enable_experimental = ["decimal"]
  }

• Define filter types (e.g. https://github.com/RobertCraigie/prisma-client-py/blob/main/src/prisma/generator/templates/types.py.jinja#L131-L154)
  • You can find out what fields to include by running prisma py dev playground --schema=tests/data/schema.prisma and looking through the GraphQL documentation on the server that is started.
• Write unit tests (e.g. https://github.com/RobertCraigie/prisma-client-py/blob/main/tests/test_types/test_bigint.py)
• Update internal snapshots (https://prisma-client-py.readthedocs.io/en/stable/contributing/contributing/#snapshot-tests)
• Update documentation (/reference/operations)

There may be other items but those are all that I can think of off the top of my head.

@gakonst Thanks for offering to help! Please be wary that adding support for a new type touches the deepest parts of this codebase, some of which are in need of a refactor and may be confusing for anyone new to the codebase. Reading through the contributing documentation will help although it is not extensive.