Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(types): improve order by type safety #255

Merged
merged 1 commit into from Jan 30, 2022

Conversation

RobertCraigie
Copy link
Owner

Change Summary

Improves the order type to signify that only one field can be present at any given time.

Checklist

  • Unit tests for the changes exist
  • Tests pass without significant drop in coverage
  • Documentation reflects changes where applicable
  • Test snapshots have been updated if applicable

Agreement

By submitting this pull request, I confirm that you can use, modify, copy and redistribute this contribution, under the terms of your choice.

@codecov
Copy link

codecov bot commented Jan 30, 2022

Codecov Report

Merging #255 (142f0ee) into main (40bbd68) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #255   +/-   ##
=======================================
  Coverage   96.74%   96.75%           
=======================================
  Files         111      111           
  Lines        5412     5420    +8     
  Branches      319      320    +1     
=======================================
+ Hits         5236     5244    +8     
  Misses        130      130           
  Partials       46       46           
Impacted Files Coverage Δ
src/prisma/generator/schema.py 95.71% <100.00%> (+0.32%) ⬆️
tests/test_find_many.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 40bbd68...142f0ee. Read the comment docs.

@RobertCraigie RobertCraigie merged commit 6cf3280 into main Jan 30, 2022
@RobertCraigie RobertCraigie deleted the feat/orderby-typesafety branch January 30, 2022 18:34
@RobertCraigie
Copy link
Owner Author

RobertCraigie commented Jan 30, 2022

If you need to order by multiple fields you can pass a list to order, for example:

posts = await Post.prisma().find_many(
	order=[
        {'created_at': 'desc'},
        {'views': 'desc'},
    ],
)

@danigosa
Copy link

Hi @RobertCraigie

The solution is extremely inflexible and broke our code in development/testing branches, because it requires a Literal and something so simple as

def some_endpoint(sorty_by_field: str):
       posts = await Post.prisma().find_many(
	order=[
        {sort_by_field: 'desc'}
    ],
)
error: Argument "order" to "find_many" of "OrganizationActions" has incompatible type "Dict[str, Literal['asc', 'desc']]"; expected "Union[...]

Does not work. We tried to make Literals, Finals, but it fails on mypy and pyright. I do not know if there is a suitable solution for that, but writing the name of the order field as Literal in the code in advance is really making the use of find_many really hard with type checkers, the only solution for now for us is to bypass with cast or type:ignore.

@RobertCraigie
Copy link
Owner Author

@danigosa oh that's unfortunate, I didn't realise that wouldn't work.

Does this work for you?

from prisma.types import PostScalarFieldKeys

def some_endpoint(sorty_by_field: PostScalarFieldKeys) -> None:
       posts = await Post.prisma().find_many(
	order=[
        {sort_by_field: 'desc'}
    ],
)

@danigosa
Copy link

danigosa commented Jun 28, 2022

Hi @RobertCraigie

I can't find any prisma.types.PostScalarFieldKeys 🤔

image

Where this type is generated?

@RobertCraigie
Copy link
Owner Author

@danigosa sorry I should've been more clear, that type is generated for every model in your schema file. So PostScalarFieldKeys is only generated if you have a model named Post in your schema file.

Are there other types like PostScalarFieldKeys generated?

@danigosa
Copy link

@danigosa sorry I should've been more clear, that type is generated for every model in your schema file. So PostScalarFieldKeys is only generated if you have a model named Post in your schema file.

Are there other types like PostScalarFieldKeys generated?

Yes! I know documenting is hard, but this is really awesome and undocumented 😆

@danigosa
Copy link

@danigosa sorry I should've been more clear, that type is generated for every model in your schema file. So PostScalarFieldKeys is only generated if you have a model named Post in your schema file.
Are there other types like PostScalarFieldKeys generated?

Yes! I know documenting is hard, but this is really awesome and undocumented laughing

Unfortunately with mypy it does not work:

@router.get(
    "",
    response_model=Paginated[CenterWithoutRelations],
    response_class=ORJSONResponse,
)
async def list_centers(
    take: int = None,
    skip: int = None,
    sort_by: CenterScalarFieldKeys = "id",
    order: SortOrder = "asc",
    db: Prisma = Depends(db_from_request),
) -> Paginated[CenterWithoutRelations]:
    """LIST"""
    if sort_by not in CenterWithoutRelations.__fields__:
        return ORJSONResponse(
            status_code=400, content={"message": "Incorrect sort_by field"}
        )

    data = await db.center.find_many(
        take=take,
        skip=skip,
        where={"deleted": False},
        order={sort_by: order},
    )
    if skip is not None:
        count = await db.center.count()
    else:
        count = len(data)

    return {"data": data, "count": count}
➜  care-api git:(1446-care-api-dev-deploy) ✗ make stylecheck
Loading .env environment variables...
src/robbie/care_api/v1/centers.py:46: error: Argument "order" to "find_many" of "CenterActions" has incompatible type "Dict[Literal['id', 'org_id', 'name', 'created_at', 'modified_at', 'deleted'], Literal['asc', 'desc']]"; expected "Union[_Center_id_OrderByInput, _Center_org_id_OrderByInput, _Center_name_OrderByInput, _Center_created_at_OrderByInput, _Center_modified_at_OrderByInput, _Center_deleted_OrderByInput, List[Union[_Center_id_OrderByInput, _Center_org_id_OrderByInput, _Center_name_OrderByInput, _Center_created_at_OrderByInput, _Center_modified_at_OrderByInput, _Center_deleted_OrderByInput]], None]"
src/robbie/care_api/v1/centers.py:46: error: Expected TypedDict key to be string literal

The types code:

CenterScalarFieldKeys = Literal[
    'id',
    'org_id',
    'name',
    'created_at',
    'modified_at',
    'deleted',
]

I think the problem is using Union of TypedDict with literal definition of key, may python typing does not support the goodies of TypeScript type DerivedCenter = {[k: keyof Center]: string | number} where you can define both types for keys and values... or I'm doing something wrong

@danigosa
Copy link

Looking at https://peps.python.org/pep-0589/#rejected-alternatives I can see why is not working, TypedDict is not supposed to support that, any workaround is welcome!

@danigosa
Copy link

danigosa commented Jun 29, 2022

Adding ScalarFields still have mypy and pyright problems, the type in checkers after applying ScalarFieldKeys is Dict[Literal['id', 'org_id', 'name', 'created_at', 'modified_at', 'deleted'] which is ok but does not work for Union[TyopedDict,...]:

src/robbie/care_api/v1/centers.py:46: error: Argument "order" to "find_many" of "CenterActions" has incompatible type "Dict[Literal['id', 'org_id', 'name', 'created_at', 'modified_at', 'deleted'], Literal['asc', 'desc']]"; 
expected "Union[_Center_id_OrderByInput, _Center_org_id_OrderByInput, _Center_name_OrderByInput, _Center_created_at_OrderByInput, _Center_modified_at_OrderByInput, _Center_deleted_OrderByInput, List[Union[_Center_id_OrderByInput, _Center_org_id_OrderByInput, _Center_name_OrderByInput, _Center_created_at_OrderByInput, _Center_modified_at_OrderByInput, _Center_deleted_OrderByInput]], None]"
src/robbie/care_api/v1/centers.py:46: error: Expected TypedDict key to be string literal
" /home/danigosa/rbcode/backend/care-api/src/robbie/care_api/v1/centers.py:46:15 - error: Argument of type "dict[str, str]" cannot be assigned to parameter "order" of type "CenterOrderByInput | List[CenterOrderByInput] | None" in function "find_many"
    Type "dict[str, str]" cannot be assigned to type "CenterOrderByInput | List[CenterOrderByInput] | None"

@RobertCraigie
Copy link
Owner Author

Yes! I know documenting is hard, but this is really awesome and undocumented 😆

Yeah the documentation needs some more love to be honest...

Any suggestions are more than welcome :)

I think the problem is using Union of TypedDict with literal definition of key, may python typing does not support the goodies of TypeScript type DerivedCenter = {[k: keyof Center]: string | number} where you can define both types for keys and values... or I'm doing something wrong

I can replicate the errors you're seeing with both pyright and mypy. It saddens me to say this but I think this is just one of those cases where you'll have to ignore the error unfortunately. If you do this in multiple places I'd recommend extracting it to a function that returns the corresponding PostOrderByInput for your model so that you only have to ignore the error in that function.

@danigosa Thanks for looking into this and raising it, I was not aware that this did not work! This should probably also be raised on typing-sig / mypy / pyright to see if it would be feasible to support officially.

If I receive signal from other users that this change should be reverted, I'll look into it but for the time being I think this change provides a good amount of value as it signals to users directly writing the order query themselves that they can only specify one field at a time.

@danigosa
Copy link

danigosa commented Jun 30, 2022

Yes! I know documenting is hard, but this is really awesome and undocumented laughing

Yeah the documentation needs some more love to be honest...

Any suggestions are more than welcome :)

I think the problem is using Union of TypedDict with literal definition of key, may python typing does not support the goodies of TypeScript type DerivedCenter = {[k: keyof Center]: string | number} where you can define both types for keys and values... or I'm doing something wrong

I can replicate the errors you're seeing with both pyright and mypy. It saddens me to say this but I think this is just one of those cases where you'll have to ignore the error unfortunately. If you do this in multiple places I'd recommend extracting it to a function that returns the corresponding PostOrderByInput for your model so that you only have to ignore the error in that function.

@danigosa Thanks for looking into this and raising it, I was not aware that this did not work! This should probably also be raised on typing-sig / mypy / pyright to see if it would be feasible to support officially.

If I receive signal from other users that this change should be reverted, I'll look into it but for the time being I think this change provides a good amount of value as it signals to users directly writing the order query themselves that they can only specify one field at a time.

Yeah we are bypassing it with cast(CenterOrderByInput, obj) this way developers within the IDE can at least look into the target type to see what we are skipping... we think it's better than simply ignoring the checker, which is less meaningful, and we have rules to check that we do not ignore cheks 😆 .

After a talk with some team mates it's really a limitation on typing, although we do not find a TypeScript solution for it either, seems like the API from Query Engine is the one enforcing a bad typing by asking a list of dictionaries of just 1 pair per order field element which is itself a little weird, IMHO.

@RobertCraigie
Copy link
Owner Author

Yeah casting is the best solution.

After a talk with some team mates it's really a limitation on typing, although we do not find a TypeScript solution for it either, seems like the API from Query Engine is the one enforcing a bad typing by asking a list of dictionaries of just 1 pair per order field element which is itself a little weird, IMHO.

As far as I'm aware, the list of dictionaries is required as the order of sorting matters when you're sorting by multiple fields. If it was sent in a single dictionary then the ordering of the fields is not strictly defined which could lead to some unexpected results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants