Add support for programmatic title generation #9183

NeevCohen · 2024-04-07T21:26:30Z

Change Summary

Implement programmatic title generation via ConfigDict and Field

Related issue number

fix #4632

Checklist

The pull request title is a good summary of the changes - it will be used in the changelog
Unit tests for the changes exist
Tests pass on CI
Documentation reflects the changes where applicable
My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer: @dmontagu

codspeed-hq · 2024-04-07T21:29:55Z

CodSpeed Performance Report

Merging #9183 will not alter performance

_{Comparing NeevCohen:feature/title-generators (577c47a) with main (20b9176)}

Summary

✅ 13 untouched benchmarks

NeevCohen · 2024-04-07T22:03:52Z

Please review

sydney-runkle

Hey @NeevCohen,

Great work, as usual!

I want to look again at _generate_schema.py and do a more in-depth look at your tests (great work btw, looks very thorough). This looks very promising though - I've included some initial feedback / some minor change requests.

One other change I'd like to see is the addition of some documentation in the concepts section of our docs!

pydantic/fields.py

pydantic/_internal/_generate_schema.py

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>

…eature/title-generators

NeevCohen · 2024-05-10T09:35:56Z

@sydney-runkle Thanks! I implemented the changes you requested.
Just one question, in what part of the concepts section do you think I should document this feature in?

PrettyWood

Question : why not having title be a string or a callable that returns a string instead of two params?
This could be extended to alias generation for example with a context passed as arg in case of callable

NeevCohen · 2024-05-14T10:25:19Z

@PrettyWood I implemented it the way the original issue suggested, I'm fine with changing it if we decide that is the way we want to go.
Also, since this is similar to the alias and alias_generator parameters, I figured it would be better to be consistent with the API. IMO if we change title to be a string or a callable, we should also change alias to be the same way.

PrettyWood · 2024-05-14T16:28:13Z

@NeevCohen totally agree. Probably a change for v3 then

NeevCohen · 2024-05-14T18:31:21Z

@PrettyWood Awesome.
@sydney-runkle did you get a chance to do a more in-depth look?

sydney-runkle · 2024-05-15T12:11:25Z

@NeevCohen,

Sorry for the delay, was out of the office last week. Looking now 🚀 !

Re your concepts docs question - your changes look great. We might want to add some more cross links with the config / fields sections, but it looks awesome thus far.

sydney-runkle

@NeevCohen,

Great work here. Really impressed with the thorough testing, and great work on the docs as well.

docs/concepts/json_schema.md

pydantic/fields.py

pydantic/_internal/_generate_schema.py

sydney-runkle · 2024-05-15T12:26:13Z

Let's have @samuelcolvin do a final review of this given that he made the original feature request. Stay tuned!

Apply suggestion Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>

sydney-runkle · 2024-05-29T15:37:51Z

@NeevCohen,

Sorry for the delay here, we'll get this across the line for 2.8!

samuelcolvin

overall looks like a good idea, but there are a few things to iron out.

samuelcolvin · 2024-05-29T15:31:28Z

docs/concepts/json_schema.md

+from pydantic import BaseModel, Field
+
+
+def make_title(field_name: str) -> str:


my main question is whether the full field should be passed to the function, not just the name.

The full field as in the FieldInfo object? That we need to pass both FieldInfo and the name of the field since FieldInfo doesn't include the name.

Then the signature of make_title would be

def make_title(field_name: str, field_info: FieldInfo) -> str: ...

docs/concepts/json_schema.md

samuelcolvin · 2024-05-29T15:35:38Z

pydantic/config.py

@@ -33,11 +33,18 @@ class ConfigDict(TypedDict, total=False):
    title: str | None
    """The title for the generated JSON schema, defaults to the model's name"""

+    class_title_generator: Callable[[str], str] | None
+    """A callable that takes a class name and returns the title for it. Defaults to `None`."""


we should be clear about whether this works for:

pydantic models?

dataclasses?

typeddicts?

It works for all model and model-like (pydantic dataclass, builtin dataclass, typeddict) types. ConfigDict can be applied to all of them so IMO it's a little redundant. Changing the name to model_title_generator makes this even more clear I feel.

I will add more documentation if you feel it's necessary.

samuelcolvin · 2024-05-29T15:36:07Z

pydantic/fields.py

@@ -106,6 +108,8 @@ class FieldInfo(_repr.Representation):
        validation_alias: The validation alias of the field.
        serialization_alias: The serialization alias of the field.
        title: The title of the field.
+        title_priority: Priority of the field's title. This affects whether a title generator is used.


Are you sure we need this? If so, I think it needs more documentation.

I guess I wanted to have a similar API to alias_generator and alias_priority. Do you think it's worth keeping? If so I'll add more documentation.

sydney-runkle · 2024-06-03T17:07:27Z

@NeevCoehn,

Just a side note, we have a pydantic slack that you can join here for faster iteration on development questions, etc :).

Thanks for all of your great work!

NeevCohen · 2024-06-03T17:08:55Z

@sydney-runkle Awesome! Will do!

Thanks

sydney-runkle · 2024-06-07T13:43:00Z

@NeevCohen,

I think @samuelcolvin is quite busy this week - I'll take another pass + review this today. Great work thus far! I think we can definitely get this across the line soon 🚀.

sydney-runkle · 2024-06-10T12:21:00Z

So I think the lingering questions / things to do here are:

Add more documentation for the title_priority logic
Decide if the title generator function should take an argument of type FieldInfo as well
Just want to circle back - are we ok with adding the title argument to the modify_model_json schema function? In general, I'm not super happy with the way we prep to modify JSON schema while building the core schema, so I'm not super excited about this change...

@NeevCohen, great job with the tests and existing docs!

sydney-runkle · 2024-06-10T14:51:58Z

Add more documentation for the title_priority logic

@NeevCohen, lmk when you've done this and I can do a final review!

Decide if the title generator function should take an argument of type FieldInfo as well

I think this could be useful down the line, but it's not a breaking change to introduce later bc we can do signature introspection, so I don't think we need to include this in this PR.

Just want to circle back - are we ok with adding the title argument to the modify_model_json schema function? In general, I'm not super happy with the way we prep to modify JSON schema while building the core schema, so I'm not super excited about this change...

Yeah, the change I'm requesting above is major, I think this is ok to add for now!

NeevCohen · 2024-06-12T19:05:42Z

@sydney-runkle regarding title_priority, after giving it some more thought I feel like it is not necessary. When I first implemented it, I did what alias_generator did. title_priority can be used to determine the priority of title over title_generator, but I think that specifying a title should always take precedence over title_generator, and there is no real use-case for title_priority, so I think I'll just remove it all together.

Regarding adding another FieldInfo parameter to the title_generator, I think that's a good idea and I can see how someone would want to generate a title based on more than just the name of the field, so I think I'll add it in this PR.

Let me know what you think.

sydney-runkle · 2024-06-12T22:00:58Z

@sydney-runkle regarding title_priority, after giving it some more thought I feel like it is not necessary. When I first implemented it, I did what alias_generator did. title_priority can be used to determine the priority of title over title_generator, but I think that specifying a title should always take precedence over title_generator, and there is no real use-case for title_priority, so I think I'll just remove it all together.

Go for it. The more simple, the better. We can always add that down the line if some realistic use case is presented.

Regarding adding another FieldInfo parameter to the title_generator, I think that's a good idea and I can see how someone would want to generate a title based on more than just the name of the field, so I think I'll add it in this PR.

Sure thing. Ping me when you'd like a review, sounds great!

sydney-runkle · 2024-06-20T14:15:21Z

@NeevCohen,

Hoping to release v2.8 before the end of June - any chance you'll have those updates in by then? Thanks for all of your work on this awesome new feature!

* Refactor model_title_generator to accept the class as an arguments instead of the name of the class

NeevCohen · 2024-06-22T22:30:53Z

Hey @sydney-runkle, sorry for the delay, I've been a little busy at work 😅
Anyway, I've implemented the changes you've requested and some more:

I've removed title_priority from FieldInfo
I've changed field_title_generator to accept the info of the field as well
I've changed model_title_generator to accept the class itself, instead of just its name

sydney-runkle · 2024-06-24T12:35:54Z

@NeevCohen,

No worries at all! Thanks so much for your work on this. I'll do a final review today 🚀 !

sydney-runkle

This is truly impressive. Great work on this significant feature!!

Thanks for your amazing work on this @NeevCohen!

sydney-runkle · 2024-06-25T00:12:07Z

pydantic/_internal/_generate_schema.py

@@ -1523,6 +1578,7 @@ def _dataclass_schema(
                        dataclass_bases_stack.enter_context(self._types_namespace_stack.push(dataclass_base))

                # Pushing a config overwrites the previous config, so iterate though the MRO backwards
+                config = None


I have a suspicion that this isn't needed, but perhaps that's wrong

sydney-runkle · 2024-06-25T00:16:37Z

If someone would like to add some more robust testing showcasing how to use both the field name and field info in the title generator function, that would be awesome! (great first issue)

Implement programmatic title generation

eea8462

NeevCohen added 2 commits April 8, 2024 00:39

Use typed dict fixture

7edfd87

Add fixture for typing.Annotated

2f5d5b0

pydantic-hooky bot added the ready for review label Apr 7, 2024

pydantic-hooky bot assigned dmontagu Apr 7, 2024

sydney-runkle added the relnotes-feature label Apr 11, 2024

sydney-runkle suggested changes Apr 30, 2024

View reviewed changes

pydantic/fields.py Outdated Show resolved Hide resolved

pydantic/_internal/_generate_schema.py Outdated Show resolved Hide resolved

pydantic/_internal/_generate_schema.py Outdated Show resolved Hide resolved

pydantic-hooky bot added awaiting author revision awaiting changes from the PR author and removed ready for review labels Apr 30, 2024

pydantic-hooky bot assigned NeevCohen and unassigned dmontagu Apr 30, 2024

NeevCohen and others added 5 commits May 9, 2024 22:24

Merge branch 'main' into 'feature/title-generators'

67240e2

Update pydantic/_internal/_generate_schema.py

823d3d5

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>

Reorder members

939c7a1

Merge remote-tracking branch 'origin/feature/title-generators' into f…

b2e5dad

…eature/title-generators

Refactor common logic into method

306ae09

Add documentation for title generation

428463e

PrettyWood reviewed May 10, 2024

View reviewed changes

sydney-runkle approved these changes May 15, 2024

View reviewed changes

docs/concepts/json_schema.md Outdated Show resolved Hide resolved

pydantic/fields.py Outdated Show resolved Hide resolved

pydantic/_internal/_generate_schema.py Show resolved Hide resolved

NeevCohen and others added 2 commits May 21, 2024 23:44

Update pydantic/fields.py

838b0c9

Apply suggestion Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>

Change field_title_generator in example

e12c5a2

samuelcolvin reviewed May 29, 2024

View reviewed changes

Rename class_title_generator to model_title_generator

eff9e02

NeevCohen requested a review from samuelcolvin May 31, 2024 15:06

sydney-runkle added ready for review and removed awaiting author revision awaiting changes from the PR author labels Jun 6, 2024

Merge branch 'main' into feature/title-generators

070fef1

sydney-runkle added awaiting author revision awaiting changes from the PR author and removed ready for review labels Jun 20, 2024

NeevCohen added 2 commits June 23, 2024 00:00

Remove title_priority from FieldInfo

712a026

* Refactor field_title_generator to accept FieldInfo argument as well

39a4dd4

* Refactor model_title_generator to accept the class as an arguments instead of the name of the class

sydney-runkle approved these changes Jun 25, 2024

View reviewed changes

Merge branch 'main' into feature/title-generators

577c47a

sydney-runkle enabled auto-merge (squash) June 25, 2024 00:17

sydney-runkle mentioned this pull request Jun 25, 2024

Get test coverage back up to 100% #7656

Open

sydney-runkle merged commit 557be4d into pydantic:main Jun 25, 2024

		from pydantic import BaseModel, Field


		def make_title(field_name: str) -> str:

Uh oh!

Add support for programmatic title generation #9183

Add support for programmatic title generation #9183

Uh oh!

Conversation

NeevCohen commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Related issue number

Checklist

Uh oh!

codspeed-hq bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #9183 will not alter performance

Summary

Uh oh!

NeevCohen commented Apr 7, 2024

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NeevCohen commented May 10, 2024

Uh oh!

PrettyWood left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NeevCohen commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PrettyWood commented May 14, 2024

Uh oh!

NeevCohen commented May 14, 2024

Uh oh!

sydney-runkle commented May 15, 2024

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sydney-runkle commented May 15, 2024

Uh oh!

sydney-runkle commented May 29, 2024

Uh oh!

samuelcolvin left a comment

Choose a reason for hiding this comment

Uh oh!

samuelcolvin May 29, 2024

Choose a reason for hiding this comment

Uh oh!

NeevCohen May 31, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samuelcolvin May 29, 2024

Choose a reason for hiding this comment

Uh oh!

NeevCohen May 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samuelcolvin May 29, 2024

Choose a reason for hiding this comment

Uh oh!

NeevCohen May 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sydney-runkle commented Jun 3, 2024

Uh oh!

NeevCohen commented Jun 3, 2024

NeevCohen commented Apr 7, 2024 •

edited

Loading

codspeed-hq bot commented Apr 7, 2024 •

edited

Loading

PrettyWood left a comment •

edited

Loading

NeevCohen commented May 14, 2024 •

edited

Loading

NeevCohen May 31, 2024 •

edited

Loading

NeevCohen May 31, 2024 •

edited

Loading

NeevCohen commented Jun 12, 2024 •

edited

Loading

NeevCohen commented Jun 22, 2024 •

edited

Loading