Enable shapes-only codegen

[JordonPhillips]

This adds an alternative generation mode that only generates "data shapes". It's a widely requested feature for both this code generator (I can't tell you how many DMs about this I've received) and for other Smithy generators.

These generated types could still be used in a rich way thanks to the schemas backing them - they could still be plugged into any ShapeSerializer or ShapeDeserializer

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[nateprewitt]

Changes look good, looks like we have a test failing with the changes though.

E ImportError: cannot import name 'ALL_QUERY_STRING_TYPES_INPUT' from 'restjson._private.schemas' (/home/runner/work/smithy-python/smithy-python/codegen/smithy-python-protocol-test/build/smithyprojections/smithy-python-protocol-test/rest-json-1/python-client-codegen/restjson/_private/schemas.py)

[JordonPhillips]

I refactored this after all the repo structure changes. I made it so that the new plugin is a separate package and abstracted out some of the functionality that was being done by the client's directed codegen so that it could be shared.

[JordonPhillips]

Note that that number of added lines is scarier than reality as I had to copy over a bunch of smithy IDL files for testing.

[haydenbaker]

Hmmm.. is this something where there could be a footgun later if they are using a (hypothetical) server plugin or the client plugin? Using type codegen in conjunction with client/server codegen will be common.

[haydenbaker]

I'm just calling it out, because, while it would always be possible, I'm not sure we want to point this capability out if it can lead to problems later.

[haydenbaker]

Also, Smithy Java generates inputs and outputs without needing the exclusion. We should be consistent.

[JordonPhillips]

I'd rather they don't generate them tbh. I can make some improvments in this area though

[JordonPhillips]

I made it a simple toggle in the settings.

[haydenbaker]

If you run the generation for the weather service example, pyright throws some errors:

  examples/weather/build/smithy/types/python-type-codegen/weather/_private/schemas.py:569:1 - error: "STRING_LIST" is constant (because it is uppercase) and cannot be redefined (reportConstantRedefinition)
  examples/weather/build/smithy/types/python-type-codegen/weather/_private/schemas.py:1139:1 - error: "STRING_LIST" is constant (because it is uppercase) and cannot be redefined (reportConstantRedefinition)
  examples/weather/build/smithy/types/python-type-codegen/weather/_private/schemas.py:1493:1 - error: "NON_EMPTY_STRING" is constant (because it is uppercase) and cannot be redefined (reportConstantRedefinition)
  examples/weather/build/smithy/types/python-type-codegen/weather/_private/schemas.py:1507:1 - error: "NON_EMPTY_STRING_LIST" is constant (because it is uppercase) and cannot be redefined (reportConstantRedefinition)

This is because the same constant is being produced in the same file:

STRING_LIST = Schema.collection(
    id=ShapeID("aws.auth#StringList"),
    shape_type=ShapeType.LIST,
    ...
)
...
STRING_LIST = Schema.collection(
    id=ShapeID("aws.protocols#StringList"),
    shape_type=ShapeType.LIST,
    ...
)

Smithy Java has the same problem, but it's not checked by anything:

    static final Schema STRING_LIST = Schema.listBuilder(ShapeId.from("aws.auth#StringList"))
        .putMember("member", PreludeSchemas.STRING)
        .build();
    ...
    static final Schema STRING_LIST = Schema.listBuilder(ShapeId.from("aws.protocols#StringList"))
        .putMember("member", PreludeSchemas.STRING)
        .build();

Whether or not this is actually problematic... if the underlying schema is always the same for a given constant, then it's not a problem.

[haydenbaker]

Same root issue, the serde methods are duplicated in models.py, leading to reportRedeclaration:

    models.py:892:5 - error: Function declaration "_serialize_string_list" is obscured by a declaration of the same name (reportRedeclaration)
    models.py:901:5 - error: Function declaration "_deserialize_string_list" is obscured by a declaration of the same name (reportRedeclaration)
    models.py:1123:5 - error: Function declaration "_serialize_string_list" is obscured by a declaration of the same name (reportRedeclaration)
    models.py:1132:5 - error: Function declaration "_deserialize_string_list" is obscured by a declaration of the same name (reportRedeclaration)

[JordonPhillips]

Yes that's because tons of dependencies have those types. This is why the integ test only generates local shapes. There's not really a great workaround other than raising a conflict exception that's a bit more explicit.

[JordonPhillips]

As it turns out, we weren't running validation! I forgot that ModelTransformer doesn't validate by itself. All those string lists were private, so they shouldn't have been pulled in in the first place.

[haydenbaker]

realistically, users probably won't add a selector, so I'd add a test (or modify this one) without the selector. You'll get the build error from pyright throwing reportConstantRedefinition and reportRedeclaration

[JordonPhillips]

Eh I disagree. I think most people aren't gonna want to generate every shape in smithy.aws or whatever random trait package they have. The only ways around that are selectors or creating your own service shape.