Refactor: Decouple web-features processing from raw schema#1827
Merged
jcscottiii merged 1 commit intomainfrom Sep 17, 2025
Merged
Refactor: Decouple web-features processing from raw schema#1827jcscottiii merged 1 commit intomainfrom
jcscottiii merged 1 commit intomainfrom
Conversation
Background: v3.0 schema is stable now. And introducing it directly was a headache. So I refactored the structs. This commit refactors the web feature ingestion process to decouple it from the raw, auto-generated JSON schema types from the `web-platform-dx/web-features` repository. This change adds support for the new v3 schema format while maintaining backward compatibility with the old format. Previously, the application directly used structs generated from the JSON schema. This created a tight coupling, making the system brittle and difficult to adapt to schema changes. To address this, the following changes were made: - A new set of internal, stable structs has been introduced in `lib/webdxfeaturetypes/types.go`. These structs define the canonical, application-specific representation of web feature data, independent of any specific schema version. - The data parser in `workflows/steps/services/web_feature_consumer/pkg/data/parser.go` has been updated. It now includes logic to process both the old schema and the new v3 schema, transforming the raw, version-specific data into the new stable `webdxfeaturetypes` structs. - All consumers of web feature data throughout the codebase (including Spanner adapters, Datastore adapters, and workflows) have been updated to use the new stable `webdxfeaturetypes` instead of the auto-generated types. - The `StringOrStringArray` type from the source data is now consistently normalized into a `[]string` in the new intermediate representation, simplifying downstream logic. This decoupling provides a robust abstraction layer that isolates the core application from upstream schema changes, making it significantly easier to support future versions of the `web-features` data format with minimal impact on the rest of the system.
jcscottiii
force-pushed
the
jcscottiii/decouple-schema
branch
from
8cc946d to
b00643a
Compare
Collaborator
|
I'll try to review this by this evening. |
Collaborator
Author
|
Many thanks @jrobbins |
jrobbins
approved these changes
Sep 17, 2025
Collaborator
jrobbins
left a comment
jrobbins
left a comment
There was a problem hiding this comment.
The number of files touched scared me off, but the change itself seems reasonable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background: v3.0 schema is stable now. And introducing it directly was a headache. So I refactored the structs.
This commit refactors the web feature ingestion process to decouple it from the raw, auto-generated JSON schema types from the
web-platform-dx/web-featuresrepository. This change adds support for the new v3 schema format while maintaining backward compatibility with the old format.Previously, the application directly used structs generated from the JSON schema. This created a tight coupling, making the system brittle and difficult to adapt to schema changes.
To address this, the following changes were made:
A new set of internal, stable structs has been introduced in
lib/webdxfeaturetypes/types.go. These structs define the canonical, application-specific representation of web feature data, independent of any specific schema version.The data parser in
workflows/steps/services/web_feature_consumer/pkg/data/parser.gohas been updated. It now includes logic to process both the old schema and the new v3 schema, transforming the raw, version-specific data into the new stablewebdxfeaturetypesstructs.All consumers of web feature data throughout the codebase (including Spanner adapters, Datastore adapters, and workflows) have been updated to use the new stable
webdxfeaturetypesinstead of the auto-generated types.The
StringOrStringArraytype from the source data is now consistently normalized into a[]stringin the new intermediate representation, simplifying downstream logic.This decoupling provides a robust abstraction layer that isolates the core application from upstream schema changes, making it significantly easier to support future versions of the
web-featuresdata format with minimal impact on the rest of the system.