Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use pydantic for config validation #455

Merged
merged 47 commits into from
Dec 28, 2020
Merged

use pydantic for config validation #455

merged 47 commits into from
Dec 28, 2020

Conversation

ITProKyle
Copy link
Collaborator

@ITProKyle ITProKyle commented Sep 16, 2020 •
edited
Loading

Why This Is Needed

resolves #431

What Changed

Added

  • added data classes for Runway & CFNgin configurations including dedicated data classes for each complex component
    • provide type checking when parsing the configuration files and through processing as values are resolved
    • transforms data as needed
    • can be used to export a JSON schema
    • raises a detailed error pinpointing invalid configuration

Changed

  • CFNgin package cache directory is now ./.runway/cache
  • the runway config class is now runway.config.RunwayConfig
  • renamed and moved Runway config component classes
  • Runway config component (definition) classes no longer handle data parsing and instead use a data class
    • they now act as a higher level API for working with the config
  • the CFNgin config class is now runway.config.CfnginConfig
    • renamed and moved config component classes
  • updated documentation for Runway and CFNgin configuration

Removed

  • removed support for stacker* keys in the CFNgin config
  • removed support for legacy environment mappings in fields such as env_vars, parameters, etc
  • removed support for legacy environments usage (functionally that was moved to parameters)

@ITProKyle ITProKyle added the feature Request or pull request for a new feature label Sep 16, 2020
@ITProKyle ITProKyle self-assigned this Sep 16, 2020
@ITProKyle ITProKyle mentioned this pull request Sep 29, 2020
Merged
@ITProKyle ITProKyle added the breaking Issue or PR is for a breaking change. Bug is a breaking bug with no workaround. label Oct 1, 2020
@ITProKyle ITProKyle marked this pull request as ready for review October 1, 2020 22:35
@ITProKyle
start implementing pydantic
6b5f6af 
- rewrite cfngin config to use it
- update functions and classes that use it to use new methods and support Path objects
- update tests
@ITProKyle
attempt to fix pyinstaller recursion error
4b09e40
@ITProKyle
remove dev print statements from cfngin
116a139
@ITProKyle
add typing_extensions and update other dependencies
1c9e39f
@ITProKyle
add new runway config, remove old config
86534d5 
- start moving cfngin config to same location
- add exception for config not found
- update runway to import the new config locations
- update some type annotations
@ITProKyle
update to pass unit tests, skip config tests
1cf0982
@ITProKyle
add class for test definition
b9c7717
@ITProKyle
update tests
9399393
@ITProKyle
add tests for runway config models, move shared code to utils
9304f32
@ITProKyle
add tests for cfngin config models
6d46c6b
@ITProKyle
move cfngin config to runway.config
aeb3efd
@ITProKyle
rename cfngin models, split out cfngin config model
8704370
@ITProKyle
enable assignment and default value validation
503671e
@ITProKyle
add tests for config model base and utils
c6228e9
@ITProKyle
add tests for runway config components
1567b2f
@ITProKyle
remove old tests
198d845
@ITProKyle
remove old config
512d7ad
@ITProKyle
add Dict[str, str] support to environments
945c789
@ITProKyle
add str support to env_vars
20e2fed
@ITProKyle
update module cache location and docs
5c6d757
@ITProKyle
fix some types, simplify code to only support defined types
98feeef
@ITProKyle
remove schematics and update lock files
9dc2298
@ITProKyle
fix staticsite integration test config
bc12edc
@ITProKyle
fix cdk integration test config
9bcf787
@ITProKyle
fix module tags integration test
38410d8
@ITProKyle
fix serverless integration tests
83f95e3 
fix serverless test runner
@ITProKyle
fix commands & serverless integration tests
5619b26
@ITProKyle
fix terraform integration tests
853b016
@ITProKyle
fix cfngin tests
b27602f
@ITProKyle
fix an issue causing errors when running from concurrent.futures
ad30bd8
@ITProKyle
create custom field type for runway_version, add schema details
e0e5a52
@ITProKyle
add enum for runway module type
3fb1abd
@ITProKyle
fix spelling
ace24d1
@ITProKyle
add cfngin schema details, correct some runway schema descriptions
49086d9
@ITProKyle
fix tests
9501743
@ITProKyle
update documentation requirements
411a7b4
@ITProKyle
update runway config docs, fix default value to match docs
20e94fb
@ITProKyle
fix unit test
452cddc
@ITProKyle
update cfngin docs, change formatting slightly for some runway docs
b7ba7ad
@ITProKyle
move hook & stack type conversions to data class
d10ad48
@ITProKyle
update lock files & fix linting post rebase
b4eca83
@ITProKyle
modify json schema to accept stack without name and lookup for bool
f3b061d
@ITProKyle ITProKyle linked an issue Dec 28, 2020 that may be closed by this pull request
[TODO] investigate using pydantic for config file validation & implement if appropriate #431
Closed
@ITProKyle
Copy link
Collaborator Author

I am going to merge this in for now to reduce the amount of rebase catchup required to keep it up to date.

After having used used this branch and the schema it produces for a while, I think it is at least vary close to feature parity if not already there. (I fixed some of the annoyances I had noticed)

If any future changes are needed, they can be addressed individually.

@ITProKyle ITProKyle merged commit 20d88e2 into rackspace:v2 Dec 28, 2020
@ITProKyle ITProKyle deleted the feature/pydantic-config branch December 28, 2020 20:52
ITProKyle added a commit that referenced this pull request Jan 15, 2021
@ITProKyle
use pydantic for config validation (#455)
16892fd
ITProKyle added a commit that referenced this pull request Feb 16, 2021
@ITProKyle
use pydantic for config validation (#455)
b6cc01e
ITProKyle added a commit that referenced this pull request Feb 25, 2021
@ITProKyle
use pydantic for config validation (#455)
5a8f6be
ITProKyle added a commit that referenced this pull request Mar 22, 2021
@ITProKyle
use pydantic for config validation (#455)
540128b
@ITProKyle ITProKyle added the changelog:major Major change resulting in a major release label Mar 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@troyready troyready Awaiting requested review from troyready

Assignees

@ITProKyle ITProKyle

Labels
breaking Issue or PR is for a breaking change. Bug is a breaking bug with no workaround. changelog:major Major change resulting in a major release feature Request or pull request for a new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[TODO] investigate using pydantic for config file validation & implement if appropriate
1 participant
@ITProKyle