Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Schema for Calibanconfig #37

Merged
merged 17 commits into from
Jul 15, 2020
Merged

Conversation

sritchie
Copy link
Collaborator

@sritchie sritchie commented Jul 13, 2020

This PR:

  • introduces the Schema library https://pypi.org/project/schema/
  • uses it to write a schema for CalibanConfig
  • replaces a few of our argparse-targeted validators with Schema versions
  • adds various functions that allow Schema and Argparse to work together
  • adds two contextmanagers that allow for nicely formatted fatal errors to appear for the user.
  • Adds new entries into calibanconfig for a few features we've been discussing.

Why?

The goal is to make it simpler to add new entries into calibanconfig. We can now share validators for calibanconfig entries with the validators we use for argparsing by wrapping the schemas in ua.argparse_schema... and they will Just Work as argument validators.

This makes #20 , #10 and other PRs like it nice and simple.

Base automatically changed from sritchie/restructure to master July 13, 2020 17:37
@codecov
Copy link

codecov bot commented Jul 15, 2020

Codecov Report

Merging #37 into master will increase coverage by 1.00%.
The diff coverage is 81.53%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #37      +/-   ##
==========================================
+ Coverage   48.29%   49.30%   +1.00%     
==========================================
  Files          27       28       +1     
  Lines        2849     2872      +23     
==========================================
+ Hits         1376     1416      +40     
+ Misses       1473     1456      -17     
Impacted Files Coverage Δ
caliban/main.py 26.13% <33.33%> (+0.55%) ⬆️
caliban/util/schema.py 78.37% <78.37%> (ø)
caliban/config/__init__.py 71.92% <84.61%> (+23.98%) ⬆️
caliban/config/experiment.py 83.05% <100.00%> (+3.56%) ⬆️
caliban/util/argparse.py 64.86% <100.00%> (+17.49%) ⬆️
caliban/util/auth.py 66.66% <0.00%> (-9.53%) ⬇️
caliban/platform/gke/util.py 70.38% <0.00%> (-1.54%) ⬇️
caliban/platform/cloud/types.py 89.47% <0.00%> (-0.66%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9e6a5ab...d67ae41. Read the comment docs.

@sritchie sritchie requested a review from ajslone July 15, 2020 04:25
Copy link
Collaborator

@ajslone ajslone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a couple of comments. Schema seems like a good choice.

@@ -263,11 +265,10 @@ def validate_experiment_config(items: ExpConf) -> ExpConf:


def load_experiment_config(s):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typing annotations would be good here, just given the ambiguity of the argument.

@@ -39,6 +40,25 @@ def expand_args(items: Dict[str, str]) -> List[str]:
return list(it.chain.from_iterable(pairs))


def argparse_schema(schema):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typing annotations would be helpful for me here, as I'm new to schema.
Would this be something along the lines of: (?)

SchemaType = Union[s.And, s.Or, s.Regex, s.Use, s.Schema, s.Const]
def argparse_schema(schema: SchemaType) -> Callable[Any, [Any]]:

super().__init__(self.message)


@contextmanager
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, context managers are very nice.

# TODO Once a release with this patch happens:
# https://github.com/keleshev/schema/pull/238,, Change `Or` to `Schema`. This
# problem only occurs for callable validators.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The composition here is quite nice.

@sritchie sritchie merged commit 850a049 into master Jul 15, 2020
@sritchie sritchie deleted the sritchie/calibanconfig_schema branch July 15, 2020 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants