first version of defaults injection #1

rkuhn · 2020-02-27T22:57:40Z

No description provided.

rkuhn

@wngr I have self-reviewed, should be good to go. See some default_for_schema test for an example of conjuring a whole instance from thin air. (real code should then validate that instance)

rkuhn · 2020-02-28T07:15:09Z

It would also be great if you could give it a spin and compare the performance to version 3.2.0, with and without making use of defaults in a reasonably large example schema. And not that I’m overly worried, but this is some complex stuff, so it might be warranted to let it keep crunching schemas and validations for a few hours while observing memory usage.

wngr

First round of reviews. Still wrapping my head around this ..

src/json_schema/keywords/items.rs

wngr · 2020-02-28T20:47:31Z

src/json_schema/keywords/mod.rs

@@ -37,10 +40,10 @@ impl fmt::Debug for dyn Keyword + 'static {
 macro_rules! keyword_key_exists {


I'm confused by this macro. Looking at the usage, maybe_val is &serde_json::Value. How can this work, given that if and else have incompatible types?

The trick is that the else branch does not yield a value but returns from the enclosing function — these macros are purely lexical, typechecking happens only after the program has been modified.

wngr · 2020-02-28T21:00:48Z

tests/schema/mod.rs

@@ -8,7 +8,7 @@ fn visit_specs<F>(dir: &path::Path, cb: F)
 where
    F: Fn(&path::Path, Value) + Copy,
 {
-    let contents = fs::read_dir(dir).ok().unwrap();
+    let contents = fs::read_dir(dir).expect(&*format!("cannot list directory {:?}", dir));


very helpful for newcomers indeed 🎩

src/json_schema/schema.rs

wngr · 2020-02-28T21:50:24Z

src/json_schema/schema.rs

+            schema.add_defaults_recursive(top, id, scope);
+        }
+
+        // step 2: use explicit default if present


why not do this before step 1? If you have a default, you don't need to go down into tree.

The point of the traversal is not to get a default for this node, it is to mutate the whole subtree to get defaults, too.

src/json_schema/schema.rs

wngr · 2020-02-28T21:58:01Z

src/json_schema/schema.rs

@@ -222,6 +325,8 @@ impl Schema {
                        if let Some(validator) = keyword.keyword.compile(def, context)? {
                            if is_exclusive_keyword {
                                validators = vec![validator];
+                            } else if keyword.keyword.place_first() {
+                                validators.splice(0..0, std::iter::once(validator));


What's this for? This will remove an existing validator.

Nope, it replaces the empty range at the beginning of the vector, i.e. pushes to the front.

wngr · 2020-02-28T22:05:18Z

src/json_schema/validators/properties.rs

+
+        // necessary due to object being mutated in the loop
+        let keys = object.keys().cloned().collect::<Vec<_>>();
+        'main: for key in keys.iter() {


does that need to be named? you don't seem to break anyway

I left it in because it clarifies the continue that appears many lines down.

wngr · 2020-02-28T22:10:15Z

src/json_schema/validators/of.rs

+        for schema in schemas.iter() {
+            let mut result = schema.validate_in(&val, path);
+            if result.is_valid() && result.replacement.is_some() {
+                *val.to_mut() = result.replacement.take().unwrap();


Shouldn't all possible replacements be merged into val instead of it being overwritten?

Yes, indeed that is the case: notice how the updated &val will be passed to the next schema two lines up.

Co-Authored-By: Oliver Wangler <oliver@wngr.de>

src/json_dsl/validators/at_least_one_of.rs

diego-actyx · 2020-03-03T14:57:07Z

src/json_schema/validators/of.rs


-        for url in self.schemes.iter() {
-            let schema = scope.resolve(url);
+        // second pass if defaults are enabled to check that the result is stable


What is an example of a schema where you might have divergent defaults? And can we add it as a test case?

good point, I shall try to make a small one

wngr

This is good to go from my point of view.

rkuhn · 2020-03-12T18:06:20Z

Performance measurements (debug build):

this PR with supply_defaults: 16.7ms compilation, 4.4ms validation
this PR without supply_defaults: 14.5ms compilation, 1.9ms validation
3.2.0: 15.7ms compilation, 1.7ms validation

measurements with release build:

this PR with supply_defaults: 1.9ms compilation, 560µs validation
this PR without supply_defaults: 1.7ms compilation, 170µs validation
3.2.0: 1.8ms compilation, 150µs validation

Conclusion: about 10% slow-down when not using the new feature.

wngr · 2020-03-12T18:37:15Z

what about a feature flag? .. but that's probably up to the maintainers

…r detail (s-panferov#67) Include the name of the unallowed property name in the errors::Properties details, enabling users to more precisely pin-point the issue in the JSON document.

* feat(json_schema): Draft 7 date and time formats * chore(*): Address warnings * feat(json_schema): Draft 6 JSON pointers * feat(json_schema): Draft 7 IDN hostname Kind of. There's no exclusion between "regular" hostnames and IDNs. * feat(json_schema): Draft 7 conditionals * feat(json_schema): Draft 7 content media type/encoding * feat(json_schema): Draft 7 remaining formats * chore(*): cleanup Co-authored-by: Simon Bihel <simon.bihel@vonage.com>

rkuhn added 8 commits February 24, 2020 21:01

fix lint

0c10e94

add supply_default config setting

999ea93

add Schema::add_defaults

3805075

wire up computation of defaults

2d65318

implement injection of default values

c64581f

wire up relevant validators

fdab81f

all tests green

3f12960

polish after self-review

7456274

rkuhn commented Feb 28, 2020

View reviewed changes

remove leftover println

4a2148b

wngr reviewed Feb 28, 2020

View reviewed changes

rkuhn and others added 3 commits March 1, 2020 19:44

Update src/json_schema/schema.rs

e0014d9

Co-Authored-By: Oliver Wangler <oliver@wngr.de>

Update src/json_schema/schema.rs

d21755f

Co-Authored-By: Oliver Wangler <oliver@wngr.de>

add comment

47230cc

FedericoPonzi reviewed Mar 3, 2020

View reviewed changes

src/json_dsl/validators/at_least_one_of.rs Show resolved Hide resolved

diego-actyx reviewed Mar 3, 2020

View reviewed changes

wngr approved these changes Mar 9, 2020

View reviewed changes

lint fix

abc96d2

add tests for conflicting and divergent allOf defaults

5883f05

rkuhn mentioned this pull request Mar 12, 2020

add facility for supplying schema defaults during validation s-panferov/valico#64

Closed

sbihel and others added 7 commits June 5, 2020 21:21

fix(json_schema): Handle new draft6 tests (s-panferov#65)

0933726

(cargo-release) version 3.3.0

f3f84f0

(cargo-release) version 3.4.0

4e684c0

(cargo-release) start next development iteration 3.4.1-alpha.0

961398a

feat(json_schema): include unallowed additional property name in erro…

e0b07e8

…r detail (s-panferov#67) Include the name of the unallowed property name in the errors::Properties details, enabling users to more precisely pin-point the issue in the JSON document.

chore: update test suite

9112ad7

rkuhn added 3 commits December 11, 2020 16:38

Merge remote-tracking branch 'origin/master' into default2

265e086

fix date validation and ignore other optional test failures

950dc21

add two comments

ad39f87

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

first version of defaults injection #1

first version of defaults injection #1

rkuhn commented Feb 27, 2020

rkuhn left a comment •

edited

Loading

rkuhn commented Feb 28, 2020

wngr left a comment

wngr Feb 28, 2020

rkuhn Mar 1, 2020

wngr Feb 28, 2020

wngr Feb 28, 2020

rkuhn Mar 1, 2020

wngr Feb 28, 2020

rkuhn Mar 1, 2020

wngr Feb 28, 2020

rkuhn Mar 1, 2020

wngr Feb 28, 2020

rkuhn Mar 1, 2020

diego-actyx Mar 3, 2020

rkuhn Mar 12, 2020

wngr left a comment

rkuhn commented Mar 12, 2020 •

edited

Loading

wngr commented Mar 12, 2020

		@@ -37,10 +40,10 @@ impl fmt::Debug for dyn Keyword + 'static {
		macro_rules! keyword_key_exists {

first version of defaults injection #1

Are you sure you want to change the base?

first version of defaults injection #1

Conversation

rkuhn commented Feb 27, 2020

rkuhn left a comment • edited Loading

Choose a reason for hiding this comment

rkuhn commented Feb 28, 2020

wngr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wngr left a comment

Choose a reason for hiding this comment

rkuhn commented Mar 12, 2020 • edited Loading

wngr commented Mar 12, 2020

rkuhn left a comment •

edited

Loading

rkuhn commented Mar 12, 2020 •

edited

Loading