my 2 cents #61

isomogyi · 2019-09-30T05:58:56Z

I really much appreciate the idea and code of this library. In our project we have 20 schemas linked via ref and so running a validation against invalid data gives you a puzzle work to spot the problem (whereas jsonschema library shows the problem directly). I wanted to have a better message so I came up with this patch, which among others contains:

fixing problem can be increased by calling the CodeGenerator constructor with verbose=True
optimizing repeated if-check-type-statements
throwing naked JsonSchemaExceptions where the message does not really matter (like in oneOf tests)
in required test, only show properties missing
fixing small inconsistencies

horejsek

Thank you for yours 2 cents. I didn't check the whole pull request. Unfortunately, you are working on the same issues I do and I have different view how to solve them. I like some ideas, though, and I would like to accept them. When I will be done, I will let you know.

horejsek · 2019-10-01T15:44:00Z

fastjsonschema/__init__.py

-# pylint: disable=redefined-builtin,dangerous-default-value,exec-used
-def compile(definition, handlers={}, formats={}):
+# pylint: disable=redefined-builtin,exec-used
+def compile(definition, handlers=None, formats=None, **kwargs):


I have dicts there on purpose. With them in documentation is clear the value should be a dict.

Since I started refactoring your library for my own purpose and I this is my first contribution ever to open-source, I stack to the coding standards/linting we use. Since we all agree that there's only python3 left - we can leave the docs to annotations. But if you prefer mutable defaults, it is a choice

True, Python 2 support is dropped so we could use annotations instead.

horejsek · 2019-10-01T15:44:28Z

fastjsonschema/__init__.py

-        validate = fastjsonschema.compile(definition, formats={
-            'foo': r'foo|bar',
-            'bar': lambda value: value in ('foo', 'bar'),
-        })


Why did you remove the doc?

sorry, this must have skipped my attention somehow.

horejsek · 2019-10-01T15:47:22Z

fastjsonschema/__init__.py

    return resolver, code_generator


-def _get_code_generator_class(schema):
+def get_code_generator_class(schema):


I don't want to make this function "public".

Clear. With the added format handlers I think they can stay private. The point is that we used the version 2.12 and there I had to duplicate the code so I could monkey patch the generator class.

horejsek · 2019-10-01T15:53:59Z

fastjsonschema/generator.py

@@ -28,17 +46,20 @@ class CodeGenerator:

    INDENT = 4  # spaces

-    def __init__(self, definition, resolver=None):
+    def __init__(self, definition, resolver=None, formats=None, verbose=False, root_name='data'):


I have better plan for exceptions. I want to enhance exception class with more information which is possible to parse with a code. So no need to verbose. If you will wait, you will see. Unfortunately you started on it when I already have something in progress.

well, i could not wait and hope that stuff will get better in the repo, so i went on my own and was finished and then realized that maybe I could give back some of my changes.

I understand.

horejsek · 2019-10-01T15:55:44Z

fastjsonschema/generator.py

-            with self.l('def {}(data):', name):
-                self.generate_func_code_block(definition, 'data', 'data', clear_variables=True)
+            self.code_break()
+            with self.l('def {{}}(data, scope="{}"):'.format(self._root_name), name):


I don't get the reason for root_name...

when you have a lot of json schema files e.g. article.json product.json etc, our api had to come up with better error messages that data.teaser[0] must be ...., so now when compiling article.json with root_name='article' the message is article.teaser[0] must be ...
the default value for scope for a function block is root_name which should be 'data' as in 2.12 for compatibility.

So your API returns in the response the message from this library? I think this can be improved with my idea of more information on the exception class, so your API can use it and generate nice messages which matches the other messages in your app.

if you want (as i read in feature requests) to add attributes to the exception - that's great. for me the most important feature is to keep track of the context/scope of where the validation process is at when the exception is thrown. so given: a schema:
article.json

"$schema": "http://json-schema.org/draft-07/schema#", "type" : "object", "additionalProperties": false, "properties": { "teaser" : { "$ref": "intro.json" }, "prolog" : { "$ref": "intro.json" } } }``` when the error occurs during the prolog validation, the exception contains an attribute with value `article.prolog.....` about where it happened (like the jsonschema library does).

I see. I will make issue for that. Or do you want to do it on top of latest master?

well, i suppose now i will have to wait for you to finish your ideas and then try to somehow keep the best of two. as i said, basically i was pretty much content with the refactor and now it is part of the production code. my pull request was merely a try to give back some of my effort for better user experience of your library

I have my changes released. I created issue for your needs. If you want, you can upgrade your version of fastjsonschema, apply that feature on top of it and do a new merge request, or I can do it. :-)

Well, I would like to contribute if you are interested but as you pointed out your vision of where to go differs from what I have proposed, so perhaps working collaboratively might be a better way than keep merging two ideas all the way. Let me know what you think.

You can check out the exceptions--they have more information which can help apps to generate better message for the user. Another step is to be able to return all errors at once. Your improvement is also good, to remember the whole path in connected definitions. If you want to implement this on top of the master, go ahead. If you want to keep your version of JSON schema, it's ok, I can implement it here one day.

horejsek · 2019-10-01T15:58:11Z

fastjsonschema/version.py

@@ -1 +1 @@
-VERSION = '2.13'
+VERSION = '2.12'


yes, as i mentioned, i forked off version 2.12 then rebased the current master onto my repo

horejsek · 2019-10-01T16:01:01Z

fastjsonschema/generator.py


    # pylint: disable=invalid-name
    @indent
-    def l(self, line, *args, **kwds):
+    def l(self, line, *args, deduplicate=False, expand_locals=True, **kwds):  # noqa


This library is using pylint.

correct. can be removed. (as i mentioned, the library code lives in the project and i had to lint it with existing tools. when the pull request gets merged, i can happily go using the official version)

I see. This merge request is actually a patch of changes you did for your copy of this library in your app.

horejsek · 2019-10-01T16:04:46Z

fastjsonschema/generator.py

+        if self._verbose:
+            message += '\\n  caused by {path} {rule}'
+            kwargs.update(path=self.context_path(), rule=self._definition)
+        self.l('raise JsonSchemaException("{}")'.format(message if expand else ''),


Is there some real performance gain with this probing? If not, I think it's then uselessly complex.

not really, but when the the _definition is huge... it is huge in the compiled schema

imi added 3 commits September 30, 2019 07:45

my 2 cents

767f39f

extend applicable parameters for compile* functions

842e7d3

change generate_format to be applicable only to string types

4ede4f8

horejsek reviewed Oct 1, 2019

View reviewed changes

horejsek mentioned this pull request Oct 8, 2019

Keep the variable name and path through references #63

Closed

horejsek closed this Oct 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

my 2 cents #61

my 2 cents #61

isomogyi commented Sep 30, 2019

horejsek left a comment

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019 •

edited

Loading

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 8, 2019

isomogyi Oct 8, 2019

horejsek Oct 9, 2019

isomogyi Oct 19, 2019

horejsek Oct 27, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

horejsek Oct 1, 2019

isomogyi Oct 1, 2019

		@@ -1 +1 @@
		VERSION = '2.13'
		VERSION = '2.12'

my 2 cents #61

my 2 cents #61

Conversation

isomogyi commented Sep 30, 2019

horejsek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

isomogyi Oct 1, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

isomogyi Oct 1, 2019 •

edited

Loading