Skip to content

Removing gson and everit dependencies #238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 10, 2023

Conversation

fjtirado
Copy link
Collaborator

@fjtirado fjtirado commented Jul 6, 2023

Many thanks for submitting your Pull Request ❤️!

What this PR does / why we need it:
It removed org.json and everit validation dependencies and add json-schema-validator for validation.
See https://github.com/everit-org/json-schema#when-to-use-this-library
Since we are using Jackson for serialization/deserialization of the workflow, we should use https://github.com/networknt/json-schema-validator instead of everit, and hence avoid having dependencies to two different json parsers.

Special notes for reviewers:
This remove unneeded dependencies for poms and slightly change the validator interface (it now returns a Collection of ValidationErrors rather than a List, which easily remove duplicated validation messages)

Additional information (if needed):

@ricardozanini
Copy link
Member

Although this is very important, we need to handle the CVEs while they don't release a new version:

https://mvnrepository.com/artifact/com.github.java-json-tools/json-schema-validator/2.2.14

  1. Upgrade Guava - CVE-2023-2976 and CVE-2020-8908
  2. Upgrade testng-core - CVE-2022-4065

@fjtirado fjtirado marked this pull request as draft July 6, 2023 12:56
Signed-off-by: Francisco Javier Tirado Sarti <ftirados@redhat.com>
@ricardozanini
Copy link
Member

We can instead use this: https://github.com/networknt/json-schema-validator

@fjtirado
Copy link
Collaborator Author

fjtirado commented Jul 6, 2023

Although this is very important, we need to handle the CVEs while they don't release a new version:

https://mvnrepository.com/artifact/com.github.java-json-tools/json-schema-validator/2.2.14

1. Upgrade Guava - [CVE-2023-2976](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2976) and [CVE-2020-8908](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908)

2. Upgrade testng-core - [CVE-2022-4065](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-4065)

To remove this CVEs, lets use https://github.com/networknt/json-schema-validator, reworking the PR

@tsurdilo
Copy link
Contributor

tsurdilo commented Jul 6, 2023

Will review. Just note that if you add this much refactoring would be nice to add some additional tests for it in addition to existing.

@fjtirado fjtirado force-pushed the removing_everit branch 2 times, most recently from 97a7857 to b12ea91 Compare July 6, 2023 15:24
@fjtirado fjtirado marked this pull request as ready for review July 6, 2023 15:25
@fjtirado fjtirado requested a review from tsurdilo July 6, 2023 15:25
Signed-off-by: Francisco Javier Tirado Sarti <ftirados@redhat.com>
@tsurdilo
Copy link
Contributor

thanks for the updates. i would really like to request some more tests since its a bigger refactor. can we add some positive/negative tests (and not just one please :) )

@fjtirado
Copy link
Collaborator Author

thanks for the updates. i would really like to request some more tests since its a bigger refactor. can we add some positive/negative tests (and not just one please :) )

@tsurdilo Since this PR, despite apparently large, is not adding any new functionality (just keeping the existing one using a library that avoid dependencies that bring vulnerabilities), fortunately (kudos for the original writer ;)), the tests that are related with the change: basically serialization/deserialization and validation, were already there. In fact, they were changed as result of this change because the valitadion error messaged are library dependent.


import com.fasterxml.jackson.databind.ObjectMapper;

public class JsonObjectMapperFactory {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure i get point of this class. Just maybe have static instance of yaml/json mappers in a util class?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This avoid creating a new object mapper instance every time is needed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok got it. i think since you return just the basic "new" mapper just define it in static field in some util class no need imo
for separate class. if you want to allow users to set specific configuration for the mappers then having it in some separate class like this maybe makes more sense

Copy link
Collaborator Author

@fjtirado fjtirado Jul 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done in a separate class to only initialize the mapper object when it is really used. If you put it into an utils class it will get initialized the firts time you use the utilities class, even in the utility method being used is unrelated (imagine an use that never use yaml format)


import com.fasterxml.jackson.databind.ObjectMapper;

public class YamlObjectMapperFactory {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, to avoid creating multiple object mapper instances

@@ -1,5 +1,5 @@
{
"$id": "https://wg-serverless.org/workflow.schema.json",
"$id": "classpath:schema/workflow.schema.json",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this change needed?

Copy link
Collaborator Author

@fjtirado fjtirado Jul 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new validation library tries to download the schema from a not existing url (Because wg-serverless-org is fake), hence changing to real location (which is library classpath)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tell me if im wrong you already define the hard-coded path to the schema in code, if so then why here? i think it was by design to put some id like this and not use classpath

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this id path is used to upload other schemas referred from this schema
So, if we leave wg-.serverless.org, the library will fail to retrieve, any nested shema, for example, start.json.

@tsurdilo
Copy link
Contributor

i understand what you are saying as in you modified the existing validation tests some and they still pass.
what i am saying is would be nice to add more than just the existing 9 and possibly catch issues the current one might be
missing

@fjtirado fjtirado force-pushed the removing_everit branch 2 times, most recently from 8fabaf5 to 39f17ff Compare July 12, 2023 12:13
@fjtirado
Copy link
Collaborator Author

i understand what you are saying as in you modified the existing validation tests some and they still pass. what i am saying is would be nice to add more than just the existing 9 and possibly catch issues the current one might be missing

I added a couple of tests to ensure that workflows with functions and events are properly validated

Signed-off-by: Francisco Javier Tirado Sarti <ftirados@redhat.com>
@fjtirado fjtirado requested a review from tsurdilo July 12, 2023 13:48
Signed-off-by: Francisco Javier Tirado Sarti <ftirados@redhat.com>
@tsurdilo
Copy link
Contributor

tsurdilo commented Jul 13, 2023

I think the validation piece (List/Collection + the added imho ambiguous uniqueness constraint) is whats holding up the pr
I dont have issue making the library change, but the pr also adds this other piece.
Would help maybe if you can remove that added uniques constraint and only have in PR the lib change stuff and then we can unblock the actual change this pr i think intends to doand discuss that uniqueness stuff in separate pr so we don't hold up this one?

Signed-off-by: Francisco Javier Tirado Sarti <ftirados@redhat.com>
@fjtirado fjtirado requested a review from tsurdilo July 13, 2023 14:16
@ricardozanini
Copy link
Member

@tsurdilo I think this PR is ready since the conversation about List or Set has been settled down and the commit reverted back to what it was?

@ricardozanini ricardozanini merged commit 7324827 into serverlessworkflow:main Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants