Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON5 parser #381

Merged
merged 10 commits into from May 11, 2021
Merged

Add JSON5 parser #381

merged 10 commits into from May 11, 2021

Conversation

msssk
Copy link
Contributor

@msssk msssk commented Jun 11, 2020

dojo/parser relies on eval to parse data attributes. For data attributes that are JSON5 compliant a JSON5 parser can be used as a drop-in replacement which improves Dojo's ability to be used with a strict CSP policy that does not allow unsafe-eval.

This is a non-breaking change with opt-in security improvements. Any Dojo application that is currently deployed in a strict CSP environment with has: { 'csp-restrictions': true } is restricted from using dojo/parser. With this change dojo/parser can be used and will parse all valid JSON5 data attributes.

Invalid JSON5 (e.g. JS expressions) will result in an error in the console:

Error: SyntaxError: JSON5: invalid character 'h' at 1:24 in data-dojo-props='lang: 'es-es', prop: this.ownProp'

Dojo applications that are currently using JS expressions in data attributes can safely upgrade. As long as they don't specify has: { 'csp-restrictions': true } then eval() will continue to be used and existing behavior and functionality will be preserved.

There is a scenario where this could be a breaking change:

  • CSP not configured OR configured to allow unsafe-eval
  • Dojo configured with has: { 'csp-restrictions': true }
  • data attributes that contain invalid JSON5 (e.g. JS expressions)

In this case upgrading to a Dojo release incorporating this patch would result in errors when parsing invalid data attributes.

Closes #380

@msssk
Copy link
Contributor Author

msssk commented Jun 11, 2020

This PR is in draft: review and input is welcome.

If we do decide to add a JSON5 parser to Dojo we'll have to figure out how we would do so. One simple possibility is to continue in the manner presented in this PR - copy the MIT-licensed source almost unmodified from the JSON5 project.

@wkeese
Copy link
Member

wkeese commented Jun 11, 2020

Nicely done. Will it satisfy people worried about security or will they still be upset because dojo/parser has a call to eval()?

@msssk
Copy link
Contributor Author

msssk commented Jun 11, 2020

This can be tested in a test app I created.

will they still be upset because dojo/parser has a call to eval()?

I haven't tested a build yet but the way this is implemented if has: { 'csp-restrictions': true } is configured in the build then dead code removal should completely remove the call to eval() in dojo/parser.

@wkeese
Copy link
Member

wkeese commented Jun 11, 2020

Nice!

@msssk msssk marked this pull request as ready for review June 16, 2020 04:24
@msssk
Copy link
Contributor Author

msssk commented Jun 16, 2020

I've added tests and the MIT license, and a README for process I'm using to import the JSON5 code into Dojo and propose be used for ongoing maintenance (feedback is welcome):
https://github.com/dojo/dojo/blob/d761911d74869ba7d3edab46727debab5fae6002/json5/README.md

Copy link
Member

@dylans dylans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me. We can land this for 1.17 (new feature so no backporting).

@msssk
Copy link
Contributor Author

msssk commented Jun 19, 2020

I'm still looking into some more uses of eval (via new Function) that I'd like to address as part of this PR.

dojo/parser relies on `eval` to parse data attributes. For data attributes
that are JSON5 compliant a JSON5 parser can be used as a drop-in replacement
which improves Dojo's ability to be used with a strict CSP policy that
does not allow `unsafe-eval`
Previously the error was silently swallowed
Add ES5 String methods used by JSON5 parser
Add JSON5 readme
string: add codePoint tests
@msssk
Copy link
Contributor Author

msssk commented Jun 23, 2020

I have reviewed use of new Function in dojo/parser and I don't think there's more to be gained by using JSON5 or other changes. We just need to include in Dojo+CSP instructions to avoid using declarative Dojo scripts (<script type="dojo/*">) and I believe we are already covering avoidance of JS expressions and non-Dojo globals (string references to the dojo global like dojo.byId can still be resolved without eval or parsing).

I'm good with this PR being merged as-is.

@schallm
Copy link
Contributor

schallm commented Jun 23, 2020

Sorry this has take a while, but my initial testing is very promising. I did have an issue building my own layers as json5\parse does have some trailing commas. Once I removed those, my app does seem to work. I will do some more testing, but just wanted to say testing has started...

 json5/parse.js | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/json5/parse.js b/json5/parse.js
index 738cb352..ac206138 100644
--- a/json5/parse.js
+++ b/json5/parse.js
@@ -540,14 +540,14 @@ define([
 		},
 		end: function () {
 			throw invalidChar(read());
-		},
+		}
 	};
 	function newToken(type, value) {
 		return {
 			type: type,
 			value: value,
 			line: line,
-			column: column,
+			column: column
 		};
 	}
 	function literal(s) {
@@ -713,7 +713,7 @@ define([
 			}
 		},
 		end: function () {
-		},
+		}
 	};
 	function push() {
 		var value;
@@ -811,7 +811,7 @@ define([
 			'\v': '\\v',
 			'\0': '\\0',
 			'\u2028': '\\u2028',
-			'\u2029': '\\u2029',
+			'\u2029': '\\u2029'
 		};
 		if (replacements[c]) {
 			return replacements[c];

@msssk
Copy link
Contributor Author

msssk commented Jun 23, 2020

@schallm thank you for the report! Trailing commas removed

@changpeng789
Copy link

I download the pr change and do the test.
There is an error :

parser.js:915 dojo/parser::parse() error Error: SyntaxError: JSON5: invalid character 'h' at 1:14 in data-dojo-props='container: this, iconClass: 'addIcon', label: 'Add Storage''
at Object.construct (parser.js:416)
at Object. (parser.js:200)
at Object.map (array.js:285)
at Object._instantiate (parser.js:194)
at parser.js:908
at signalListener (Deferred.js:37)
at Promise.Deferred.then.promise.then (Deferred.js:258)
at Object.parse (parser.js:907)
at Object._beforeFillContent (_WidgetsInTemplateMixin.js:45)
at Object.buildRendering (_AttachMixin.js:90)

Could you help resolve this issue? Thanks.

Copy link
Contributor

@edhager edhager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Good work Mangala.

@edhager
Copy link
Contributor

edhager commented May 11, 2021

I download the pr change and do the test.
There is an error :

parser.js:915 dojo/parser::parse() error Error: SyntaxError: JSON5: invalid character 'h' at 1:14 in data-dojo-props='container: this, iconClass: 'addIcon', label: 'Add Storage''
at Object.construct (parser.js:416)
at Object. (parser.js:200)
at Object.map (array.js:285)
at Object._instantiate (parser.js:194)
at parser.js:908
at signalListener (Deferred.js:37)
at Promise.Deferred.then.promise.then (Deferred.js:258)
at Object.parse (parser.js:907)
at Object._beforeFillContent (_WidgetsInTemplateMixin.js:45)
at Object.buildRendering (_AttachMixin.js:90)

Could you help resolve this issue? Thanks.

@changpeng789 This is an example of what @msssk pointed out in the PR description. Your props attribute does not contain valid JSON. More specifically, the container property value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dojo would benefit from adding a JSON5 parser
6 participants