Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Escaping quotes doesn't work correctly #177

Closed
vpzomtrrfrt opened this issue Jan 25, 2019 · 5 comments

Comments

Projects
None yet
2 participants
@vpzomtrrfrt
Copy link
Member

commented Jan 25, 2019

Attempting to escape a single quote within a string seems to add extra characters after it.

  • very a is '\'' is turned into var a = '\' ';
  • plz foo with '\'' is turned into foo('\', ');

@AnEmortalKid AnEmortalKid added the Bug label Jan 25, 2019

@AnEmortalKid

This comment has been minimized.

Copy link
Member

commented Jan 25, 2019

Probably something wrong with our regex https://regex101.com/r/i2zi27/1/

[^']+|\s+

I guess this could be solved by a non-regex based matching (lexer?) that returns the "tokens"

@AnEmortalKid

This comment has been minimized.

Copy link
Member

commented Feb 23, 2019

Well...... im a bit puzzled at what's happening here:

$ node
> var line = "plz foo with '\''";
undefined
> line
'plz foo with \'\'\''
> line.match(/'[^']+'|\S+/g)
[ 'plz', 'foo', 'with', '\'\'\'' ]
> '\''.match(/'[^']+'|\S+/g)
[ '\'' ]

So the 'tokenizer' is identifying 4 chunks, at least through node. When I debug through it it somehow matches 5:

image

@AnEmortalKid

This comment has been minimized.

Copy link
Member

commented Feb 23, 2019

I have something partial here (before trying to swap out the tokenizer, I wanted to test it out)

https://github.com/AnEmortalKid/dogescript/blob/tokenizer/test/tokenizer.test.js#L19

Surprisingly, that last test doesn't fail.

@AnEmortalKid

This comment has been minimized.

Copy link
Member

commented Mar 26, 2019

I think the problem might be related to something escaping the input:

$ cat escaped.djs
very a is '\''
$ dogescript escaped.djs
var a = '\' '
;

IN REPL

DOGE> very a is '\''
undefined
DOGE> undefined
DOGE> a
'\' '
@AnEmortalKid

This comment has been minimized.

Copy link
Member

commented Mar 26, 2019

@vpzomtrrfrt I think i might have a fix for this:

DOGE> very a is '\''
undefined
DOGE> a
'\''

However, some of the tests sort of expect extra lines......

# multi-props
not ok 202 should be equal
  ---
    operator: equal
    expected: |-
      'function foo() {\n    // 1\n}\n\nfunction bar(arg) {\n    // 2\n}\n\nmodule.exports.boo = foo\nmodule.exports.far = bar'
    actual: |-
      'function foo() {\n    // 1\n}\n\nfunction bar(arg) {\n    // 2\n}\nmodule.exports.boo = foo\nmodule.exports.far = bar'

@AnEmortalKid AnEmortalKid self-assigned this Mar 26, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.