Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading files with "rb" causes issues in Windows #162

Closed
pauloscustodio opened this issue Nov 6, 2016 · 2 comments

Comments

@pauloscustodio
Copy link
Contributor

commented Nov 6, 2016

In input.cc:18 the .re file is opened in binary mode - fopen (file_name.c_str (), "rb");
This causes a test failures on a git clone of the repository on Windows:

  • code_points_error_esc.re:
    re2c: warning: line 2: column 5: escape has no effect: '' [-Wuseless-escape]

because the back-slash is followed by a '\r' and then a '\n'

  • code_points_error_lf.re
    re2c: error: line 2, column 7: syntax error

column number is 6 is the reference file, result is 7 due to extra '\r'

In git, when a repository is cloned in Windows, all text files get '\r\n' line endings, which are then converted to '\n' on commit. By opening in binary mode in input.cc, the '\r' are not filtered by stdio.

skvadrik added a commit that referenced this issue Nov 7, 2016

Recognize newlines in character strings and classes.
As for now, newline inside of a character string or class is an error:
re2c should emit clear error message. Different styles of newlines
should be recognized ("\n", "\r\n").

This commit fixes bug #162 reported by pauloscustodio:
    Reading files with "rb" causes issues in Windows
@skvadrik

This comment has been minimized.

Copy link
Owner

commented Nov 7, 2016

I think the issue is not in "rb" mode, rather in handling of newlines. re2c should handle all kinds of newlines, and it does for the most part, but in some places newlines are unexpected.

Please see the fix: 54b50c0,
does it work for you?

@pauloscustodio

This comment has been minimized.

Copy link
Contributor Author

commented Nov 7, 2016

Yes, 54b50c0
54b50c0
solves
the issue I reported.

Thanks.

On Mon, Nov 7, 2016 at 10:00 AM, skvadrik notifications@github.com wrote:

I think the issue is not in "rb" mode, rather in handling of newlines.
re2c should handle all kinds of newlines, and it does for the most part,
but in some places newlines are unexpected.

Please see the fix: 54b50c0
54b50c0
,
does it work for you?


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#162 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEUdbgWYL1vWzH_4gBCY7bn3_rMT8apks5q7vbYgaJpZM4Kqtd0
.

@skvadrik skvadrik closed this Nov 8, 2016

skvadrik added a commit that referenced this issue Jul 30, 2017

Fixed line endings in output files on Windows (#162, #163).
This fix consists of two issues, both reported and fixed by pauloscustodio.

1. #162 "Open text files with "wb" causes issues on Windows"

    Text files need to be opened for writing with "w", so that stdio does
    the right thing in respect to the correct line endings for the current OS.
    ("\r\n" in Windows, "\n" in Linux).

2. #163 "Reading files with "rb" causes issues in Windows"

    re2c reads input files in binary mode and writes the generated output in
    text mode. This caused CR LF conversion to CR CR LF on Windows: first CR
    comes from reading input in binary mode, second CR is added when writing
    output in text mode. This only happened to those parts of input which are
    not transformed by re2c: we used to copy-paste verbatim, now we patch line
    endings. Now we convert all line endings to LF before writing the generated
    code to file.

skvadrik added a commit that referenced this issue Jul 30, 2017

Recognize newlines in character strings and classes.
As for now, newline inside of a character string or class is an error:
re2c should emit clear error message. Different styles of newlines
should be recognized ("\n", "\r\n").

This commit fixes bug #162 reported by pauloscustodio:
    Reading files with "rb" causes issues in Windows
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.