Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected END_OF_INPUT error (URL parsing?) #3

Closed
jdnewmil opened this issue Feb 27, 2018 · 11 comments
Closed

Unexpected END_OF_INPUT error (URL parsing?) #3

jdnewmil opened this issue Feb 27, 2018 · 11 comments

Comments

@jdnewmil
Copy link

The attached zip file contains an RStudio notebook file illustrating the error.

If I remove the url field from the bibtex entry, get_bibentries works without a problem, so it looks to me like there is a parsing problem with URLs (perhaps when % encoded characters are present?)

test.zip

@GeoBosh
Copy link
Owner

GeoBosh commented Feb 28, 2018

Thanks for the report. As you suspected, the error is caused by the symbols encoded with the html % codes. If the "%-encoded letters are replaced with the actual letters, as in
"http://www.physics.arizona.edu/~cronin/Solar/References/Irradiance Models and Data/GUE93.pdf",
the command works:

testbib2 <-
"@article{gueymard_critical_1993,
title = {Critical {Analysis} and {Performance} {Assessment} of {Clear} {Sky} {Solar} {Irradiance} {Models} using {Theoretical} and {Measured} {Data}},
 volume = {51},
 url = {http://www.physics.arizona.edu/~cronin/Solar/References/Irradiance Models and Data/GUE93.pdf},
 number = {2},
 journal = {Solar Energy},
 author = {Gueymard, Christian A.},
 year = {1993},
 pages = {121--138}
 }"

bibfname2 <- tempfile()
writeLines( text = testbib2, con = bibfname2)
b <- get_bibentries( bibfile = bibfname2 )
b

In fact, the command succeeds also with the percent-encoded url - the error comes only when the result is printed, since the percents confuse the default print method. This is a known conundrum with bibentry objects. If you assign the result, you can then print it in various styles:

testbib <- "@article{gueymard_critical_1993,\n\ttitle = {Critical {Analysis} and {Performance} {Assessment} of {Clear} {Sky} {Solar} {Irradiance} {Models} using {Theoretical} and {Measured} {Data}},\n\tvolume = {51},\n\turl = {http://www.physics.arizona.edu/%7ecronin/Solar/References/Irradiance%20Models%20and%20Data/GUE93.pdf},\n\tnumber = {2},\n\tjournal = {Solar Energy},\n\tauthor = {Gueymard, Christian A.},\n\tyear = {1993},\n\tpages = {121--138}\n}\n"
bibfname <- tempfile()
writeLines( text = testbib, con = bibfname  )
a <- get_bibentries( bibfile = bibfname )

print(a, style = "html")
print(a, style = "BibTeX")
print(a, style = "citation")

(edit on 2018-03-01 - the example with percents gives errors - I somehow messed up the testing of the above code and didn't get errors the other day.)

Would this resolve your problem?

bug.zip

@jdnewmil
Copy link
Author

Thanks for following up, but the workaround doesn't resolve my problem:

a) The percents are needed to make the URL clearly distinguishable as having single spaces [1]

b) I am actually using the insertRef macro in roxygen documentation per Rdpack recommendation.

c) I think the parameter option is "format"?

d) The output of "print(a, format="latex")" still has a superflous period and closing curly brace that confuses LaTeX. The presence of these characters may be the original trigger for the other problems.

[1] https://stackoverflow.com/questions/497908/is-a-url-allowed-to-contain-a-space

@GeoBosh
Copy link
Owner

GeoBosh commented Feb 28, 2018

As you get the error when using insertRef macro, that is a bug which I will need to sort out somehow.
Have you checked if it works when you produce html documentation instead of pdf?

I think that the argument for print is "style", see ?print.bibentry but I see the error with print(a, style = "latex"), as well.

@jdnewmil
Copy link
Author

jdnewmil commented Mar 1, 2018

I am seeing the both the error and spurious brace+period with several versions of your print statement [1]. As for building the help in html... the warning messages and extra brace are in the html print output anyway.

[1] test2.zip

@GeoBosh
Copy link
Owner

GeoBosh commented Mar 1, 2018

I pushed a draft solution for the problem - insertRef should now work smoothly with percent encoded entries. Could you check? For the moment I have actually included the reference from your example in REFERENCES.bib in Rdpack and it is cited in ?inserRef - I checked that the links are ok in the html and pdf versions of the doc.

Please note that I have not touched get_bibentries yet.

@jdnewmil
Copy link
Author

jdnewmil commented Mar 2, 2018

v0.5-6 fixed insertref, which solves my immediate problem... thank you! As you indicated, get_bibentries still gives warnings and prints extra characters after the url.

@jdnewmil
Copy link
Author

jdnewmil commented Mar 2, 2018

Looking forward to v0.6 so I can add a minimum version dependency in my DESCRIPTION file. Or, use two-dot version notation...

@GeoBosh
Copy link
Owner

GeoBosh commented Mar 3, 2018

I cleaned up a little bit and updated also get_bibentries(), further details are oin NEWS. By the way, do you use get_bibentries() or just came to it in the process of investigating the bug concerning insertRef?

I will bump the version to 0.6 once I do a further round of testing or get a confirmation from you that it looks ok. If you are about to submit a package depending on it on CRAN, I will publish it but otherwise I would prefer to delay a little bit since I had two releases in less than two months.

@jdnewmil
Copy link
Author

jdnewmil commented Mar 3, 2018 via email

@GeoBosh
Copy link
Owner

GeoBosh commented Mar 3, 2018

For R it doesn't matter, 0.5.6 and 0.5-6 are the same, you can use dots. I probably didn't know that 8-10 years ago when I started using it and equally there seems little point in changing it now.

Thanks for going into the trouble of searching for the cause of the warnings. If you wonder what is the difference between get_bibentries and insert_ref - here It is. insertRef needs to convert the reference to Rd format by definition. I actually think that the bug there is in tools::toRd, which I use to do that. There are two reasons why I thing this is so. First, WRE explicitly stated that special letters should be percent encoded and the percents escaped. Second, leaving the percents unescaped in Rd text creates invalid Rd text, as you discovered. Even more,there is a case to argue that toRd should actually percent encoded special symbols.

On the other hand, get_bibentries creates a bibentry object and in principle it is not clear if percents should be escaped there. Actually, what matters is how these are exported by the print methods for the different styles. There is no universally applicable solutions since some hinted styles do the escaping themselves, but most simply transfer the content of the URL field.
Now get_bibentries() returns an object from a class with a print method which ensures (or so I hope) that the links produced in the basic styles are fine. (The processing is not ideal but can be remedied if the need arises.)

@GeoBosh
Copy link
Owner

GeoBosh commented Mar 10, 2018

After the debugging and improvement of get_bibentries() prompted by your comments, I switched \insert_ref to calling it. \insertRef now handles percent encoded URL's everywhere, not only in the url field.
(%'s used for other purposes may cause trouble but can be dealt with if the need arises).

It is sufficient to require Rdpack (>= 0.6).

Thanks again.

@GeoBosh GeoBosh closed this as completed Mar 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants