Data import error

[e-molitor]

Dear getTBinR gurus,

this is a great application for anyone interested in tuberculosis data. Sadly enough, however, I came across a problem. After installation according to the vignette, trying to use

tb_burden <- get_tb_burden()

leads to apparant downloading of the data, they are not read in correctly, however. The screen log is:

tb_burden <- get_tb_burden()
Downloading data from: https://extranet.who.int/tme/generateCSV.asp?ds=estimates
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 272k 0 272k 0 0 192k 0 --:--:-- 0:00:01 --:--:-- 194k
Downloading the data using fread::data.table has failed. Trying
again using utils::read.csv
Downloading data has failed after 1 tries.
Attempting data download in 3.4 seconds.
Downloading data from: https://extranet.who.int/tme/generateCSV.asp?ds=estimates
Downloading the data using fread::data.table has failed. Trying
again using utils::read.csv
Downloading data has failed after 2 tries.
Attempting data download in 2.9 seconds.
Downloading data from: https://extranet.who.int/tme/generateCSV.asp?ds=estimates
Downloading the data using fread::data.table has failed. Trying
again using utils::read.csv
Downloading data has failed after 3 tries.
Attempting data download in 4.4 seconds.
Downloading data from: https://extranet.who.int/tme/generateCSV.asp?ds=estimates
Downloading the data using fread::data.table has failed. Trying
again using utils::read.csv
Downloading data has failed after 4 tries.
Attempting data download in 2.5 seconds.
Downloading data from: https://extranet.who.int/tme/generateCSV.asp?ds=estimates
Downloading the data using fread::data.table has failed. Trying
again using utils::read.csv
Downloading data has failed after 5 tries.
Attempting data download in 1.2 seconds.
Fehler in get_data(url = url, download_data = download_data, data_trans_fn = trans_burden_data, :
Data downloading has failed, check your internet connection.
If this issue is not resolved, contact the package author.
Zusätzlich: Warnmeldungen:
1: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
URL 'https://extranet.who.int/tme/generateCSV.asp?ds=estimates': status was 'Failure when receiving data from the peer'
2: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
URL 'https://extranet.who.int/tme/generateCSV.asp?ds=estimates': status was 'Failure when receiving data from the peer'
3: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
URL 'https://extranet.who.int/tme/generateCSV.asp?ds=estimates': status was 'Failure when receiving data from the peer'
4: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
URL 'https://extranet.who.int/tme/generateCSV.asp?ds=estimates': status was 'Failure when receiving data from the peer'
5: In scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
URL 'https://extranet.who.int/tme/generateCSV.asp?ds=estimates': status was 'Failure when receiving data from the peer'

Using wget https://extranet.who.int/tme/generateCSV.asp?ds=estimates , the data are read into a file on my local machine (GNU/Kubuntu Linux 16.04.3 LTS running Linux 4.15.5, R 3.4.3) like a charm.

I'd love a clue on how I should proceed to have the data read in properly.

Best wishes and regards,

Ernst

[e-molitor]

Dear getTBinR gurus,

after some browsing through your well-documented and readable sources, I found the error lay
completely within my own installation: R has been linked to an outdated version of curl; installing the current version fixes the problem I have mistakenly thought to be caused by your impressive package. Sorry for my mistake.

[seabbs]

Hi e-molitor,

Thanks for opening this - sorry you had an issue using the package. It sounds like you have resolved your immediate problem - this actually links to a wider issue that I have been looking at for a while (see CRAN checks!

I hadn't been able to find the download issue (and tbh thought it might be transient and related to the WHO database) - I am adding a patch now so that data download works with an outdated version of curl. It would be great if you could share the version you were on before you upgraded?

Best,

Sam

[e-molitor]

Dear Sam, thank you so much for providing this most useful and impressive package. I love teaching, and my slides will become much better due to the beautiful graphs your package allows to arrive at so easily. Trying to find the cause of the problem I have seen, I started trying to use read.csv together with url() and curl(). This led to an error message describing a GNU TLS problem. The message helped my to spot that differen curl libraries live on my computer in different directories. My Kubuntu-16.04.3 seems to use curl-7.47.0; I have installed curl-7.58.0 from source. Due to libraries living in directories I would not have expected to contain them, e.g., /usr/lib/x86_64-linux-gnu, and due to different versions of libcurl, libcurl.so.4.4.0 and libcurl-gnutls.so.4.4.0, in this location, R and/ or the CRAN-curl-package linked to libraries different from those the header files belonged to after I had installed the current version from source. Only after introducing links libcurl.so.4 -> libcurl.so.4.5.0 and libcurl-gnutls.so.4 -> libcurl.so in /usr/lib/x86_64-linux-gnu the problem I had seen with fetchting the data from the WHO site has disappeared. Obviously, manually introducing links like those is very much different from proper maintenance of a computer system, but I don't have time (and abilities) to dig deep enough to find out why the different curl libraries exist on my box and what should, canonically, be done to fix this issue properly (I'm refraining from calling it a mess because I assume there are good reasons for it I'm, sadly enough, unable to understand for the time beeing). Please do not hesitate to contact me if you feel I could offer any useful information. Again,thank you loads for your package. Kindest regards, Yours Ernst Am Sonntag, 25. Februar 2018, 13:19:54 CET schrieb Sam Abbott: Hi e-molitor, Thanks for opening this - sorry you had an issue using the package. It sounds like you have resolved your immediate problem - this actually links to a wider issue that I have been looking at for a while (see CRAN checks[1]! I hadn't been able to find the download issue (and tbh thought it might be transient and related to the WHO database) - I am adding a patch now so that data download works with an outdated version of curl. It would be great if you could share the version you were on before you upgraded? Best, Sam —You are receiving this because you modified the open/close state.Reply to this email directly, view it on GitHub[2], or mute the thread[3].[4]

…

-- Dr. Ernst Molitor Deputy Director Institute of Medical Microbiology, Immunology & Parasitology University Hospital Bonn

[seabbs]

I am glad you have found it useful. It is great to hear that you are using it to teach! Let me know (i.e open an issue) if you find a feature that you think is missing whilst using the package. This could be an additional plotting function for example.

Hmm sounds interesting and as you say quite complicated. I have added an additional download route to the package (via download.file) in case this sort of issue happens again. I am not sure that this will completely prevent a reoccurrence though. For information the pull request is here.

Going to close this issue for now as I think the problem has been fixed. I may add a snapshot of the data to the package at a later date if these issues continue.

Thanks again for the detailed information.

Sam