Skip to content
A Modern and Flexible Web Client for R
C R Other
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
R Check for improperly specified smtp-server parameter #195 Jul 31, 2019
bugs Render docs Jun 22, 2018
docker/r-devel-san MASS rename Mar 14, 2017
examples Trigger build Aug 17, 2018
inst
man
revdep Update revdep Jul 18, 2017
src Do 'not' use vanilla (respect local proxy settings) Aug 1, 2019
tests Download to temporary file, fixes #172 Jul 26, 2019
tools Make curl_symbols a function instead of a table Jun 11, 2019
vignettes Tweak vignette a bit Jul 9, 2018
.Rbuildignore Make curl_symbols a function instead of a table Jun 11, 2019
.gitignore Make curl_symbols a function instead of a table Jun 11, 2019
.travis.yml Test Travis bionic Jul 18, 2019
DESCRIPTION
LICENSE Update news and symbol table Jun 11, 2019
NAMESPACE Make curl_symbols a function instead of a table Jun 11, 2019
NEWS Download to temporary file, fixes #172 Jul 26, 2019
README.md Trigger build Jun 24, 2017
appveyor.yml
cleanup
configure
configure.win Stub for windows Jan 7, 2015
curl.Rproj More NSlookup Jan 23, 2016
httpbin.md rebase master onto v1.2 Sep 12, 2016
release.R Update release file Apr 12, 2017

README.md

curl

A Modern and Flexible Web Client for R

Build Status AppVeyor Build Status Coverage Status CRAN_Status_Badge CRAN RStudio mirror downloads Research software impact Github Stars

The curl() and curl_download() functions provide highly configurable drop-in replacements for base url() and download.file() with better performance, support for encryption (https, ftps), gzip compression, authentication, and other libcurl goodies. The core of the package implements a framework for performing fully customized requests where data can be processed either in memory, on disk, or streaming via the callback or connection interfaces. Some knowledge of libcurl is recommended; for a more-user-friendly web client see the 'httr' package which builds on this package with http specific tools and logic.

Devel version

Note that you cannot use devtools::install_github() because it uses curl ;)

install.packages("https://github.com/jeroen/curl/archive/master.tar.gz", repos = NULL)

Documentation

About the R package:

Other resources:

Hello World

There are three download interfaces (memory, disk and streaming). Always start by setting up a request handle:

library(curl)
h <- new_handle(copypostfields = "moo=moomooo")
handle_setheaders(h,
  "Content-Type" = "text/moo",
  "Cache-Control" = "no-cache",
  "User-Agent" = "A cow"
)

Perform request and download response in memory:

# Perform the request
req <- curl_fetch_memory("http://httpbin.org/post", handle = h)

# Show some outputs
parse_headers(req$headers)
cat(rawToChar(req$content))
str(req)

Or alternatively, write response to disk:

tmp <- tempfile()
curl_download("https://httpbin.org/post", tmp, handle = h)
readLines(tmp)

Or stream response via Connection interface:

con <- curl("https://httpbin.org/post", handle = h)
open(con)

# Get 3 lines
readLines(con, n = 3)

# Get remaining lines and close connection
readLines(con)
close(con)

Installation

Binary packages for OS-X or Windows can be installed directly from CRAN:

install.packages("curl")

Installation from source on Linux requires libcurl. On Debian or Ubuntu use libcurl4-openssl-dev:

sudo apt-get install -y libcurl-dev

On Fedora, CentOS or RHEL use libcurl-devel:

sudo yum install libcurl-devel

On OS-X libcurl is included with the system so nothing extra is needed. However if you want to build against the most recent version of libcurl, install and force-link curl from homebrew:

brew install curl
brew link --force curl

Note that on OS-X you must recompile the R package from source after force-linking curl, otherwise you get a version conflict with the system version of libcurl.

Development version

Because devtools and httr depend on curl, installing with install_github does not work well. The easiest way to install the development version of curl is a clean R session:

install.packages("https://github.com/jeroen/curl/archive/master.tar.gz", repos = NULL)

Of course windows users need Rtools to compile from source.

Enable HTTP/2

To use HTTP/2 the libcurl library has to be built --with-nghttp2 which is usually not the default. Use curl_version() in R to check if it is enabled. To enable it on OSX use:

brew install curl --with-nghttp2
brew link curl --force

And then reinstall the curl package from source:

install.packages("https://github.com/jeroen/curl/archive/master.tar.gz", repos = NULL)

Check again curl_version() to see if HTTP/2 is enabled now.

You can’t perform that action at this time.