Swaggerfile support for array parameters, object, files

[meztez]

Fixes #187

Add swagger support for array parameters per https://swagger.io/docs/specification/serialization/ using square brackets around typed parameters.

Add swagger support for openapi object type to be listed under requestBody using plumber types list, df and data.frame.

Add swagger/plumber support for files upload, multipart content-type

Add support for detection of endpoint plumberExpression parameters.

Add support for custom body parsers, like serializers, more raw.

Around 30% overheads reduction for POST request with json body.

Syntax support for parameters would become :

• in: query:
  • name
  • name:type
  • name:type*
  • new name:[type]*
• in: path
  • <name>
  • <name:type>
  • new <name:[type]>

So something like below works with a matching openapi file to go with it.

library(plumber)

#* Sum numbers
#* @get /names3/<number:[integer]>/path
function(number) {
  sum(c(number, 5L))
}

#* Do something
#* @param iris2:df* Use something like jsonlite::toJSON(list(iris2=iris), auto_unbox = TRUE)
#* @png
#* @post /smoore
function(iris2) {
  iris2$Species <- as.factor(iris2$Species)
  plot(iris)
}

#* Do some more
#* @param owners:[chr]
#* @param dogId:[int]
#* @get /whosagoodboy
function(owners, dogId) {
  #query string is still parsed as character
  dogId <- as.integer(dogId)
  list(sample(owners, 1), sample(dogId , 1))
}

#* Will that work? Test detection, examples and object type
#* @post /risky
function(owners = list(name = c("Bob", "Larry"),
                       cars = c("Corvette", "Tesla")),
         dogId = 1L:2L,
         moredata = women) {
  list(owners_equal = all.equal(owners, list(name = c("Bob", "Larry"), cars = c("Corvette", "Tesla"))),
       dogId_equal = all.equal(as.integer(dogId), 1L:2L),
       moredata_equal = all.equal(moredata, women))
}

#* @param pdf:file
#* @post /upload
function(pdf) {
  f <- tempfile()
  on.exit(unlink(f), add = TRUE)
  writeBin(as.raw(pdf), f, useBytes = TRUE)
  finfo <- file.info(f)
  finfo$mode <- as.character(finfo$mode)
  finfo$real_filename <- attr(pdf, "filename")
  finfo
}

Notes:
object parameters would not be supported in path parameters.
Default upload location set to getOption("plumber.upload.dir", "."). Use multipart parse from mime package. (copied with very slight change to buffer size, filename decoding and file args response)
openapi field example for object feeded from plumberExpression arguments.

Fixes:

• Swagger definition absent when defining a parameter in path that is not on a @param line.
• Prevent example from being removed by swagger remove null
• Swagger parameters now show in the order they were defined

Breaking:
req$postBody is raw since it can contains file and other breaking havoc on string elements.

Related:
#516, #512, #469, #509, #417, #523, #502, #463, #320

At this point I would need help to do more testing across different platforms.

Thank you, comments are welcome.

PR task list:

• Update NEWS
• Add tests
• Update documentation with devtools::document()
  See other PR for documentation, will update there if this one pass.

[CLAassistant]

All committers have signed the CLA.

[meztez]

Performance testing on a puny old laptop from the ages with library loadtest, simple enough.
TLDR : it is faster.

library(plumber)
tmpf <- tempfile()
writeLines('
#* Echo back the input
#* @param msg:character The message to echo
#* @post /test8
function(msg = list(part1 = "a", part1 = "b", part1 = "c")) {
  do.call(paste, msg)
}
', tmpf)
pr <- plumb(tmpf)
pr$run(port = 8477, swagger = FALSE)

plumber_0.4.6.pdf
plumber_0.4.7.9000.pdf
plumber_PR532.pdf

[schloerke]

Good PR to code review tomorrow

[cpsievert]

Looking pretty good to me, pending the suggestions. @schloerke is to:

• Make sure that the unit (or other) tests cover the important code paths.
• Look into and possibly resolve Swaggerfile support for array parameters, object, files #532 (comment)
• In a different PR, leverage webutils::parse_query() inside parseQS()

[schloerke]

I'm off for the weekend. I'll look into it on Monday. Enjoy your weekend!

[JosiahParry]

Super excited about multipart file upload! Is this documented? I cannot find it in rplumber.io.

[meztez]

No documented yet on rplumber.io. I believe the goal is to use pkgdown to build plumber documentation. There is another branch in development for that.

I'm super pump too.

[ThHarbig]

I tried it, works great! Is there a way to upload an arbitrary number of files at the same time?

[meztez]

@ThHarbig It would probably work via curl, I would mostly have to adapt the openapi spec generation. Please open a different issue so I can reference it in a PR.

[chrischung28]

This is a great addition! Thank you very much. Just wondering if it would be possible to get an example of the file upload feature? I'm having some trouble getting it to work with the example above when i try to upload some pdfs.

[meztez]

In the implementation that made it to master, you get a vector of raw bytes to play with. Here is a working example with the current master.

#* @apiTitle Upload test

#* @param pdf:file
#* @post /upload
function(pdf) {
  f <- tempfile()
  on.exit(unlink(f), add = TRUE)
  writeBin(as.raw(pdf), f, useBytes = TRUE)
  finfo <- file.info(f)
  finfo$mode <- as.character(finfo$mode)
  finfo$real_filename <- attr(pdf, "filename")
  finfo
}

[chrischung28]

@meztez Thank you very much! This works great.

[s-fleck]

@meztez sorry to bug you, but could you provide an example how to use this with httr::POST()? It is not clear to me where the "filename" attributes comes from. Is it possible with this approach to have an endpoint where I can upload with curl -F and httr::POST() without jumping through too many hoops and have the filename available in plumber?

[meztez]

yeah, the filename comes from the multipart form-data.

plumber::options_plumber(port = 8004)

#* @param a_file:file
#* @post /upload
function(a_file) {
  # Get a named list. The names correspond
  # to the files names. The content of the list
  # will depend on the content-type set
  # for each files.
  return(name_of_file = list(names(a_file), file_content_length = nchar(a_file[[1]])))
}

with httr::POST

f <- tempfile("stempy")
writeLines(stringi::stri_rand_strings(1, 100), f)
res <- httr::POST("http://127.0.0.1:8004/upload", 
                  body = list(a_file = httr::upload_file(f)),
                  encode = "multipart")
httr::content(res)

Should be similar with curl, files is specific to my system

curl -F a_file=@bobettebo.txt http://127.0.0.1:8004/upload

[s-fleck]

thanks! i didn't know about httr::upload_file(), i was always sending raw binary data