POST requests from another server

[eralpdogu]

Is there a way of getting data and reading it in Shiny using POST request? My data will flow from another software and I need to import it to analyze. Already tried the solution here https://stackoverflow.com/questions/25297489/accept-http-request-in-r-shiny-application but I need a more solid one.

[daattali]

I'm assuming you mean you want to expose POST endpoints that will be available FROM this shiny server, TO other servers (that's what the title says, but the description is less clear on that, but I'll go with that assumption). You might like this video from RStudio Conf a few months ago, where @jcheng5 announced that this feature (called APIs) will be coming in the future as built into shiny. Start watching the video at minute 15:00.

As of right now I don't think it's released yet, but you can probably experiment with the developmental version , I believe this feature is in one of the branches of this repo. I hope rstudio doesn't mind this info being publicized...

[jcheng5]

No problem @daattali :) We are just waiting to release it until we have server-side support for authentication, which involves some other groups here at RStudio.

[eralpdogu]

Thanks a lot for the responses @daattali @jcheng5! The video is very helpful. A group of people using a certain software wants to send their data and feed my shiny without browsing and uploading the data. In design, it will be a button in this software which will open my app and run it. I wasn't able to respond their POST requests. So, I think we need wait for the new release...

[bborgesr]

@jcheng5 I don't think our API branch actually covers this use case, right? What Shiny APIs allow for is to serve up artifacts from your app to another program through an http GET request. But it doesn't allow Shiny to receive http requests from the outside in any new way (right?) It seems that the "state of the art" there is still at the level of the second answer in the stackoverflow post linked above. You should take a quick read @jcheng5 (and @wch) -- I myself had no idea that was possible. Maybe we should expose that whole mechanism (handling GET and POST requests from other apps that flow into Shiny) a bit better? It seems that it would be a nice feature to have polished up by the time APIs are out. They're basically the mirror of one another...

I'm bringing this up mostly because, if it's true, we really don't want @eralpdogu waiting for the API release, only to find that it doesn't help him at all...

[bborgesr]

@daattali Just FYI, in addition to Joe's video, around that time, I also posted something on shiny-discuss that guides you through using the right branch from Github and the basic functionality of the API feature. If you're directing people to the video, this may be something good to pair it with, since there's a copy-pasteable example :)

[daattali]

Good to know about that example code.

My understanding is that he wants a program to send a POST request to a shiny app, and if I remember correctly then Joe did mention in his talk that both GET and POST are supported. Perhaps I'm misunderstanding his requirement though.

[labkey-jeckels]

Hi everyone,

Thanks for the replies! I'm working with Eralp to understand what options are possible.

Yes, the goal here is to integrate with a separate application that can launch a web browser and automatically POST data (the real payload is effectively an HTML <textarea> that contains a a CSV) to a configurable URL. Eralp has a Shiny app that currently prompts the user for a CSV file (via shinyUI() and fileInput()) and receives it via shinyServer() and observeEvent().

The hope was that we automatically launch the Shiny app and save the user from having to save the CSV and upload the file manually. From a quick review of the video you kindly linked, that sounds like it may work for this scenario, and that there wouldn't be a way to do it without those features.

Thanks,
Josh

[jcheng5]

@bborgesr That stackoverflow answer won't work for this case. registerDataObj can't be relied on because it returns URLs that are tied to a specific session (i.e. there's no stable URL that an outside server/client can refer to).

And no, there's no specific reason why the API feature would be limited to GET, though I think generally GET calls will get more mileage out of a Shiny app's existing reactive expressions, so I'm emphasizing that use case.

[eralpdogu]

@bborgesr we have already implemented this approach but as @jcheng5 mentioned we had problems with URLs. @daattali, @bborgesr and @jcheng5 can you have a look at @labkey-jeckels comments above? this summarizes what we need. Please let us know if you have any recommendations.

[assaron]

Hi everyone, I have a similar issue. As a workaround I've put a python script that accepts POST requests, put the files in a folder accessible to shiny app and generates a dataset key, which then can be passed via GET to a shiny app.

from os import curdir
from os.path import join as pjoin
import sys
from tempfile import NamedTemporaryFile

from http.server import BaseHTTPRequestHandler, HTTPServer
import random
import string

global base_dir

N=16

class StoreHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = self.headers['content-length']
        data = self.rfile.read(int(length))

        key=''.join(random.SystemRandom().choice(string.ascii_lowercase + string.digits) for _ in range(N))

        store_path = pjoin(base_dir, key)

        with open(store_path, 'wb') as fh:
                fh.write(data)

        self.send_response(200)
        self.send_header("Content-type", "text/html")
        self.end_headers()

        self.wfile.write(key.encode("ascii"))

if __name__ == "__main__":
    if len(sys.argv) == 1:
        print("upload.py <port> <dir>?")
        sys.exit(1)

    port = int(sys.argv[1])
    if len(sys.argv) > 2:
        base_dir = sys.argv[2]
    else:
        base_dir = curdir
    print(base_dir)
    server = HTTPServer(('127.0.0.1', port), StoreHandler)
    server.serve_forever()

[javerrecchia]

Hi everyone,
Thanks for the useful information in the thread and the wonderful work behind it! I am looking for updates on this very topic, because I have a similar issue: wanting to feed and launch my shiny app from outside the app, through the POST method (i.e. with data embedded in a JSON that are input values for my shiny app).

For instance (this is a voluntarily dumb example): say I have a HTML form asking for a name, and I would like for it to redirect to my shinyapp when the action button is clicked, and for the shinyapp to "read" the name and display it. But even though this is very easy with the GET method (by accessing session$clientData$url_search), it seems that through POST I cannot even access the app, as if it voluntarily prevents POST requests to launch.

Is there, by any chance, new info on the API feature release? I am having a hard time finding info on that. I am especially interested on what @jcheng5 mentions at the end of the video, on slide 24 (Recap): "URL parameters or JSON request bodies become input values".
Is it implemented on the branch that @bborgesr mentions and/or do you have any documentation or examples? That would be of great help!

Thank you!

[javerrecchia]

No one? Just a small indication on the basic function(s) to call?

[chrismclarke]

@unplusun Could you convert the json to a set of query parameters which you pass via iframe url parameters? I don't think you'd be able to make it responsive (assume query strings parsed on first render?), but should be fine for simple cases where you just want to feed some initial data in. Here's the shiny doc reference for parseQueryString. https://shiny.rstudio.com/reference/shiny/0.14/parseQueryString.html

[javerrecchia]

@chrismclarke Thank you for your answer!
I had thought about this approach, but there is a limit on the length of the URL, so I am afraid it might not fully resolve my problem.
The closest way if to have a script that writes the JSON on the disk and then pass the new file's name via the url to my shiny app. It was, I think, the solution provided by @assaron.

I have seen on the api branch, developped by @jcheng5, the existence of the parseJSONBody function, which seems to cover what I am looking for (I might be wrong) :

parseJSONBody <- function(req) {
  if (identical(req[["REQUEST_METHOD"]], "POST")) {
    if (isTRUE(grepl(perl=TRUE, "^(text|application)/json(;\\s*charset\\s*=\\s*utf-8)?$", req[["HTTP_CONTENT_TYPE"]]))) {
      tmp <- file("", "w+b")
      on.exit(close(tmp))

      input_file <- req[["rook.input"]]
      while (TRUE) {
        chunk <- input_file$read(8192L)
        if (length(chunk) == 0)
          break
        writeBin(chunk, tmp)
      }

      return(jsonlite::fromJSON(tmp))
    }

    if (is.null(req[["HTTP_CONTENT_TYPE"]])) {
      if (!is.null(req[["rook.input"]]) && length(req[["rook.input"]]$read(1L)) > 0) {
        stop("Invalid POST request (body provided without content type)")
      }
      return()
    }

    stop("Invalid POST request (content type not supported)")
  }
}

It seems to be requiring the use of the apiHandler function, defined also on the api branch.
My problem is that when I try to call my shinyapp via a POST request, nothing happens. Thus my impossibility to dig further into these functions!

[jcheng5]

Is there, by any chance, new info on the API feature release?

It got bumped in 2017 by the async features, which should be shipping soon. It also felt like lower priority because 1) plumber gaining popularity and 2) not very many people had expressed interest in the API stuff after my talk last year. But I received feedback from several people during rstudio::conf 2018 that they were waiting for it. So we'll try to get it back on the board after the async release, and I'll keep the use cases in this thread in mind.

[emcrisan]

Adding a vote for this feature (session$api$data) from ticket # 33095. Thanks!

[aljrico]

Hi! Do we have any update on this? Is there a recommended way of accepting POST requests in a shiny server?