Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restructure body parsers to use content type. Add many parsers #584

Merged
merged 79 commits into from Jul 31, 2020
Merged

Restructure body parsers to use content type. Add many parsers #584

merged 79 commits into from Jul 31, 2020

Conversation

schloerke
Copy link
Collaborator

@schloerke schloerke commented Jul 3, 2020 •
edited

Fixes #583
Fixes #586

Reference: https://community.rstudio.com/t/uploading-tsv-files-to-plumber-api/71705/3

Changes:

  • Restructure parser list to use the pattern as the key and parser as the value.
    • Allows the same parser to exist under different content type values.
    • Provide duplicate parsers for commonly seen content types
  • Allow for arguments to be supplied to parsers
  • Add tsv parser
  • Add csv parser
  • If a content-type of application/octet is provided, inspect the file name to see what it's mime-type is and use that as the content type value.
    • Allows for RDS files to be shipped using application/octet, but processed using application/rds which maps to parser_rds()
  • Use other parsers to reduce copy/pasta
  • Move the JSON first inspection after the content type has a chance to be used. Avoids possible situations where the plain text starts with a {, but is truly plain text and not JSON.
  • Rename (new for next release) functions to snake case. (add_parser(), parse_body(), ...)

PR task list:

  • [NA] Update NEWS
  • Add tests
  • Update documentation with devtools::document()

@meztez Can you take a look at this PR.

meztez and others added 12 commits Jul 3, 2020
@meztez
add check before guessing content-type
2c05be1
@meztez
PR suggestion, gitattributes
5c78836
@meztez
remove to readd
680bd8e
@meztez
readd binary
161164e
@schloerke
Add back ... args to safeFromJSON()
2954c74
@schloerke
Adjust parsers to allow for arguments. Use direct matches only
4233155 
* Restructure parser list to use the pattern as the key and parser as the value. Allows the same parser to exist under different content type values.
* Provide duplicate parsers for commonly seen content types
* add tsv parser
* add csv parser
* Allow for arguments to be supplied to parsers
* use other parsers to reduce copy/pasta
@schloerke
Merge branch 'respect_ctype' of https://github.com/meztez/plumber int…
f392e60 
…o post_body_content_type
@schloerke
use stringi methods
dee07fa
@schloerke
Add comments as to why the content type is being altered
0f27683
@schloerke
Merged name change
30ca5ad
@schloerke
Add a default file name in case one isn't provided
ddaa3c5
@schloerke
Add tsv and csv tests. Rename test file.
11d459c
@schloerke schloerke added this to the v0.5.0 - Next CRAN release milestone Jul 3, 2020
@schloerke schloerke self-assigned this Jul 3, 2020
meztez
meztez reviewed Jul 3, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
@meztez
Copy link
Collaborator

meztez commented Jul 3, 2020

This is masterfully crafted code. Beautiful. Loves the generalization where parser reuse other parser. Last time I've seen that great of a collab was Snoop & Dre.

Seriously. Great stuff.

@schloerke
Copy link
Collaborator Author

schloerke commented Jul 3, 2020

😆😆 Made my day!

schloerke
schloerke reviewed Jul 3, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
schloerke
schloerke reviewed Jul 3, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
@schloerke schloerke requested a review from cpsievert Jul 6, 2020
schloerke
schloerke reviewed Jul 7, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
schloerke
schloerke reviewed Jul 7, 2020
View changes
R/json.R Outdated Show resolved Hide resolved
@schloerke schloerke mentioned this pull request Jul 7, 2020
Closed
meztez
meztez reviewed Jul 28, 2020
View changes
R/plumb-block.R Outdated Show resolved Hide resolved
@schloerke schloerke mentioned this pull request Jul 28, 2020
Closed
2 tasks
schloerke and others added 2 commits Jul 28, 2020
@schloerke
Merge branch 'master' into post_body_content_type
c141fa5 
* master:
  Replace plus sign with space character when parsing a query string (#618)
@cpsievert @meztez
Eval in proper env (related to #620)
18655f7 
Co-authored-by: Bruno Tremblay <bruno.tremblay@lacapitale.com>
cpsievert
cpsievert reviewed Jul 29, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
cpsievert
cpsievert reviewed Jul 29, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
cpsievert
cpsievert reviewed Jul 29, 2020
View changes
R/parse-body.R Show resolved Hide resolved
cpsievert
cpsievert reviewed Jul 29, 2020
View changes
R/plumber-step.R Outdated Show resolved Hide resolved
cpsievert
cpsievert reviewed Jul 29, 2020
View changes
R/parse-body.R Outdated Show resolved Hide resolved
cpsievert
cpsievert approved these changes Jul 29, 2020
View changes
Copy link
Contributor

@cpsievert cpsievert left a comment

LGTM pending last few comments

schloerke and others added 13 commits Jul 31, 2020
@schloerke @cpsievert
Forgot to change the docs
d95325d 
Co-authored-by: Carson Sievert <cpsievert1@gmail.com>
@schloerke
Move parser_read_file down in file
71b6aa8
@schloerke
Allow for readr to take original raw value directly
1710168
@schloerke
rename test file to have it be executed last. (it's slow)
d4b6339
@schloerke
merge master
8481248
@schloerke
scope readr parser args properly
c6c3a09
@schloerke
safe guard against temp files possibly being deleted by user
538e75a
@schloerke
add comment about how readRDS does not work with rawConnection
0761880
@schloerke @cpsievert
Forgot to add function in the pseudo docs
16e01a9 
Co-authored-by: Carson Sievert <cpsievert1@gmail.com>
@schloerke
Merge branch 'post_body_content_type' of https://github.com/rstudio/p…
7989213 
…lumber into post_body_content_type
@schloerke
Do not export make_parsers(). Copy over appropriate docs
6959dd7
@schloerke
Do not display docs for make_parser()
4d8591f
@schloerke
update news entry
7be4fec
@schloerke schloerke changed the title Restructure body parsers to use content type. Add TSV and CSV parsers Restructure body parsers to use content type. Add many parsers Jul 31, 2020
@schloerke schloerke merged commit 0254d8b into master Jul 31, 2020
15 of 16 checks passed
@schloerke schloerke deleted the post_body_content_type branch Jul 31, 2020
@meztez
Copy link
Collaborator

meztez commented Jul 31, 2020

Pop a can of Dr Pepper, you did it!!!!

@schloerke
Copy link
Collaborator Author

schloerke commented Jul 31, 2020

😌 Hope to get #598 this afternoon so we can test the two of them together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@meztez meztez

@cpsievert cpsievert

Assignees

@schloerke schloerke

Labels
None yet
Projects
None yet
Milestone
v1.0.0
Development

Successfully merging this pull request may close these issues.

Make body parsers opt in
3 participants
@schloerke @meztez @cpsievert