Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DocService to accept non JSON requests #5218

Closed
ikhoon opened this issue Oct 2, 2023 Discussed in #5214 · 1 comment · Fixed by #5381
Closed

DocService to accept non JSON requests #5218

ikhoon opened this issue Oct 2, 2023 Discussed in #5214 · 1 comment · Fixed by #5381
Labels
Milestone

Comments

@ikhoon
Copy link
Contributor

ikhoon commented Oct 2, 2023

Discussed in #5214

Originally posted by vthacker September 30, 2023
Hello Armeria folks,

I'm using annotated doc service @Post("/_bulk") that can also parse NDJSON strings. This is the only dataformat that Elasticsearch/Opensearch accepts ( https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html )

The service works as expected, however I am running into an issue with the DocService

I am passing this as my HTTP Headers within the doc service

{
  "content-type": "application/x-ndjson"
}

In the request body if I input NDJSON data, for example

{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }

The doc serice parser is complaining since it's expecting data to be JSON

Error: Failed to parse a JSON object in the request body:
SyntaxError: Unexpected non-whitespace character after JSON at position 49 (line 2 column 1)

So my first questionn is there a way for me to disable the JSON parsing of the request body?

My second quesion is the DocService dseems to overwrite the content-type header. For example if I change my request body to be a valid JSON

{ "index" : { "_index" : "test", "_id" : "1" } }

and then "Copy as a curl command" I see this

curl -XPOST -H 'content-type: application/json; charset=utf-8' https:/service.com'/_bulk' -d '{ "index" : { "_index" : "test", "_id" : "1" } }

If I change the HTTP Header to be

{
  "Content-Type": "application/x-ndjson"
}

then I see the request being formed as

curl -XPOST -H 'Content-Type: application/x-ndjson' -H 'content-type: application/json; charset=utf-8' https://service.com'/_bulk' -d '{ "index" : { "_index" : "test", "_id" : "1" } }
@ikhoon ikhoon added the defect label Oct 2, 2023
@ikhoon ikhoon added this to the 1.26.0 milestone Oct 2, 2023
@heowc
Copy link
Contributor

heowc commented Oct 5, 2023

I've created similar issues in the past.
See #2198

@minwoox minwoox modified the milestones: 1.26.0, 1.27.0 Oct 11, 2023
@ikhoon ikhoon modified the milestones: 1.27.0, 1.28.0 Jan 16, 2024
ikhoon pushed a commit that referenced this issue Apr 8, 2024
Motivation:

1) Trying to use json pretty even though it's a string that can't use
json pretty, like json-lines.
2) When copying as a curl command, it uses an array of characters to
insert duplicate semantically identical headers.

Modifications:

1) Trying to use json pretty even though it's a string that can't use
json pretty, like json-lines.
    - Fix some mime types to ignore pretty
2) When copying as a curl command, it uses an array of characters to
insert duplicate semantically identical headers.
    - Change to `Headers` instead of string array

Result:

- Closes #5218

---------

Co-authored-by: jrhee17 <guins_j@guins.org>
Co-authored-by: minwoox <songmw725@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants