-
-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] enhance awscurl support create/send multipart POST REST request (from file) #194
Labels
Comments
Seems like last curl command works without multipart ? Let’s have a look at what it decides to send: please rerun it with -v parameter and post more details. On Mar 8, 2024, at 5:26 AM, llasherme ***@***.***> wrote:
Versions
OS :
cURL => via Windows 10
awscurl => via gitlab runner (Docker)
OpenSearch 2.11
What's my problem ?
First of all, this closed issue : opensearch-project/opensearch-cli#76
Seems to be close to my problem, but I'm not using opensearch-cli curl. Only awscurl
I try to migrate dashboards objects from one opensearch cluster to another one.
To do that I'm using api provided by opensearch : /_dashboards/api/saved_objects/_import?overwrite=true
But I get errors when I'm trying to post my objects.
How that's done ?
I'm exporting the objects via the opensearch UI (From my first cluster). Dashboards Management > Saved objects > Export (when my dashboard is selected). And I get my objects in a .ndjson file.
After that i'm posting my .ndjson file with awscurl :
Without Content Type header
awscurl --service es --region ${region} -X POST ${domain_url}/_dashboards/api/saved_objects/_import?overwrite=true \
-H 'osd-xsrf: true' \
-d '${dashboard_content}'
{
"statusCode":415,
"error":"Unsupported Media Type",
"message":"Unsupported Media Type"
}
Without Boundary
awscurl --service es --region ${region} -X POST ${domain_url}/_dashboards/api/saved_objects/_import?overwrite=true \
-H 'Content-Type: multipart/form-data' \
-H 'osd-xsrf: true' \
-d '${dashboard_content}'
{
"statusCode":400,
"error":"Bad Request",
"message":"Invalid content-type header: multipart missing boundary"
}
With a random boundary
awscurl --service es --region ${region} -X POST ${domain_url}/_dashboards/api/saved_objects/_import?overwrite=true \
-H 'Content-Type: multipart/form-data; boundary=---------BOUNDARY;' \
-H 'osd-xsrf: true' \
-d '${dashboard_content}'
image.png (view on web)
image.png (view on web)
By only using curl, and don't put Content Type header, it works fine.
curl -X POST ${domain_url}/_dashboards/api/saved_objects/_import?overwrite=true \
-H "osd-xsrf: true"
-F ***@***.***/test_dashboard.ndjson"
{
"successCount":6,
"success":true,
"successResults":[
{
"type":"index-pattern",
"id":"610267e0-7f21-11ee-80e6-370892a64263",
"meta":{
"title":"documents-metadata-v2",
"icon":"indexPatternApp"
},
"overwrite":true
},
{
"type":"visualization",
"id":"da95cc90-7f22-11ee-9e31-e14f5ec8461b",
"meta":{
"title":"count by cat_doc",
"icon":"visualizeApp"
},
"overwrite":true
},
{
"type":"index-pattern",
"id":"d3d7af60-4c81-11e8-b3d7-01146121b73d",
"meta":{
"title":"opensearch_dashboards_sample_data_flights",
"icon":"indexPatternApp"
},
"overwrite":true
},
{
"type":"search",
"id":"571aaf70-4c88-11e8-b3d7-01146121b73d",
"meta":{
"title":"[Flights] Flight Log",
"icon":"discoverApp"
},
"overwrite":true
},
{
"type":"visualization",
"id":"94003870-dbb5-11ee-81ba-b7ae96977aa8",
"meta":{
"title":"test_object_dashboard",
"icon":"visualizeApp"
},
"overwrite":true
},
{
"type":"dashboard",
"id":"2d7aceb0-7f23-11ee-9e31-e14f5ec8461b",
"meta":{
"title":"test_dashboard",
"icon":"dashboardApp"
},
"overwrite":true
}
]
}
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
Yes, cURL doesn't need to explicit the Content Type. curl -v -X POST ${domain_url}/_dashboards/api/saved_objects/_import?overwrite=true \
-H "osd-xsrf: true" \
-F "file=@data/test_dashboard.ndjson" And the result :
|
So, I think -F has extra processing to send a request with "data" *and*
"files". the second part creates the multi-part sections.
So we'd need to create that option in awscurl
…On Tue, Mar 12, 2024 at 4:53 AM llasherme ***@***.***> wrote:
Yes, cURL doesn't need to explicit the Content Type.
Here's the cURL with verbose option :
curl -v -X POST ${domain_url}/_dashboards/api/saved_objects/_import?overwrite=true \
-H "osd-xsrf: true" \
-F ***@***.***/test_dashboard.ndjson"
And the result :
Note: Unnecessary use of -X or --request, POST is already inferred.
* Uses proxy env variable no_proxy == '****'
* Trying [::1]:8080...
* Trying ${domain_ip}:8080...
* Connected to localhost (127.0.0.1) port 8080
> POST /_dashboards/api/saved_objects/_import?overwrite=true HTTP/1.1
> Host: ${domain_url}
> User-Agent: curl/8.4.0
> Accept: */*
> osd-xsrf: true
> Content-Length: 40384
> Content-Type: multipart/form-data; boundary=------------------------wOWPtrGc1TrXmDPpX5GfNt
>
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Cache-Control: private, no-cache, no-store, must-revalidate
< Content-Length: 947
< Content-Type: application/json; charset=utf-8
< Date: Tue, 12 Mar 2024 09:44:45 GMT
< Osd-Name: osd
< Set-Cookie: security_authentication_saml1=Fe26.2**838c01d554bcb2b7d4e5ead20dafc498fcdd74e1faf6b74c5d1348a2326b713c*D_5paBXbChAFcjTpZ00nLg*YUmGokrp_eV_EEv_2S0NZecOyi1Lwhrgas5VLH69YDC6nBJDau20hsaz8ipoTcUPxMS2AxQoGXqv5bUMo8sY8ulz783TWCck7iibWpBdYn_FvqXpBoCu9C23MAD-v6CDe7CJ5ch2KczWBZhnI-jc_z53Lz-Dt62h2PQ1OAAq7-He5QR7Jx-lg87XpaWpCL2A0-uAvfASBUoZ7ui2IvPHuLmw5myZ5acmFZsxpgg9QLrRKlBlUvpNRz0-rG6EZY2uI1UoVRnsaYFLUkRKe3YOZYSCsvLWxGDi6XBEUfdLH9bl3WSV1u0eDdAEvt0qViIcp8Njt0_iT1iyQiHR6ju-Xg**8def9b476d52a739f02c24f8ed8783b8585d01177dcd04e624d2ec17b0a17c61*nVfuzR87kXvFQPaesZZZfY1aPWiT87yLITvEMUJOj5s; Secure; HttpOnly; Path=/_dashboards
< Set-Cookie: security_authentication_saml2=; Max-Age=0; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Secure; HttpOnly; Path=/_dashboards
< Set-Cookie: security_authentication_saml3=; Max-Age=0; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Secure; HttpOnly; Path=/_dashboards
< Set-Cookie: security_authentication=Fe26.2**59193956774e2ff6b56087734d72a99abd64c27811b759e7c10e85f995ae4792*CFtdPl74zukFptAGTWbAXw*bHObA_QMNUiaQI8V471pQhS_Z2GRkmQuy4-G6e5ycbyBSGZ_8W6ThTG7f3SlTgtQLD5AmqSa6mGKeZ5j6JJZoY0zTX2T5Zimkr8JWEljMBIlFOK_G-SSNmEtZPVuKHUfbKOVH-4_6WlW6DVVQO7rGYe9dnm615ssNvTuzFQNJ8BTZ5HVXutfXhMrOzipcs1nh4A2K6PaZnbT-RNpS823WT_JVmdfNcrHXj2xWfjYbHQLC2gv5FTcOHAMd0yPCDS0**9e47d862c46786b7011a75ddbde639601e554e9a30ca09366ede38184d642392*vsovYjoNWeD0yiX3QUmj3ryV1V0IJTuejNc3rBMOLP8; Secure; HttpOnly; Path=/_dashboards
< Strict-Transport-Security: max-age=31536000
< X-Amzn-Requestid: a56e2412-d6f8-4119-84ac-9eead1de5731
<
{"successCount":6,"success":true,"successResults":[{"type":"index-pattern","id":"610267e0-7f21-11ee-80e6-370892a64263","meta":{"title":"documents-metadata-v2","icon":"indexPatternApp"},"overwrite":true},{"type":"visualization","id":"da95cc90-7f22-11ee-9e31-e14f5ec8461b","meta":{"title":"count by cat_doc","icon":"visualizeApp"},"overwrite":true},{"type":"index-pattern","id":"d3d7af60-4c81-11e8-b3d7-01146121b73d","meta":{"title":"opensearch_dashboards_sample_data_flights","icon":"indexPatternApp"},"overwrite":true},{"type":"search","id":"571aaf70-4c88-11e8-b3d7-01146121b73d","meta":{"title":"[Flights] Flight Log","icon":"discoverApp"},"overwrite":true},{"type":"visualization","id":"94003870-dbb5-11ee-81ba-b7ae96977aa8","meta":{"title":"test_object_dashboard","icon":"visualizeApp"},"overwrite":true},{"type":"dashboard","id":"2d7aceb0-7f23-11ee-9e31-e14f5ec8461b","meta":{"title":"test_dashboard","icon":"dashboardApp"},"overwrite":true}]}
* Connection #0 to host ${domain}left intact
```^
As we can see, cURL putting itself the Content Type to multipart/form-data and setting the boundary to my data.
—
Reply to this email directly, view it on GitHub
<#194 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADUYXTNJPP66DUNWI5GRJDYX3GBXAVCNFSM6AAAAABEMU6HUGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJRGIYTQNBSG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
okigan
added a commit
that referenced
this issue
Mar 14, 2024
@llasherme try out this branch, his are the relevant steps:
once that works try with your ES cluster |
@llasherme interesting, try latest please (de473d8) |
@llasherme - would love some context, did you run into any issues or something else came up?On Mar 18, 2024, at 3:44 AM, llasherme ***@***.***> wrote:
Closed #194 as not planned.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were assigned.Message ID: ***@***.***>
|
This issue was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Versions
What's my problem ?
First of all, this closed issue : opensearch-project/opensearch-cli#76
Seems to be close to my problem, but I'm not using opensearch-cli curl. Only awscurl
I try to migrate dashboards objects from one opensearch cluster to another one.
To do that I'm using api provided by opensearch :
/_dashboards/api/saved_objects/_import?overwrite=true
But I get errors when I'm trying to post my objects.
How that's done ?
I'm exporting the objects via the opensearch UI (From my first cluster). Dashboards Management > Saved objects > Export (when my dashboard is selected). And I get my objects in a .ndjson file.
After that i'm posting my .ndjson file with awscurl :
Without Content Type header
Without Boundary
With a random boundary
By only using curl, and don't put Content Type header, it works fine.
The text was updated successfully, but these errors were encountered: