OdataV4 in sapui5

[siasty]

Hi, I am opening again and asking for help!

I wrote the controller test and it does not agree with what is written.
as you can see boundry return \r\n

new link for odataV4 http://51.68.141.73:5000/odata/
and error in fiori app

Best regards Maciej
..................................................................................................................................................
Hi Maciej (@siasty ),

the issue seems to be with the response. Let me start with https://www.ietf.org/rfc/rfc2046.txt:

5.1.1. Common Syntax
...
The boundary delimiter line is then defined as a line
consisting entirely of two hyphen characters ("-", decimal value 45)
followed by the boundary parameter value from the Content-Type header
field, optional linear whitespace, and a terminating CRLF.

When checking in _Batch, function _deserializeBatchResponse I observe that the first boundary is not detected. This is because the CR (ASCII 13, \r) is missing. You can check yourself by setting a breakpoint in the mentioned function. In my cases sResponseBody.charCodeAt(52) shows the LF (ASCII 10, \n). sResponseBody.charCodeAt(51) does not return 13 as sResponseBody[51] is the last character of the boundary, e.g. 6.

Could you please check why your service is not returning the CR as specified in RFC2046?

Best regards
Mathias.

Originally posted by @uhlmannm in #2830 (comment)

[Shtilianova]

Hello @siasty ,
Thank you for sharing this finding. I've created an internal incident 2080375210. The status of the issue will be updated here in GitHub.
Regards, Diana

[flovogt]

Hi @siasty ,
could you provide us the link to the sample application again?
So, its much more easier to debug your issue.
Best Regards,
Florian

[siasty]

OK, I just need to prepare new app, I only checked on the webide.

[alexanderriess]

Hi,
your issue has reached the ui5 core area,
are there some updates providing a sample app?
Best regards
Alex

[siasty]

Hi,
I was finally able to share the applications
http://51.68.141.73/

the problem occurs but I noticed that when I run the application locally there is no error

[Silkinator]

Dear sender,

I can not see anything on that app. I can neither reproduce nor identify your error.
Could you please investigate in make it reproducable?
Does the error still persist?

Regards
Silke

[siasty]

hi,
yes, the problem still exists and you can see it in the browser console as soon as we open the link.

when checking batch request we receive status code 200 but data not loaded to table

--batchresponse_9e53c3eb-655f-491d-b222-f575f25f6032
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
OData-Version: 4.0
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true

{"@odata.context":"http://51.68.141.73/odata/$metadata#Products(Id,Name)","value":[{"Id":1,"Name":"Cola"},{"Id":2,"Name":"Fanta"},{"Id":3,"Name":"Pepsi"},{"Id":4,"Name":"Sprajt"}]}
--batchresponse_9e53c3eb-655f-491d-b222-f575f25f6032--

[Silkinator]

Hi,

thanks I see it now. You mentioned that when you run locally there is no error. Currently you are using the UI5 version 1.81.0 (built at 2020/08/12 14:09h).

Is this also the version you use locally?

Regards
Silke

[siasty]

Yes exactly, I don't have this problem locally
I have the same problem when i use WebIDE but i dont see any error in odata controller ( always returns data )

[Silkinator]

Hm,

as @uhlmannm already said the batch response

--batchresponse_f3fab6fa-8b47-4bad-813c-59a9c74b8769
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
OData-Version: 4.0
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true

{"@odata.context":"http://51.68.141.73/odata/$metadata#Products(Id,Name)","value":[{"Id":1,"Name":"Cola"},{"Id":2,"Name":"Fanta"},{"Id":3,"Name":"Pepsi"},{"Id":4,"Name":"Sprajt"}]}
--batchresponse_f3fab6fa-8b47-4bad-813c-59a9c74b8769--

does not get correctly deserialized.

He also mentioned the problem
5.1.1. Common Syntax
...
The boundary delimiter line is then defined as a line
consisting entirely of two hyphen characters ("-", decimal value 45)
followed by the boundary parameter value from the Content-Type header
field, optional linear whitespace, and a terminating CRLF.

A bacht response should look like:
--4405850C4D39DA3A0534CA6422A3A29A0
Content-Type: application/http
Content-Length: 2120
content-transfer-encoding: binary

HTTP/1.1 200 OK
Content-Type: application/json;ieee754compatible=true;odata.metadata=minimal
Content-Length: 1677
odata-version: 4.0
cache-control: no-cache, no-store, must-revalidate
sap-messages: [{"code":"ZUI5_EPM_SAMPLE/001","message":"SAP UI5con learn.explore.connect","numericSeverity":1,"target":""},{"code":"ZUI5_EPM_SAMPLE/002","message":"System Maintenance on 20200901 at 12h until 15h UTC","numericSeverity":3,"target":""}]

{"@odata.context":"$metadata#SalesOrderList(CurrencyCode,GrossAmount,Note,SalesOrderID,SO_2_BP(BusinessPartnerID,CompanyName))","@odata.metadataEtag":"W/"20200825150822"","@odata.count":"5153","value":[...]}
--4405850C4D39DA3A0534CA6422A3A29A0--

So if I understand that correct the http://51.68.141.73/odata/ must vanish from the batch response.
@uhlmannm suggested to get in touch with the backend.
What about that so far. Can we compare the batch responses?

Regards
Silke

[siasty]

I don't really understand why response does not get correctly deserialized.
difference between working

--batchresponse_0cb702ce-a606-4ac9-94ee-e9bd2d164272
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true
OData-Version: 4.0

{"@odata.context":"http://localhost:5000/odata/$metadata#Products(Id,Name)","value":[{"Id":1,"Name":"Cola"},{"Id":2,"Name":"Fanta"},{"Id":3,"Name":"Pepsi"},{"Id":4,"Name":"Sprajt"}]}
--batchresponse_0cb702ce-a606-4ac9-94ee-e9bd2d164272--

and not working

--batchresponse_f3fab6fa-8b47-4bad-813c-59a9c74b8769
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
OData-Version: 4.0
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true

{"@odata.context":"http://51.68.141.73/odata/$metadata#Products(Id,Name)","value":[{"Id":1,"Name":"Cola"},{"Id":2,"Name":"Fanta"},{"Id":3,"Name":"Pepsi"},{"Id":4,"Name":"Sprajt"}]}
--batchresponse_f3fab6fa-8b47-4bad-813c-59a9c74b8769--

is only in the address in response. That shouldn't be a problem

[Silkinator]

Hi,
the server response looks quite similar.
May I ask two other things:

1. Use localhost and the URL you gave me the same server?
2. Are both UI5 versions the same?

Regards
Silke

[siasty]

Hi,

runtime lib and Ui5 is the same in both cases.
the difference is when run localy i use windows (serwer is on linux) and change in mainfest odata destination to localhost

M.

[Silkinator]

Ok just to ask again if I understand:
Server in both cases is not the same but UI5 version.

The server of local host is a WINDOWS but the server for your URL here is a LINUX.
Is that correect?

Regards
Silke

[siasty]

Yes localhost is a windows
server where is the app is LINUX

[Silkinator]

Hi,

ok I checked again with the team and there seems also to be another similar issue addressing that problem.
I can just verify the the server sends no \r and this is not conform to the specification.
I aggree with @uhlmannm -
This is no UI5 problem but a problem of the server. You need to get in touch with the server.
I have to close.

Regards
Silke

[siasty]

ok, if it's not a problem, ui do tell me what to look for?
the difference between the inquiries is only in the address
in the server logs I do not see any query being blocked

manual checking also shows request return CRLF

[Silkinator]

Hi,

I can not verify that. What comes during the javascript code directly from the server has no \r carriage return.,
How did you test this here? I did a try with the Advanced REST client and could not reproduce your screenshot.

Regards
Silke

[siasty]

Hi, to simulate batch request i copy reqest from app.
for example in screenshot, reqest has body:

--batch_id-1597058164852-14
Content-Type:application/http
Content-Transfer-Encoding:binary

GET Products?$orderby=Name&$select=Id,Name&$skip=0&$top=20 HTTP/1.1
Accept:application/json;odata.metadata=minimal;IEEE754Compatible=true
Accept-Language:pl
Content-Type:application/json;charset=UTF-8;IEEE754Compatible=true

--batch_id-1597058164852-14--

i also write unit test in .net to check first boundry in respond (result in first coments)

[Silkinator]

Hey,

but all copying you do here goes through autoformat of editors.
I looked in the original output the java script coding gets. So what ever you do here gives no proof on how the original message looks like.
Get in touch with the server guys.

Regards
Silke

[codeworrior]

@siasty: can you please open the two app variants (local vs. public deployment) and for both of them record and save a HAR file in the Chrome developer tools? When you open those HAR files in a text editor, you can search for the .../odata/$batch request and check the "response" object.

When I do the same for the public deployment (http://51.68.141.73/), I get the following base64 (1) encoded string:

          "content": {
            "size": 468,
            "mimeType": "multipart/mixed",
            "compression": -12,
            "text": 
"LS1iYXRjaHJlc3BvbnNlXzZiNzc1Mjc3LWI3MTUtNDJlOS04ZTE3LTllODBiZTA2ZDRmOApDb250ZW5\
0LVR5cGU6IGFwcGxpY2F0aW9uL2h0dHAKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogYmluYXJ5Cg\
pIVFRQLzEuMSAyMDAgT0sKT0RhdGEtVmVyc2lvbjogNC4wCkNvbnRlbnQtVHlwZTogYXBwbGljYXRpb\
24vanNvbjsgb2RhdGEubWV0YWRhdGE9bWluaW1hbDsgb2RhdGEuc3RyZWFtaW5nPXRydWUKCnsiQG9k\
YXRhLmNvbnRleHQiOiJodHRwOi8vNTEuNjguMTQxLjczL29kYXRhLyRtZXRhZGF0YSNQcm9kdWN0cyh\
JZCxOYW1lKSIsInZhbHVlIjpbeyJJZCI6MSwiTmFtZSI6IkNvbGEifSx7IklkIjoyLCJOYW1lIjoiRm\
FudGEifSx7IklkIjozLCJOYW1lIjoiUGVwc2kifSx7IklkIjo0LCJOYW1lIjoiU3ByYWp0In1dfQotL\
WJhdGNocmVzcG9uc2VfNmI3NzUyNzctYjcxNS00MmU5LThlMTctOWU4MGJlMDZkNGY4LS0K",
            "encoding": "base64"
          },

Decoding this gave the following content (Hex representation, I've manually inserted blanks before each 0a and a newline after it):

2d2d6261746368726573706f6e73655f36623737353237372d623731352d343265392d386531372d396538306265303664346638 0a 
436f6e74656e742d547970653a206170706c69636174696f6e2f68747470  0a  
436f6e74656e742d5472616e736665722d456e636f64696e673a2062696e617279 0a  
0a  
485454502f312e3120323030204f4b 0a
4f446174612d56657273696f6e3a20342e30 0a  
436f6e74656e742d547970653a206170706c69636174696f6e2f6a736f6e3b206f646174612e6d657461646174613d6d696e696d616c3b206f646174612e73747265616d696e673d74727565 0a
0a
7b22406f646174612e636f6e74657874223a22687474703a2f2f35312e36382e3134312e37332f6f646174612f246d657461646174612350726f64756374732849642c4e616d6529222c2276616c7565223a5b7b224964223a312c224e616d65223a22436f6c61227d2c7b224964223a322c224e616d65223a2246616e7461227d2c7b224964223a332c224e616d65223a225065707369227d2c7b224964223a342c224e616d65223a22537072616a74227d5d7d 0a
2d2d6261746368726573706f6e73655f36623737353237372d623731352d343265392d386531372d3965383062653036643466382d2d 0a

From my POV, this proves that the browser receives a 0x0a only, no 0x0d before (2).

When you do the same for your local setup, we would expect to see a 0x0d 0x0a there. This still would not explain where the difference comes from, but we hopefully would be at the same page regarding why the UI5 OData model fails to parse the response.

(1) browsers might chose to base64 encode the content field when it might contain binary data, see the HAR file spec for details, if interested
(2) it is only a proof if we assume that the browser stores the response "as is". According to the previously mentioned HAR file spec, this should the case (citing the spec):

Before setting the text field, the HTTP response is decoded (decompressed & unchunked), than trans-coded from its original character set into UTF-8. Additionally, it can be encoded using e.g. base64. Ideally, the application should be able to unencode a base64 blob and get a byte-for-byte identical resource to what the browser operated on.

[siasty]

thank you!
you are right now i see what the problem is.

[flovogt]

Hi @siasty ,
looks like issue is clarified. With that I'll close the issue.
All the best,
Florian