"Email/import" fails with "Message contains invalid header"

[zvasilev]

Hello,

My app tests are based on uploading MIME messages via JMAP upload endpoint.
It uses to work with www.fastmail.com/jmap/upload endpoint.
Now I'm trying to switch to the official JMAP endpoints returned by https://jmap.fastmail.com/.well-known/jmap and "Email/import" started to fail

The upload is fine but API call claim that "Message contains invalid header"

,"type":"invalidEmail".

This is the request

{"methodCalls":[["Email/import",{"accountId":"u606140f3","emails":{"f99da7d7-cae3-4294-9c5f-ee1cfb66911d":{"blobId":"Gc1df680e393d04a2a3b050a06db652e9ab37f761","keywords":{},"mailboxIds":{"fc4559dd-fccb-4d79-bf17-887d0e290277":true}}}},"B4JYCQ==#11"]],"using":["urn:ietf:params:jmap:core","urn:ietf:params:jmap:mail","urn:ietf:params:jmap:submission"]}

And this is the response
{"accountId":"u606140f3","created":{},"notCreated":{"f99da7d7-cae3-4294-9c5f-ee1cfb66911d":{"description":"Message contains invalid header","type":"invalidEmail"}}}

After skim the Cyrus code seems that error is too generic and is not related to an email address at all.
Seemd it is fails here
in jmap_mail.c _cyrusmsg_from_buf function.

The uploaded blob attached as zip file.

msg1.zip

Any hint will be welcome.

Thanks,
Zhivko Vasilev

[brong]

"blobId":"Gc1df680e393d04a2a3b050a06db652e9ab37f761",

That's not the blobId I get when I import it:

cyrusimap/cassandane@568a98e

Which suggests that whatever codepath you're using for uploading is converting line endings, and hence the headers are invalid! You need to upload with the \r\n intact.

JMAP UPLOAD $VAR1 = {
  'content' => '{
  "accountId": "cassandane",
  "blobId": "Ge7dff9ffb9547bfe0817ee03ec3c8396c3a1be53",
  "size": 248,
  "expires": "2020-07-26T08:39:07Z",
  "type": "message/rfc822"
}',

[zvasilev]

Thanks for your response Bron!
It turns out that I've uploaded with "content-encoding", "gzip".
The https://jmap.fastmail.com/.well-known/jmap is HTTP 1.1 only, and I turned on gzip for all request.
When turning it off for upload the "Email/import" passed successfully.
Still, the returned error is a bit misleading.

[brong]

I guess - I mean if it's gzipped it's nothing like an email! That's the downside of the blob system, it's just a bunch of bytes.

[zvasilev]

Hi, I'm' happy to continue to implement JMAP, so I'll move on :) .

If it is a regular HTTP server I would expect HTTP error code 415 Unsupported Media Type when trying to upload with unsupported "Content-Encoding".

[brong]

That seems reasonable. @ksmurchison do we want to implement Content-Encoding checking (and maybe decoding) in Cyrus?