Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MIME type #194

Open
kyrias opened this issue Apr 4, 2015 · 24 comments

Comments

Projects
None yet
@kyrias
Copy link

commented Apr 4, 2015

It would be nice if there was a registered MIME type for msgpack, for use in eg. APIs.

@methane

This comment has been minimized.

Copy link
Member

commented Apr 10, 2015

I agree.
"application/x-msgpack" should be documented somewhere.

@kyrias

This comment has been minimized.

Copy link
Author

commented Apr 10, 2015

Well, x- is explicitly unregistered, and it’s also deprecated now since they made registering a proper media type simpler, so it should be rather easy to get application/msgpack registered.

@kuenishi

This comment has been minimized.

Copy link
Member

commented Apr 11, 2015

Fmmm. Do you know how easy is it to register? Looks like there's an application page and vendor/personal formats are rather easier to register. Standard formats requires being standardised, that's what library maintainers are not interested in.

@edsiper

This comment has been minimized.

Copy link

commented Jun 14, 2016

@arteymix

This comment has been minimized.

Copy link

commented Aug 3, 2016

Hi, just doing a follow-up here as I need to use some media type for a data service.

Is it reasonable to assume that vnd.msgpack will eventually be registered and use it as a value?

@Qix-

This comment has been minimized.

Copy link

commented Aug 3, 2016

@kyrias Which RFC deprecates x-? To my knowledge it's still a thing (RFC 2045/2046, which have not been obsoleted yet).


Looks like there's an application page and vendor/personal formats are rather easier to register.

@kuenishi correct. You'll want to register it as application/vnd.messagepack or application/vnd.msgpack. MsgPack is considered a vendor media type as per RFC 6838.

Admittedly I've been reading way too much of the MIME type RFCs lately for work. While I haven't submitted a media type before, I hear the IANA is pretty good about it.

@kyrias

This comment has been minimized.

Copy link
Author

commented Aug 4, 2016

@Qix-

This comment has been minimized.

Copy link

commented Aug 4, 2016

@kyrias oh interesting, I had no idea. Thanks for the link!

@cemremengu

This comment has been minimized.

Copy link

commented Nov 8, 2016

Just wondering what's the current status of this issue ?

@kyrias

This comment has been minimized.

Copy link
Author

commented Nov 8, 2016

The status essentially isn't.

@tagomoris

This comment has been minimized.

Copy link
Member

commented Apr 13, 2017

@frsyuki ping?
application/vnd.msgpack looks good to me.

@frsyuki

This comment has been minimized.

Copy link
Member

commented Apr 13, 2017

"x-" was deprecated by RFC 6648 which was published June 2012, but msgpack exists since February 2009.
From Github code search, application/x-msgpack is commonly used.

Therefore, registering application/x-msgpack officially is more beneficial than registering application/vnd.msgpack for the users, if possible.

RFC 6638 admits that there were wide use of "x-" names and Appendix A mentions that unfaceted "x-" names can also be registered. Although it also mentions some restrictions, and IANA or IESG or IETF or something unknown might advice me to take other ways, but I think it's worth to try.

@frsyuki

This comment has been minimized.

Copy link
Member

commented Apr 18, 2017

I started crafting the application texts for https://www.iana.org/form/media-types based on RFC 6838.
There're many examples I can refer to fill out the forms at Media Types list. Especially, applicatoin/json and vnd.apache.thrift.binary are similar to msgpack. application/x-www-form-urlencoded is an example of x- name, and application/sql is an example of format that has various implementatins clearly separated from the format specification.

Some fields are just obvious:

  • Type Name: "application"
  • Subtype Name: "Standards Tree (no prefix)", "x-msgpack"
  • Encoding Considerations: "binary"
  • Deprecated alias names for this type: N/A
  • Magic number: N/A
  • Macintosh File Type Code: N/A
  • Object Identifier: N/A
@frsyuki

This comment has been minimized.

Copy link
Member

commented Apr 18, 2017

Forms that are easy to fill out:

  • Required Parameters
  • Optional Parameters

They are charset= of text/html; charset=utf-8, for example. Nothing available for MessagePack.
They're N/A.

  • Fragment Identifier Considerations

This is +git of ssh+git, for example. N/A.

  • Restrictions on Usage

RFC 6848 section 4.11 says that this needs to describe something if the MIME type is not intended to be used widely. Maybe the RFC intends MIME types that wants to secure a name but doesn't want people to use it. N/A for MessagePack.

  • Published specification

Many existent MIME Types just mentions a website here.
MessagePack can use http://msgpack.org.
(O'Reilly's http://shop.oreilly.com/product/0636920032175.do is a good book that mentions about MessagePack, by the way)

  • Intended usage

Common.

Here could describe some details but that's mainly intended for other usage types (Limited use or Obsolete). So, nothing here for MessagePack.

@frsyuki

This comment has been minimized.

Copy link
Member

commented Apr 18, 2017

The other forms are more complex.

  • Security Considerations

JSON says:

it is possible to use that language's "eval()" function to parse JSON texts. This generally constitutes an unacceptable security risk

But this doesn't apply to MessagePack.

Thrift says:

Apache Thrift does not enforce a security mechanism.
Users define security features at application or transport level.
The data carried by Apache Thrift is essentially arbitrary, and can include
active or executable content. Apache Thrift does not provide or enforce a
security mechanism, so appropriate mechanisms must be implemented at the
application or transport level.

I think almost same text with Thrift works for MessagePack. But a difference is that Thrift includes RPC but MessagePack doesn't. Saying "transport level" is not always accurate. So, MessagePack would say...

MessagePack does not enforce a security mechanism. Users define security features at application, underlying transport level or data store.
The objects serialized by MessagePack are essentially arbitrary objects that can contain executable or secret content. Appropriate mechanisms must be implemented at the user application, underlying transport level or data store.
Deserializing too deeply nested objects, too long strings, or too long arrays can consume
large amount of memory rapidly depending on the deserialization implementation. When the
application deserializes objects serialized by untrusted sources, the application must reject
them appropriately to be tolerant of DoS attack.

@frsyuki

This comment has been minimized.

Copy link
Member

commented Apr 18, 2017

  • Application Usage

An existent mime type just says "Various" but it should be more specific to aim at standard tree, I think.

JSON says a good example:

JSON has been used to
exchange data between applications written in all of these
programming languages: ActionScript, C, C#, Clojure, ColdFusion,
Common Lisp, E, Erlang, Go, Java, JavaScript, Lua, Objective CAML,
Perl, PHP, Python, Rebol, Ruby, Scala, and Scheme.

MessagePack can take the same approach. So...

MessagePack has been used to exchange data between applications written in
various programming languages and environments including: ActionScript, Arduino,
C, C#, C++, Clojure, Crystal, D, Dart, Delphi, Elixir, Erlang, F#, Go, HHVM,
Haskell, Haxe, J, Java, JavaScript, Julia, Lua, Node.js, Objective CAML,
Objective-C, PHP, Pascal, Python, Qt, Ruby, Ruby on Rails, mruby, Rust,
Standard ML, Scala, Scala.js, Scheme, Smalltalk, and Swift.

@mnot

This comment has been minimized.

Copy link

commented Nov 8, 2017

@frsyuki if you want to register application/x-msgpack, that counts as a registration in the standards tree, so you'll need to author an Internet-Draft and get it published as an RFC; you can't just submit the form. That can be fairly easy, and I'm happy to help if you need it.

Note that it'll need to have a convincing case for a "grandfathered'" media type if you want to keep the x-, as per Appendix A of RFC6838. I suspect that it may be felt that application/x-msgpack isn't widely used enough to justify grandfathering, but we can see what happens. Getting application/msgpack registered would be easier (although still require an RFC).

While you're at it, it might be worth considering whether people would want a +msgpack suffix, similar to +xml, so you could have things like application/vnd.me.myformat+msgpack. That's becoming more popular for meta-formats like this.

If you don't want to bother with an RFC, you can get application/vnd.msgpack without going through that process; the form will be fine for it.

If we can get a sense from the msgpack community as to what it wants, I can talk to folks at IETF next week and figure out the best path forward.

@Qix-

This comment has been minimized.

Copy link

commented Nov 9, 2017

Huge 👍 to application/msgpack if possible. Also 👍 to +msgpack as MsgPack can be used as an alternative representation of the same data (like you said, similar to +json and +xml). I think it'd be worth it if the process is minimal since it would only help to encourage adoption.

I think the security considerations clause could be reduced a bit. Something like the following should suffice IMO.

MessagePack is a generic data format specification that doesn't specify, or otherwise concern itself with, any security mechanisms, requirements or guarantees. Likewise, the MessagePack format freely allows secret or executable data, thus requiring any and all security measures be implemented at the underlying application or transport levels.

It's probably a safe assumption that any consumers of MessagePack libraries are aware of the usual considerations that come with software engineering; namely that consuming third-party libraries has the perpensity to cause overhead and that consumers should be aware of what that overhead looks like.

This includes the potential for high memory/CPU usage, etc. I don't see a need to include that, personally - I don't think it adds anything to a section about security considerations specific to MessagePack itself.

@mnot

This comment has been minimized.

Copy link

commented Nov 12, 2017

Had a quick chat with an Area Director about this; it should be pretty straightforward.

The decisions that this community needs to make are:

  • Which media type(s) do you want to register?
  • Do you want a suffix?
  • Do you want to have a draft just for registering the media type, or do you want to make MSGPACK's specification an RFC?

Happy to answer any process questions, and help with draft authoring / review.

@frsyuki

This comment has been minimized.

Copy link
Member

commented Nov 12, 2017

@mnot Thank you for the comments & actions.

  • Certainly application/msgpack is the media type we would like to use: Reasons why we currently use application/x-msgpack is that application/msgpack is the straightforward MIME type MessagePack should use and x- was the common way to use such "unofficial" MIME types. I'm very sure that having application/msgpack officially registered is the ideal goal the community shares for this media type issue.
  • Having vnd + suffix is a compromise: I don't see technical benefits from having a suffix media type excepting a fact that it feels slightly more official compared to "application/x-msgpack". It seems that using application/x-msgpack is somewhat common way and I don't want to make confusion to those existent users. I thank that I should not decide to use it until we are certain that application/msgpack or application/x-msgpack is unrealistic.
  • If the standarization process of RFC makes the registered version of MessagePack incompatible to existent MessagePack spec, registering the media type only is the way to go. If we can make current specification of MessagePack an RFC without changes of the data format, I would like to make MessagePack an RFC.

There was a long story about RFC/IETF starting from this comment, 5 years ago.
There was an unfortunate that the topic was mixed with a discussion whether MessagePack should have Binary type or not (now we have). But our conclusion was not to do it because making MessagePack RFC os too hard process we/I could take at the time, and even worse, it may break compatibility because IETF tends to start building specification from scratch, from our understanding. We can't accept incompatible changes because there are big amount data already serialized & stored in MessagePack format. It ended up having RFC 7049 Concise Binary Object Representation led by another person. It's the current status around standarization of us, although I really don't think it's a good situation.

I suspect that it may be felt that application/x-msgpack isn't widely used enough to justify grandfathering, but we can see what happens.
Getting application/msgpack registered would be easier (although still require an RFC).

I understand about it and I didn't know have an idea that application/msgpack could be easier. What is the best way for us to continue this discussion?

@chintan-mishra

This comment has been minimized.

Copy link

commented Nov 28, 2017

Which MIME type should be used currently? I am using MsgPack for de-/serialization between Mobile app and Elixir servers. I cannot decide on which MIME type to select for HTTP request.

@mnot

This comment has been minimized.

Copy link

commented Jan 12, 2018

@frsyuki sorry for the delay.

It should be possible to make the current specification of MsgPack an RFC without changes; it would be published as an Informational RFC.

I mentioned a +msgpack suffix because it would allow other people to register media types like application/myformat+msgpack, to indicate that my format uses msgpack (just like application/myformat+xml does for XML). This doesn't rule out registering application/msgpack as well (see XML for an example here too).

Next step should be to generate an Internet-Draft that asks for the registration. I'm happy to help there; if you create a new repo (maybe media-type-registration?) and give me push access to it, I can sketch out the shape of what that would look like, if you want.

@rustonaut

This comment has been minimized.

Copy link

commented Aug 17, 2018

One argument against application/x-msgpack is that there is probably also application/x.msgpack out in the wild (I mean that is given RFC6838 the right think to use until msgpack is registered) , and just because application/msgpack is standarized it doesn't mean that your server stops accepting x-msgpack.

@ziggythehamster

This comment has been minimized.

Copy link

commented Jan 23, 2019

I'd like to give my +1 to application/msgpack and +msgpack, ignoring the current wild west situation of x-msgpack and vnd.msgpack. It's probably not too late to move to the correct MIME-type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.