custom mimetype support - add direct JSON

[DougBurke]

Can we add some way to pass JSON through displayDataToJson for the new custom mimetype? Rarndom thoughts are

a) add a separate mimetype - e.g. MimeCustomJSON Text - which can then be used in disolayDataToJson in a similar way to how MimeJson/MmeVegaite/MimeVega is handled

b) or change the MimeCustom type to treat the value in displayDataToJson directly as an Aeson Value

The bit I'd like to be able to handle is to pass a custom mimetype with a Value, not a String, in the followig.

-- from https://github.com/gibiansky/IHaskell/blob/master/ipython-kernel/src/IHaskell/IPython/Types.hs

displayDataToJson :: DisplayData -> (Text, Value)
displayDataToJson (DisplayData MimeJson dataStr) =
    pack (show MimeJson) .= fromMaybe (String "") (decodeStrict (Text.encodeUtf8 dataStr) :: Maybe Value)
displayDataToJson (DisplayData MimeVegalite dataStr) =
    pack (show MimeVegalite) .= fromMaybe (String "") (decodeStrict (Text.encodeUtf8 dataStr)
:: Maybe Value)
displayDataToJson (DisplayData MimeVega dataStr) =
    pack (show MimeVega) .= fromMaybe (String "") (decodeStrict (Text.encodeUtf8 dataStr) :: Maybe Value)
displayDataToJson (DisplayData mimeType dataStr) =
  pack (show mimeType) .= String dataStr

[vaibhavsagar]

What is the problem you're trying to solve here? Could you encode the Value as a Text and use the existing data types? If it turns out to be necessary to add a new mimetype we can do that but I'm wary because each time it breaks backwards compatibility and complicates the interface.

[DougBurke]

I am trying to test out the custom mimetype by changing instance IHaskellDisplay VegaLiteLab where display (VLL vl) = let js = LT.unpack (encodeToLazyText (fromVL vl)) in pure (Display [vegalite js]) to instance IHaskellDisplay VegaLiteLab where display (VLL vl) = let js = LT.unpack (encodeToLazyText (fromVL vl)) vegalite' = "application/vnd.vegalite.v4+json" in pure (Display [custom vegalite' js]) and it does not work because the custom message gets encoded as a String, whereas the vegalite message gets special cased to handle JSON. It is actually rather wasteful because I convert from Aeson to lazy text to string, and them the vegalite encoding in displayDataToJson converts back ti Aeson. In this case an alternative is to just bump the vegalite mimetype from v2+ to v4+, but a want to make sure we can handle the next time the version gets updated (ie use the custom message until ihaskell can get updated ) Doug

…

On Sat, May 16, 2020 at 10:29 PM Vaibhav Sagar ***@***.***> wrote: What is the problem you're trying to solve here? Could you encode the Value as a Text and use the existing data types? If it turns out to be necessary to add a new mimetype we can do that but I'm wary because each time it breaks backwards compatibility and complicates the interface. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1173 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABW27SP3G2RXZUXZSS6GUTRR5DYRANCNFSM4NDCHBPA> .

[vaibhavsagar]

Okay, I'd accept a PR adding this additional mimetype.

[vaibhavsagar]

I think this might be tricky to implement at least in part because of this line:

https://github.com/gibiansky/IHaskell/blob/d7dc460a421abaa41e04fe150e264bc2bab5cbad/ipython-kernel/src/IHaskell/IPython/Types.hs#L828

How would read know that something is a MimeCustomJSON instead of a regular MimeCustom?

[DougBurke]

#1176 is my initial attempt at this, but without resolving the readsPrec issue

[DougBurke]

I now am wondering if it would be better to just add an extra parameter to CustomMimetype to allow you to decide how the Text payload is converted to JSON - e.g. just Aeson.String or the fromMaybe (String "") (decodeStrict (Text.encodeUtf8 dataStr) :: Maybe Value) displayDataToJson (DisplayData mimeType dataStr) approach

[DougBurke]

Darn it. I was playing around with

data MimeType = PlainText
              | MimeHtml
              | MimeBmp Width Height
              | MimePng Width Height
              | MimeJpg Width Height
              | MimeGif Width Height
              | MimeSvg
              | MimeLatex
              | MimeMarkdown
              | MimeJavascript
              | MimeJson
              | MimeVega
              | MimeVegalite
              | MimeVdom
              | MimeCustom Text (Text -> Value)
  deriving (Eq, Typeable, Generic)

but the need instances give me pause (due to Text -> Value). We could instead have a simpler version, that lets you chose between String and fromMaybe (String "") (decodeStrict (Text.encodeUtf8 dataStr) :: Maybe Value)

[DougBurke]

I've added a second commit which uses a different design (moves the choice for the encoding to the CustumMimetype type).

[vaibhavsagar]

It still doesn't look like this new approach fixes the problem with readsPrec. I'm actually now leaning towards just keeping the wasteful old behaviour, it's far from ideal but I can't see another way that would also work sensibly in that case.

[DougBurke]

Do you want me to switch #1176 back to the original version?

[vaibhavsagar]

I don't think it makes a difference, the original version had the same issue.

[DougBurke]

Given the way the Read instance works for other types (e.g. hardcoding width / height) I'm personally don't think it is a serious issue (and technically I think the second version is "better" for Read, but not massively so).

[DougBurke]

In coming back to this, we have

MimeCustom Text

which is - I think - processed by this step

displayDataToJson (DisplayData mimeType dataStr) =
  pack (show mimeType) .= String dataStr

What I want is for it to not encode the data as a String but as JSON, that is something like

displayDataToJson (DisplayData (MimeCustom mimeType) dataStr) =
    mimeType .= textToJson dataStr

-- pulled out as it's used in several mime types
textToJson :: Text -> Value
textToJson dataStr = fromMaybe (String "") (decodeStrict (Text.encodeUtf8 dataStr) :: Maybe Value)

Last time I tried I don't think I was able to do this as a user of IHaskell.Display - that is create the JS code and convert it to a String value for use with a custom mimetype, but I should go back and check this.

[DougBurke]

So, I've created a notebook showing that I can't seem to use the custom mimetype to replicate the vega-lite mimetype because of this. You can find it at https://gist.github.com/DougBurke/11009ad97320e352c21a0d7558bb0b8b but the gist "auto display" doesn't seem to want to work for me.

In writing this I did note that the vega-lite mimetype has been bumped up to v4 so that means I personally am not affected by this for the moment (although the vega/vega-lite schema keeps changing so it could be an issue in the future). So I'm probably/hopefully not going to be spending much time on this in the near future.