Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration guide 1.5 -> 2.0 #881

Open
andreasabel opened this issue Oct 11, 2021 · 13 comments
Open

Migration guide 1.5 -> 2.0 #881

andreasabel opened this issue Oct 11, 2021 · 13 comments

Comments

@andreasabel
Copy link
Member

andreasabel commented Oct 11, 2021

The changelog isn't explicit about how to migrate from aeson-1.5 to -2.0.
Please add a migration guide with the most common fixes needed.

@phadej
Copy link
Collaborator

phadej commented Oct 11, 2021

it's a changelog, not a migration guide. You can contribute your observations here.

You can pick ideas from

In summary, change import qualified Data.HashMap.Strict as HM to import qualified Data.Aeson.KeyMap as KM. Apply CPP to your liking.

@andreasabel
Copy link
Member Author

Here is one more: agda/agda@8037346

@tfausak
Copy link

tfausak commented Oct 16, 2021

So essentially the official advice is "good luck"? That's disappointing.

Here's my summary of the changes and what you'll need to do to support them:

  • Change Object to use an opaque KeyMap interface #866: Introduced the Data.Aeson.KeyMap.KeyMap type and replaced any HashMaps with KeyMaps. If your project depends on Aeson's Object type being a HashMap, then you'll have to update your code to use the new KeyMap instead. Since the interfaces are similar, usually you can use CPP to import either Data.HashMap.Strict or Data.Aeson.KeyMap depending on the version of Aeson.

  • Data.Aeson.Key #868: Introduced the Data.Aeson.Key.Key type and replaced any Text keys with Keys. If your project depends on keys being Text, for example if you use the pair function with something other than an overloaded string, you'll have to update your code to use the new Key representation instead. Since Key's interface is minimal, you'll probably have to introduce fromKey and toKey functions that are defined with CPP so that you can continue to work with Text values and convert to Keys as late as possible.

  • Drop GHC-7 support #872: Dropped support for GHC 7.10 and earlier. If you use an affected version of GHC, you can't upgrade to Aeson 2.

  • Add ordered keymap option #871: Added the ordered-keymap flag. When set, this flag changes the representation of KeyMap from HashMap to Map. This is the change that will prevent a hash flood attack. The flag is not enabled by default with Aeson 2.0.0.0, but it is enabled by default with version 2.0.1.0. If it's important that you avoid hash flood attacks, you should explicitly enable this flag!

  • Change +inf/-inf :: Double/Float serialization #873: Changed representation of infinite floating point values. Previously they were represented as null, but now they are "+inf" for positive infinity and "-inf" for negative infinity. This probably does not require any changes in your code, but be aware that JSON generated by Aeson may be different than before when it includes infinite floating point values.

  • Accept all values in FromJSON () and FromJSON (Proxy tag) #874: Changed FromJSON instances for () and Proxy a to accept any value. Previously they only accepted an empty array. This probably does not require any changes in your code, but be aware that JSON parsed by Aeson will now accept other values for these types.

  • If you do anything non-trivial with Aeson, you're probably going to need CPP (or something equivalent to it) in order to support these changes.

  • Since this is a super major version upgrade (from 1.5.x.y to 2.0.x.y), be sure to get your version bounds correct! Don't do >= 1.5 && < 2.1, which would allow a hypothetical version 1.6, which would have breaking changes. Instead do >= 1.5 && < 1.6 || >= 2.0 && < 2.1, or something equivalent to it like ^>= 1.5 || ^>= 2.0.

@Bodigrim
Copy link
Contributor

So essentially the official advice is "good luck"? That's disappointing.

Could we please stop bashing maintainers for doing free work? @phadej fixed the vulnerability (together with @Boarders), made a release, and marked this issue as "help wanted". I'm not sure what exactly is disappointing here.

@googleson78
Copy link

googleson78 commented Oct 16, 2021

While I do strongly agree with the sentiment of "let's not bash people doing free labour", I'd also like to express thanks to @tfausak for the write-up. This was not expressible in a clear way with emojis, so I wrote a comment instead.

@frasertweedale
Copy link

Here's my diff for jose: frasertweedale/hs-jose@ffc0cde

Apart from updating Text -> Key there are some places where I filter maps by key, which required a bit of explicit converting between Map Text <-> KeyMap (these have the same representation with +ordered-keymap, so should be zero runtime overhead) Overall it was a non-trivial but definitely not painful update.

@googleson78
Copy link

So it might be worth extending the KeyMap interface with a filtering mechanism?

@frasertweedale
Copy link

@googleson78 not necessarily. How to filter efficiently can depend on the underlying representation (e.g. if you want to keep/drop keys in an existing Set or HashSet).

Anton-Latukha added a commit to haskell-nix/hnix that referenced this issue Oct 28, 2021
Anton-Latukha added a commit to haskell-nix/hnix that referenced this issue Oct 28, 2021
Anton-Latukha added a commit to haskell-nix/hnix that referenced this issue Oct 29, 2021
gergoerdi added a commit to gergoerdi/clashilator that referenced this issue Dec 11, 2021
gergoerdi added a commit to gergoerdi/clashilator that referenced this issue Dec 11, 2021
@jneira
Copy link
Member

jneira commented Jan 10, 2022

I would like to note that a possible way to make the code work across bot Map implementations could be use aeson lens like operators or directly lens combinators with the help of lens-aeson, if you are comfortable with lenses and are already using them.
I learnt about that from the wise @michaelpj who added support for lsp without any cpp:

-addNullField :: Text -> Value -> Value
-addNullField s (Object o) = Object $ HM.insertWith (\_new old -> old) s Null o
+addNullField :: String -> Value -> Value
+addNullField s (Object o) = Object $ o <> fromString s .= Null
-  A.Object $
-    HMap.adjust
-      ( \(unsafeValueToObject -> o) ->
-          A.Object $ HMap.insert "plugin" elems o -- inplace the "plugin" section with our 'elems', leaving others unchanged
-      )
-      "haskell"
-      (unsafeValueToObject (A.toJSON defaultConfig))
+ -- Use 'ix' to look at all the "haskell" keys in the outer value (since we're not
+ -- setting it if missing), then we use '_Object' and 'at' to get at the "plugin" key
+ -- and actually set it.
+ A.toJSON defaultConfig & ix "haskell" . _Object . at "plugin" ?~ elems

Another example:

- "changes" `HM.member` editParams @? "Contains changes"
- not ("documentChanges" `HM.member` editParams) @? "Doesn't contain documentChanges"
+ (editParams & has (ix "changes"))  @? "Contains changes"
+ not (editParams & has (ix "documentChanges")) @? "Doesn't contain documentChanges"

@phadej
Copy link
Collaborator

phadej commented Jan 10, 2022

@jneira except we'll probably change (or break if you prefer that term) lens-aeson/lens-optics later on, when aeson-2.0 catches up. See lens/lens-aeson#37 (comment)

EDIT: that example will work as at is overloaded, and "plugin" is OverloadedStrings literal; but still, using lens-aeson is not a fail proof solution.

@jneira
Copy link
Member

jneira commented Jan 10, 2022

Well at least we have a lens-aeson bridge version with support for pre and post 2.0, so we can leverage it to avoid cpp in client code for now (with a little bit of luck cpp will not needed later :-) )

@bergmark
Copy link
Collaborator

bergmark commented Mar 7, 2022

@phadej from my perpective, the changelog is a perfectly fine place to put a migration guide along with the changes. Let me emphasize that I don't exptect you to do the work of writing it. If someone else did the work, would you accept it into the changelog, or is there another place where you would find it more suitable?

NorfairKing pushed a commit to NorfairKing/validity that referenced this issue Apr 26, 2022
NorfairKing pushed a commit to NorfairKing/validity that referenced this issue Apr 26, 2022
NorfairKing pushed a commit to NorfairKing/validity that referenced this issue Apr 26, 2022
Javran added a commit to Javran/shower that referenced this issue Aug 30, 2022
aeson-2 switched to use KeyMap interface instead of HashMap.

Fix haskell/aeson#881
Javran added a commit to Javran/shower that referenced this issue Aug 30, 2022
aeson-2 switched to use KeyMap interface instead of HashMap.

Some inpiration from haskell/aeson#881

Fix monadfix#17
@arobertn
Copy link

arobertn commented Jan 5, 2023

I'd just like to point out that many of the excellent suggestions provided here by @tfausak and others are in fact directed towards developers interested in building in both aeson-1 and aeson-2 environments. If you just want to migrate from 1.x to 2.x, like the title of this issue suggests, it is much easier. Switching out HashMap for KeyMap and utilizing the Key.{from,to}{String,Text} methods handles the lion's share. (KeyMap could use a ! method for more complete compatibility but that's not a major inconvenience.) No CPP needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants