(Done) April 9th 2025: Discussion: fast-edn and its broader implications

[frenchy64]

👍 to RSVP

https://madclj.com/

Wednesday, April 9th, 2025
6:30 PM to 9:00 PM CT

StartingBlock Madison
821 E Washington Ave 2nd floor · Madison, WI

fast-edn claims to parse EDN 6x faster than Clojure. It is based on charred, a similar library but for JSON.

Questions:

• How does fast-edn manage to outperform other approaches?
• Are you aware of applications that might run faster with 6x faster EDN parsing?
• Can these insights be used elsewhere in the Clojure compiler?
• fast-edn claims parity with clojure.edn. Which correspondences are important?
• How would you test these correspondences hold?
• Why is clojure.edn separate from Clojure's parser, despite EDN being a subset of Clojure syntax?
• fast-edn is faster than transit. When would use choose transit over fast-edn?

Schedule:

• 6:30pm arrive, food & drink provided
• 7pm, start discussion
• 9pm end meetup

Organizer notes:

• need paper plates + sparkle water

[frenchy64]

@madclj/meetup The April 9th meetup will be a overview and discussion about fast-edn and its broader implications.

[brunchboy]

This sounds interesting! Unfortunately I’ll be landing in Mexico City around then. ✈️ 😄

[ericnormand]

I can bring paper plates and water.

[frenchy64]

Thanks Eric!

I need to look into why the RSVP count doesn't show up on madclj.com.

[sivakusayan]

Something came up and I can't attend tomorrow 🙁 hope ya'll have fun though!

[frenchy64]

I'd like to add a quick group hacking session today if we have time.

The main theme will be improving test coverage of fast-edn's parity against clojure.edn. i.e., writing tests that increase evidence for fast-edn being a drop-in replacement for clojure.edn.

I have a specific idea that I can drive for:

There are already good tests in fast-edn, but I think generative tests can be improved. Currently, it tests something like:

(forall [x gen/any] (= (edn/read-string (pr-str x)) (fast/read-string (pr-str x))))

This only tests that the output of pr-str is parsed the same way in both. This does not test things like whitespace since pr-str always prints the same, or invalid edn.

For example, this tests if "[:a :b]" is parsed the same but not "[:a, :b]" or "[:a :b])".

[djwhitt]

Can't make it tonight unfortunately, but 👍 to both the topic and hacking session.

[frenchy64]

We found and fixed 3 bugs in about 45 minutes using the power of property-based testing. I'll put it here if anyone wants to continue hacking on it (make a PR against the madclj branch) and then we can report it all upstream. Maybe we can spin this into a generic "edn parser parity" library.

Written by @frenchy64 @ericnormand @balloneij @dpetranek

https://github.com/madclj/fast-edn/pull/1/files

Some observations we discussed:

• Clojure's edn parser is a copy-paste of the Clojure parser with the unsupported syntax deleted
• fast-edn should have a peek function for this idiom:
  • https://github.com/tonsky/fast-edn/blob/3a4cdbdae294a5386a56fa1eb59271a472456c22/src/fast_edn/EdnParser.java#L400-L401
• fast-edn manually implements buffered readers with arrays, even specializing them for certain readers:
  • https://github.com/tonsky/fast-edn/blob/3a4cdbdae294a5386a56fa1eb59271a472456c22/src/fast_edn/EdnParser.java#L91-L119
• fast-edn has some fascinating optimizations around bit masking, where clojure.edn uses IFn[]
  • https://github.com/tonsky/fast-edn/blob/3a4cdbdae294a5386a56fa1eb59271a472456c22/src/fast_edn/EdnParser.java#L1151-L1182
• fast-edn is beautifully implemented it was a breeze to fix the bugs
• we don't know why the existing data.generative tests didn't pick up the bugs we found, especially (edn/read-string "{0 -}")
  • https://github.com/tonsky/fast-edn/blob/3a4cdbdae294a5386a56fa1eb59271a472456c22/test/fast_edn/gen_test.clj#L33-L43

[brunchboy]

That's awesome, I wish I'd been able to be a part of it!

[frenchy64]

It was a lot of fun and we barely scratched the surface.

@madclj/meetup Anyone want to continue same time next week April 16th 7pm-9pm, while it's still fresh? 👍 if yes.