Handling JSON from OCaml

[echuber2]

This came up on Slack. OCaml doesn't make it obvious how to deal with JSON exports. Here are some example uses of the Yojson library for this. (Forgive my amateurish OCaml. I only had version 4.07.1 handy in a Docker.)

Looking at several StackOverflow threads, I also realized some of the conventional wisdom about JSON encoding isn't quite right with regard to the standard. For example, many people say that you must put \\n everywhere, but the two-character sequence \n is acceptable in JSON. Where you have an OCaml String containing \n (which is one OCaml char as an escaped literal), Yojson and the OCaml standard libraries seem to accurately translate it to \n (two ASCII characters). It does similarly for \" and \\ (only where the backslash is meant literally, and not part of another escape sequence).

What this means is that when working with String literals in OCaml, as long as you're already using \", \\, and \n as normal, it should just work. I don't think it's necessary to use String.escaped or to do regex replacement. The escaped function encodes extended Unicode characters in a way that's not compatible with JSON decoders either.

So, here are some examples with Yojson. `Assoc of (string * t) list (for "associative" array?) is the basic constructor for an object.

#use "topfind";;
#require "yojson";;

let output_lines = [
  "More lines here with \"dangerous\" Unicode characters like 危険 😅";
  "Extra linebreak in\n  ...the middle of this line! \\ Etc."
];;
let combined_output = String.concat "\n" output_lines;;
let percentage_score = 0.85;;

let results_json = `Assoc [
  ("succeeded", `Bool true);
  ("message", `String "Testing completed.");
  ("output", `String combined_output);
  ("score", `Float percentage_score)
];;

(* This shows the algebraic data type in detail: *)
Format.printf "%a" Yojson.Basic.pp results_json;;
print_endline "";;

(* Preview the string format: *)
print_endline (Yojson.Basic.to_string ~std:true results_json);;

(* Write to file: *)
Yojson.Basic.to_file ~std:true "results.json" results_json;;

This produces the file results.json:

{"succeeded":true,"message":"Testing completed.","output":"More lines here with \"dangerous\" Unicode characters like 危険 😅\nExtra linebreak in\n  ...the middle of this line! \\ Etc.","score":0.85}

Python loads this without complaint:

>>> import json
>>> with open('results.json') as f:
...   j = json.load(f)
... 
>>> j
{'succeeded': True, 'message': 'Testing completed.', 'output': 'More lines here with "dangerous" Unicode characters like 危険 😅\nExtra linebreak in\n  ...the middle of this line! \\ Etc.', 'score': 0.85}

However, note that Python is overzealous about escaping Unicode sequences in its own encoder:

>>> print(json.dumps(j))
{"succeeded": true, "message": "Testing completed.", "output": "More lines here with \"dangerous\" Unicode characters like \u5371\u967a \ud83d\ude05\nExtra linebreak in\n  ...the middle of this line! \\ Etc.", "score": 0.85}

Other references:
https://dev.realworldocaml.org/json.html
https://ocaml-community.github.io/yojson/yojson/Yojson/Basic/index.html

The Yojson.Basic type is the simplest and seems to map directly to JSON, but Yojson.Safe has variants that could be used to write more advanced wrappers for OCaml types and extended integer ranges.