Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Allow keys to be munged when encoding #34

Closed
wants to merge 1 commit into from

2 participants

Jeremy Heiler Lee Hinman
Jeremy Heiler

Basically the opposite of #27.

Jeremy Heiler jeremyheiler commented on the diff
src/cheshire/generate.clj
((7 lines not shown))
57 58 `(do
58 59 (.writeStartObject ~jg)
59 60 (doseq [m# ~obj]
60 61 (let [k# (key m#)
61 62 v# (val m#)]
62   - (.writeFieldName ~jg (if (keyword? k#)
63   - (.substring (str k#) 1)
64   - (str k#)))
  63 + (.writeFieldName ~jg (~key-fn (if (keyword? k#)
  64 + (.substring (str k#) 1)
  65 + (str k#))))
1

On second thought, maybe it should invoke key-fn or the existing code, not both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Jeremy Heiler

I am bit curious as to why the build failed. It appears to be an issue with test.generative 1.4.0 not supporting Clojure 1.2 (the pom.xml for it says Clojure 1.3) but the master builds pass. What am I missing?

Lee Hinman
Owner

@jeremyheiler sorry, I don't mean to be ignoring this, I'm out of town on vacation, so I won't be able to look at it until Monday or Tuesday evening, just wanted to give you a heads up.

Lee Hinman
Owner

So I've been thinking about this for the last couple of days on vacation. I think something like this is out of scope for a JSON encoder. Changing the data before calling json/encode is something the user's application should handle.

In addition, adding this change makes all generation take about 3.75 times longer, which is definitely too much of an overhead.

As a workaround, something like this https://gist.github.com/485ad11b922598c0c77c could be used.

Jeremy Heiler

Thanks for the feedback and the workaround.

I guess it bothers me that I cannot make the following return true by providing a lower-case function.

(let [data "{\"foo\":\"bar\"}" upper-case (fn [k] (.toUpperCase (name k)))]
  (= data (json/encode (json/decode data upper-case))))

Do you have any ideas about the cause of the performance decrease?

Lee Hinman
Owner

The performance decrease in most likely because of the map destructuring in the generate method.

Also, this doesn't correctly work on sub-maps, so something like:

(json/encode {:foo "bar" :bar {:baz "quux"}} {:key-fn (fn [k] (.toUpperCase k))})

Renders:

{"FOO":"bar","BAR":{"baz":"quux"}}

instead of:

{"FOO":"bar","BAR":{"BAZ":"quux"}}
Jeremy Heiler

That's unfortunate. My goal was to not break backwards compatibility by making it optional.

If you feel strongly about key transformation being out of scope, then I might just take some time to create a library that does just that. If you don't mind, I would like to use your gist as a starting point. Otherwise, I would be happy to continue looking into allowing full key transformation in cheshire.

Lee Hinman
Owner

I think it would be better implemented as a library so that it would be widely used (I am quite sure there are people who would like to do key transformations outside of json encoding). That way it is easily used for people who would like it for their json encoding, but not all users incur the performance penalty of it.

Jeremy Heiler

So, I created this library. It works like this:

(transform-keys (comp keyword clojure.string/lower-case camel->dash)
                {"FooBar" [{"Fancy1" nil "R2D2" nil} {"MoreNo1se" nil}]})
;=> {:foo-bar [{:fancy1 nil, :r2-d2 nil} {:more-no1se nil}]}

It's still very alpha, so I would appreciate any feedback :smile:

Repository: https://github.com/jeremyheiler/wharf

Lee Hinman
Owner

Neat, I'll check it out.

Lee Hinman
Owner

Closing this since it was addressed in #40 and released in 5.1.0.

Lee Hinman dakrone closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Jan 24, 2013
Jeremy Heiler jeremyheiler Allow keys to be munged when encoding ef5dd47
This page is out of date. Refresh to see the latest.
4 README.md
Source Rendered
@@ -79,6 +79,10 @@ encoders.
79 79 ;; generate JSON escaping UTF-8
80 80 (generate-string {:foo "It costs £100"} {:escape-non-ascii true})
81 81 ;; => "{\"foo\":\"It costs \\u00A3100\"}"
  82 +
  83 +;; generate JSON and munge keys with a custom function
  84 +(generate-string {:foo "bar"} {:key-fn (fn [k] (.toUpperCase k))})
  85 +;; => "{\"FOO\":\"bar\"}"
82 86 ```
83 87
84 88 In the event encoding fails, Cheshire will throw a JsonGenerationException.
3  src/cheshire/core.clj
@@ -28,7 +28,8 @@
28 28 (.enable generator JsonGenerator$Feature/ESCAPE_NON_ASCII))
29 29 (gen/generate generator obj
30 30 (or (:date-format opt-map) factory/default-date-format)
31   - (:ex opt-map))
  31 + (:ex opt-map)
  32 + :key-fn (or (:key-fn opt-map) identity))
32 33 (.flush generator)
33 34 (.toString sw))))
34 35
18 src/cheshire/generate.clj
@@ -53,15 +53,16 @@
53 53
54 54 (declare generate)
55 55
56   -(definline generate-map [^JsonGenerator jg obj ^String date-format ^Exception e]
  56 +(definline generate-map [^JsonGenerator jg obj ^String date-format ^Exception e
  57 + key-fn]
57 58 `(do
58 59 (.writeStartObject ~jg)
59 60 (doseq [m# ~obj]
60 61 (let [k# (key m#)
61 62 v# (val m#)]
62   - (.writeFieldName ~jg (if (keyword? k#)
63   - (.substring (str k#) 1)
64   - (str k#)))
  63 + (.writeFieldName ~jg (~key-fn (if (keyword? k#)
  64 + (.substring (str k#) 1)
  65 + (str k#))))
65 66 (generate ~jg v# ~date-format ~e)))
66 67 (.writeEndObject ~jg)))
67 68
@@ -79,13 +80,14 @@
79 80 ;;(println :inst? k obj)
80 81 `(instance? ~k ~obj))
81 82
82   -(defn generate [^JsonGenerator jg obj ^String date-format ^Exception ex]
  83 +(defn generate [^JsonGenerator jg obj ^String date-format ^Exception ex
  84 + & {:keys [key-fn] :or {key-fn identity}}]
83 85 (cond
84 86 (nil? obj) (.writeNull ^JsonGenerator jg)
85 87 (get (:impls JSONable) (class obj)) (#'to-json obj jg)
86 88 (i? IPersistentCollection obj) (condp instance? obj
87 89 clojure.lang.IPersistentMap
88   - (generate-map jg obj date-format ex)
  90 + (generate-map jg obj date-format ex key-fn)
89 91 clojure.lang.IPersistentVector
90 92 (generate-array jg obj date-format ex)
91 93 clojure.lang.IPersistentSet
@@ -95,7 +97,7 @@
95 97 clojure.lang.ISeq
96 98 (generate-array jg obj date-format ex)
97 99 clojure.lang.Associative
98   - (generate-map jg obj date-format ex))
  100 + (generate-map jg obj date-format ex key-fn))
99 101 (i? Number obj) (number-dispatch ^JsonGenerator jg obj ex)
100 102 (i? Boolean obj) (.writeBoolean ^JsonGenerator jg ^Boolean obj)
101 103 (i? String obj) (write-string ^JsonGenerator jg ^String obj )
@@ -103,7 +105,7 @@
103 105 (if-let [ns (namespace obj)]
104 106 (str ns "/" (name obj))
105 107 (name obj)))
106   - (i? Map obj) (generate-map jg obj date-format ex)
  108 + (i? Map obj) (generate-map jg obj date-format ex key-fn)
107 109 (i? List obj) (generate-array jg obj date-format ex)
108 110 (i? Set obj) (generate-array jg obj date-format ex)
109 111 (i? UUID obj) (write-string ^JsonGenerator jg (.toString ^UUID obj))
4 test/cheshire/test/core.clj
@@ -207,6 +207,10 @@
207 207 (is (= {"foo" "bar"} (json/decode "{\"foo\": \"bar\"}" false)))
208 208 (is (= {:foo "bar"} (json/decode "{\"foo\": \"bar\"}" true))))
209 209
  210 +(deftest t-custom-encode-key-fn
  211 + (is (= "{\"FOO\":\"bar\"}"
  212 + (json/encode {:foo "bar"} {:key-fn (fn [k] (.toUpperCase k))}))))
  213 +
210 214 (deftest test-add-remove-encoder
211 215 (gen/remove-encoder java.net.URL)
212 216 (gen/add-encoder java.net.URL gen/encode-str)

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.