Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Allow keys to be munged when encoding #34

Closed
wants to merge 1 commit into from

2 participants

@jeremyheiler

Basically the opposite of #27.

@jeremyheiler jeremyheiler commented on the diff
src/cheshire/generate.clj
((7 lines not shown))
`(do
(.writeStartObject ~jg)
(doseq [m# ~obj]
(let [k# (key m#)
v# (val m#)]
- (.writeFieldName ~jg (if (keyword? k#)
- (.substring (str k#) 1)
- (str k#)))
+ (.writeFieldName ~jg (~key-fn (if (keyword? k#)
+ (.substring (str k#) 1)
+ (str k#))))

On second thought, maybe it should invoke key-fn or the existing code, not both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jeremyheiler

I am bit curious as to why the build failed. It appears to be an issue with test.generative 1.4.0 not supporting Clojure 1.2 (the pom.xml for it says Clojure 1.3) but the master builds pass. What am I missing?

@dakrone
Owner

@jeremyheiler sorry, I don't mean to be ignoring this, I'm out of town on vacation, so I won't be able to look at it until Monday or Tuesday evening, just wanted to give you a heads up.

@dakrone
Owner

So I've been thinking about this for the last couple of days on vacation. I think something like this is out of scope for a JSON encoder. Changing the data before calling json/encode is something the user's application should handle.

In addition, adding this change makes all generation take about 3.75 times longer, which is definitely too much of an overhead.

As a workaround, something like this https://gist.github.com/485ad11b922598c0c77c could be used.

@jeremyheiler

Thanks for the feedback and the workaround.

I guess it bothers me that I cannot make the following return true by providing a lower-case function.

(let [data "{\"foo\":\"bar\"}" upper-case (fn [k] (.toUpperCase (name k)))]
  (= data (json/encode (json/decode data upper-case))))

Do you have any ideas about the cause of the performance decrease?

@dakrone
Owner

The performance decrease in most likely because of the map destructuring in the generate method.

Also, this doesn't correctly work on sub-maps, so something like:

(json/encode {:foo "bar" :bar {:baz "quux"}} {:key-fn (fn [k] (.toUpperCase k))})

Renders:

{"FOO":"bar","BAR":{"baz":"quux"}}

instead of:

{"FOO":"bar","BAR":{"BAZ":"quux"}}
@jeremyheiler

That's unfortunate. My goal was to not break backwards compatibility by making it optional.

If you feel strongly about key transformation being out of scope, then I might just take some time to create a library that does just that. If you don't mind, I would like to use your gist as a starting point. Otherwise, I would be happy to continue looking into allowing full key transformation in cheshire.

@dakrone
Owner

I think it would be better implemented as a library so that it would be widely used (I am quite sure there are people who would like to do key transformations outside of json encoding). That way it is easily used for people who would like it for their json encoding, but not all users incur the performance penalty of it.

@jeremyheiler

So, I created this library. It works like this:

(transform-keys (comp keyword clojure.string/lower-case camel->dash)
                {"FooBar" [{"Fancy1" nil "R2D2" nil} {"MoreNo1se" nil}]})
;=> {:foo-bar [{:fancy1 nil, :r2-d2 nil} {:more-no1se nil}]}

It's still very alpha, so I would appreciate any feedback :smile:

Repository: https://github.com/jeremyheiler/wharf

@dakrone
Owner

Neat, I'll check it out.

@dakrone
Owner

Closing this since it was addressed in #40 and released in 5.1.0.

@dakrone dakrone closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 24, 2013
  1. @jeremyheiler
This page is out of date. Refresh to see the latest.
View
4 README.md
@@ -79,6 +79,10 @@ encoders.
;; generate JSON escaping UTF-8
(generate-string {:foo "It costs £100"} {:escape-non-ascii true})
;; => "{\"foo\":\"It costs \\u00A3100\"}"
+
+;; generate JSON and munge keys with a custom function
+(generate-string {:foo "bar"} {:key-fn (fn [k] (.toUpperCase k))})
+;; => "{\"FOO\":\"bar\"}"
```
In the event encoding fails, Cheshire will throw a JsonGenerationException.
View
3  src/cheshire/core.clj
@@ -28,7 +28,8 @@
(.enable generator JsonGenerator$Feature/ESCAPE_NON_ASCII))
(gen/generate generator obj
(or (:date-format opt-map) factory/default-date-format)
- (:ex opt-map))
+ (:ex opt-map)
+ :key-fn (or (:key-fn opt-map) identity))
(.flush generator)
(.toString sw))))
View
18 src/cheshire/generate.clj
@@ -53,15 +53,16 @@
(declare generate)
-(definline generate-map [^JsonGenerator jg obj ^String date-format ^Exception e]
+(definline generate-map [^JsonGenerator jg obj ^String date-format ^Exception e
+ key-fn]
`(do
(.writeStartObject ~jg)
(doseq [m# ~obj]
(let [k# (key m#)
v# (val m#)]
- (.writeFieldName ~jg (if (keyword? k#)
- (.substring (str k#) 1)
- (str k#)))
+ (.writeFieldName ~jg (~key-fn (if (keyword? k#)
+ (.substring (str k#) 1)
+ (str k#))))

On second thought, maybe it should invoke key-fn or the existing code, not both.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
(generate ~jg v# ~date-format ~e)))
(.writeEndObject ~jg)))
@@ -79,13 +80,14 @@
;;(println :inst? k obj)
`(instance? ~k ~obj))
-(defn generate [^JsonGenerator jg obj ^String date-format ^Exception ex]
+(defn generate [^JsonGenerator jg obj ^String date-format ^Exception ex
+ & {:keys [key-fn] :or {key-fn identity}}]
(cond
(nil? obj) (.writeNull ^JsonGenerator jg)
(get (:impls JSONable) (class obj)) (#'to-json obj jg)
(i? IPersistentCollection obj) (condp instance? obj
clojure.lang.IPersistentMap
- (generate-map jg obj date-format ex)
+ (generate-map jg obj date-format ex key-fn)
clojure.lang.IPersistentVector
(generate-array jg obj date-format ex)
clojure.lang.IPersistentSet
@@ -95,7 +97,7 @@
clojure.lang.ISeq
(generate-array jg obj date-format ex)
clojure.lang.Associative
- (generate-map jg obj date-format ex))
+ (generate-map jg obj date-format ex key-fn))
(i? Number obj) (number-dispatch ^JsonGenerator jg obj ex)
(i? Boolean obj) (.writeBoolean ^JsonGenerator jg ^Boolean obj)
(i? String obj) (write-string ^JsonGenerator jg ^String obj )
@@ -103,7 +105,7 @@
(if-let [ns (namespace obj)]
(str ns "/" (name obj))
(name obj)))
- (i? Map obj) (generate-map jg obj date-format ex)
+ (i? Map obj) (generate-map jg obj date-format ex key-fn)
(i? List obj) (generate-array jg obj date-format ex)
(i? Set obj) (generate-array jg obj date-format ex)
(i? UUID obj) (write-string ^JsonGenerator jg (.toString ^UUID obj))
View
4 test/cheshire/test/core.clj
@@ -207,6 +207,10 @@
(is (= {"foo" "bar"} (json/decode "{\"foo\": \"bar\"}" false)))
(is (= {:foo "bar"} (json/decode "{\"foo\": \"bar\"}" true))))
+(deftest t-custom-encode-key-fn
+ (is (= "{\"FOO\":\"bar\"}"
+ (json/encode {:foo "bar"} {:key-fn (fn [k] (.toUpperCase k))}))))
+
(deftest test-add-remove-encoder
(gen/remove-encoder java.net.URL)
(gen/add-encoder java.net.URL gen/encode-str)
Something went wrong with that request. Please try again.