A test case for showing Om Transit encoding failing with Pedestal and Figwheel
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
config
dev
resources/public
script
src
test/bot
.gitignore
README.md
project.clj

README.md

bot - Bad Om Transit

Test case for Om-Transit encoding issue with Reloaded Pedestal-backed Om Next stack.

FIXED With some help from Robert Stuttaford in the #clojure Slack channel, I was able to update the reset function in my Reloaded workflow to prevent files copied by Figwheel from being loaded by the server backend.

Goal

Create an Om Next application with a Pedestal remote. The Pedestal server uses Stuart Sierra's Component for system management and the Reloaded pattern for server development. Use Figwheel for client development.

Transit encoding for Om is handled by an onresponse interceptor from a gist by Andre R.

(defn body-writer [body]
  (fn [^OutputStream output-stream]
    (transit/write (om/writer output-stream) body)
    (.flush output-stream)))

(defn om-transit-json-body-fn
  [response]
  (let [body (:body response)
            content-type (get-in response [:headers "Content-Type"])]
        (log/info :fn ::om-transit-json-body :body body)
        (if (and (coll? body) (not content-type))
          (let [response'
                (-> response
                    (ring-resp/content-type "application/transit+json;charset=UTF-8")
                    (assoc :body (body-writer body)))]
            (log/info :fn ::om-transit-json-body :body (:body response'))
            response')
          response)))

(def om-transit-json-body
  "https://gist.github.com/rauhs/c137c0518cb7067f58ee"
  (on-response ::om-transit-json-body om-transit-json-body-fn))

Issue

When client code is compiled by Figwheel and server code is run in a repl, after reloading the server using (reset), #om/id tagged literals are no longer transit encoded correctly.

To test the Transit encoding, the server serves a hard-coded edn value body which is encoded by the interceptor. The edn value is:

(def edn-body
  `{todo/new-item
    {:tempids
     {~(tempid/tempid "2e486bfc-aacb-4736-8aa2-155411274e84") 852154481843896390}}})

To replicate

  1. Start Figwheel (which compiles the client code).

     rlwrap lein run -m clojure.main script/figwheel.clj
    
  2. Start a server repl (using lein repl or CIDER):

     lein repl
    
  3. Start the server via the repl

     (go)
    
  4. Confirm server requests properly Transit encode the #om/id tagged literal.

    The /om-str endpoint encodes the EDN directly in the handler, and its interceptor chain does not include the om-transit-json-body interceptor.

     curl localhost:8083/om-str
     ["^ ","~$todo/new-item",["^ ","~:tempids",["~#cmap",[["~#om/id","2e486bfc-aacb-4736-8aa2-155411274e84"],"~i852154481843896390"]]]]
    

    The /om-bw endpoint passes a function that accepts the output stream to which it will write the Transit-encoded body. This also does not include the om-transit-json-body interceptor in the interceptor chain.

     curl localhost:8083/om-bw
     ["^ ","~$todo/new-item",["^ ","~:tempids",["~#cmap",[["~#om/id","2e486bfc-aacb-4736-8aa2-155411274e84"],"~i852154481843896390"]]]]
    

    The /om-interceptor endpoint passes the edn as the body and includes the om-transit-json-body interceptor in the interceptor chain.

     curl localhost:8083/om-interceptor
     ["^ ","~$todo/new-item",["^ ","~:tempids",["~#cmap",[["~#om/id","2e486bfc-aacb-4736-8aa2-155411274e84"],"~i852154481843896390"]]]]
    

    Note all three encoded the #om/id tagged literal correctly.

  5. Reload the server via the repl

     (reset)
     ;; :reloading (om.util om.tempid om.transit bot.service bot.service-test cljs.stacktrace om.next.impl.parser bot.server user om.next.protocols)
    
  6. Confirm #om/id tagged literal is no longer being properly encoded.

     curl localhost:8083/om-str
     ["^ ","~$todo/new-item",["^ ","~:tempids",["~#cmap",[["^ ","~:id","2e486bfc-aacb-4736-8aa2-155411274e84"],"~i852154481843896390"]]]]
    
     curl localhost:8083/om-bw
     ["^ ","~$todo/new-item",["^ ","~:tempids",["~#cmap",[["^ ","~:id","2e486bfc-aacb-4736-8aa2-155411274e84"],"~i852154481843896390"]]]]
    
     curl localhost:8083/om-interceptor
     ["^ ","~$todo/new-item",["^ ","~:tempids",["~#cmap",[["^ ","~:id","2e486bfc-aacb-4736-8aa2-155411274e84"],"~i852154481843896390"]]]]
    

    All three no longer encode the #om/id tagged literal correctly.

Notes

  1. Starting Figwheel/compiling the client code is necessary for issue to occur, though Figwheel does not need to be running at the time requests are made to the server. The repl must be started (or restarted) after figwheel compiled the client code.

  2. The issue only occurs after reloading the server code via (reset).

The Fix

When Figwheel compiles the front-end application, it copies required .cljs (ClojureScript) and .cljc (multi-target files) into the resource directory. By default, clojure.tools.namespace.repl/refresh used in the Reloaded workflow to reload code looks for any Clojure files (.clj and .cljc) in the classpath, which includes the resource directory. So on reset, any .cljc files in resource were included that were not included in the original compilation. And somewhere backend code was getting stomped on by the code in resource.

Here's a list of those files:

> find ./resources/public/js -name "*.cljc"
./resources/public/js/cljs/stacktrace.cljc
./resources/public/js/om/next/impl/parser.cljc
./resources/public/js/om/next/protocols.cljc
./resources/public/js/om/tempid.cljc
./resources/public/js/om/transit.cljc
./resources/public/js/om/util.cljc

The culprit is om/transit.cljc. I ran Figwheel, deleted om/transit.cljc, and then ran (reset) and confirmed that the code worked as expected. Deleting another file, such as om/tempid.cljc, did not fix the bug. However, deleting files from resources isn't a very convenient or robust solution.

clojure.tools.namespace.repl includes a set-refresh-dirs function that lets you to specify where refresh will look. I updated my Reloaded code to set this to the classpath (the default) excluding the resource directory.

(ns user
  (:require ; ...
            [com.stuartsierra.component :as component]
            [clojure.tools.namespace.repl :as repl]
            [clojure.java.classpath :as cp]
            [clojure.java.io :refer [resource]])
  (:import [java.io File]))
; ...
(defn refresh-dirs
  "Remove `resource` path from refresh-dirs"
  ([] (refresh-dirs repl/refresh-dirs))
  ([dirs]
   (let [resources-path (-> "public" resource .getPath File. .getParent)
         exclusions #{resources-path}
         ds (or (seq dirs) (cp/classpath-directories))]
     (remove #(contains? exclusions (.getPath %)) ds))))

(defn reset
  "Destroys, initializes, and starts the current development system"
  []
  (stop)
   (apply repl/set-refresh-dirs (refresh-dirs))
  (repl/refresh :after 'user/go))

Problem solved.