StudyGate

StudyGate is is a full-stack Clojure(Script) app that uses the Fulcro framework, a batteries-included inheritor of Om Next.

See it live at My trial Dynamics 365 accounts have all run out, so there's no live version available :'(

Disclaimer

• Regarding the styles completely stolen from StudyTeam, please consider it flattery.

The Scenario

Many institutions use Microsoft's Dynamics CRM to manage the enormous and complex data required and generated by clinical research. Its flexibility, ease of use, and existing third-party solutions (such as SAGlobal's) make it a natural choice, especially in the realm of managing participant information and outcomes.

Since this is the case, and since every participants' journey in a clinical study begins with collecting their personal information (usually lots of it), wouldn't it be nice if there was a clean, easy-to-use gateway into the study, where user-inputted data was automatically persisted to CRM?

The Solution

Enter StudyGate, an online survey interface backed by Dynamics CRM. It is entirely data-driven- each survey is a pure function of the CRM's metadata, allowing the end users (in this case, the researchers) to define the survey as a normal CRM entity via a familiar drag-and-drop interface, and let StudyGate handle the rest. The stored data is just normal CRM data, allowing users to leverage all the OOB techniques CRM provides for custom business logic, reporting, and even integrations with email, Zapier, etc.

The Tech

• Clojure and ClojureScript
• fulcro
• figwheel
• SCSS
• dynamics-clj (a helper library I wrote for working with the CRM Web API)

Why this is Awesome

UI through Pure Functions!

Fulcro's data-driven story is amazing: you write a component as pure function of its colocatted query, and the underlying machinery reconciles them against a normalized database atom. Mutations against this database are an assoc-in cake-walk.

Content through Pure Functions!

StudyGate has exactly two concerns: pure rendering of CRM metadata, and writing user input back to CRM. Let the experts (the end users) define what data is important to them, how it should be collected and used. Surveys can be modified and deployed with a button-click.

What You Should Do About It

Give me an interview, of course! Or, if you're still not convinced, keep reading to learn more about the app and how I made it.

Experience Report/Code Walkthrough

This is my first Clojure web app. So, if I'm aiming to get an interview with Reify Health, why would I pick Fulcro over one the technologies they're using, namely Om Next or Re-frame?

I'm fascinated with Om Next. It appeals to me for web development for all the same reasons Clojure and ClojureScript do: Shared tools between client- and server-side development, DOM as pure functions, and powerful data abstractions. I genuinely believe its vision of unified client-server interaction can produce remarkable simplicity. However, as Eric Normand points out in his discussion of Om Next vs. Re-frame

"[Om Next] has too many degrees of freedom."

I'm still at the stage where I need the masters to define strong lanes I can play in, and where Re-frame looks wonderful for this, Om Next does not (nor claim to).

Fulcro aims to provide the best of both worlds: it's Om Next, but with batteries-included. Whereas Om Next leaves you with lots of decisions to make, Fulcro makes many7 of these for you, and in exchange provides the same architecture in a more compact and easy-to-use framework. With very little code on both the client and server, I was able to get a complete, data-driven app working. More importantly, Fulcro has provided the lanes to guide my understanding of how Om Next works, such that I would feel much more confident using in the future.

TDD and the Wisdom of Bill Murray

While I didn't write tests with this app, I still learned a tremendous amount of practical TDD in writing it. There were so many new variables (Clojure/ClojureScript/Fulcro/Figwheel... the list goes on), that I had to be extremely conservative in development, and adopted Bill Murray's profound wisdom of "Baby Steps". My strategy was to take Tony Kay's working TodoMVC implementation, and change one thing and one thing only at a time, see if it worked, fix what I broke, rinse, and repeat. I was able to perform this process all the way to the end, which resulted in a better grasp of Fulcro and a far more advanced app than I would have come up with otherwise. Thanks to Figwheel and hot code reload on the server, this was still quite a fast process.

The Architecture

A Fulcro client app consists of one or more UI components, which are pure functions of a single, normalized graph database representing all of the apps state. Browser events (user-interaction, timeouts, etc.) can trigger mutations against the state database, causing the app to re-render. Fulcro emphasizes the "keyframe" like nature of state => UI, which allows all sorts of cool things from time-travel debugging to visual regression testing, etc. Behind the scenes, Fulcro parses both the queries and mutations, and, when appropriate, sends them over the wire to one or more "remotes", which respond with the results of the operations. Under the hood, the Fulcro client optimizes both the React re-renders and the network traffic to make all of this as lightweight as possible.

That leaves you with only 3 things to write in a basic Fulcro app:

• The UI components and their queries.
• Server-side code capable of serving the queries
• Mutations (mirrored on the client and server)

The UI

You can find the UI in the aptly named ui.cljs. A few of the highlights:

• Each component, created with the defsc macro, includes its query, ident, and optional initial-state. These get composed into a tree, then Fulcro uses the the ident property to automatically normalize the state database for you. While this was quite tricky to figure out at first, the result is very loose coupling between each of the components, as well as between components and the database, which self-heals as the components are refactored.
• Every call to prim/transact! or routing/nav! (which calls transact!) triggers a mutation, addressed in the next section.
• The fun starts with the SurveyList component. Fulcro queries use a subset of Datomic's pull syntax, and here you can start to see that at work.

  • :query [:db/id [:ui/selected-survey '_] {:survey-list/surveys (prim/get-query Survey)}] translate to: "give me the :db/id property in the current node, the :ui/selected-survey property in the root node, and all the properties that the Survey component asks for, joining on the property :survey-list/survey in the current node.

  • At line 98, things get a little confusing if new to Fulcro. The SurveyList has one or more SurveyTile children. Tony Kay explains it best in the Fulcro Dev Guide:

    • The item itself cannot be truly composable if it has to know details of the parent. But a parent must always know the details of a child (it rendered it, didn’t it?).

    What this translates to is that in Fulcro, you must pass callbacks on other parent-computed data through prim/computed which decorates it in such a way that Fulcro doesn't clobber the properties on re-render. This allows child components to be agnostic of where the data comes from, and allows Fulcro to optimize re-renders as much as possible.

• The rest of the UI is general riff on these themes until the Survey component, which uses a multimethod to render the question appropriately based on its type (which comes straight from the Dynamics CRM).
  • This means extending the datatypes supported by the app is as simple as extending this multimethod!
  • The more "Fulcro" way to do this would be to use a union query. In fact, all the DOM switching that occurs based on if or case statements could be replaced with union queries, which are more robust and maintainable.

The Server Side Queries

• One of the major benefits of Fulcro is the ability parser enabling the server to respond to client queries in a decoupled way. However, for such as small app, I chose to instead create one endpoint that responds with the data tree needed by the UI. Fulro auto-normalizes this on the frontend, using the ident props on the components as discussed earlier.
• defquery-root lets you define your queries. Under the hood, it's just a multimethod.
• The root query, :surveys was one of the most fun pieces of the app to write. The main function is get-surveys, which looks through the CRM metadata for entities prefixed with survey_. On finding any, it fans out into several queries that comb through the entity attributes and convert the results into the shape needed by the UI. The result is something I've wanted to achieve in Dynamics CRM development for a long time: just add entities in the CRM, and the app automatically incorporates them!

The Mutations

• The mutations are split between the client-side and server-side code.
• The client mutations, which are the only source of change in the entire app, are dead simple. swap! on the state atom. Because of the database normalization, the properties are never more than a few layers deep. Easy day!
• Any mutation with a (remote ...) clause sends the mutation (represented as a quoted clojure data structure) to the server. (remote true) simply means to send the mutation as it was received (as opposed to modifying it beforehand).
• The client-side mutations are just as easy. In this case there's only one, submit, which writes survey to Dynamics CRM.
• The laconic nature of the mutation code speaks volumes for itself!

The Rest of the App

• routing.cljs hooks up HTML5 routing to the routing mutations in api.cljs.
  • In line 38 configure-routing! closes of the app's root-level "reconciler".
    • This is the very easiest way of doing routing, but admittedly NOT the best way, as every routing action triggers an app-level re-render. Fulcro provides its own idiomatic way of routing (union queries) to alleviate this.
• Some minimal bootstrapping happens in client_setup.cljs, client_main.cljs and server_main.clj, and that's it! Not bad for a completely data-driven forms app...

Running it

Figwheel/Clojurescript Build

Start figwheel (the JVM options tell figwheel which builds to run):

JVM_OPTS="-Ddev" lein run -m clojure.main script/figwheel.clj

which should start auto-building the cljs source and show a browser REPL.

You can do this in IntelliJ using a regular Clojure Main REPL that runs script/figwheel.clj (Parameters field). Put the -Ddev options can go in the JVM arguments field.

Our internal figwheel support uses Java system properties to select the builds you want to start (so you can create multiple run profiles for different tasks that target only specific builds). The supported build IDs are whatever builds are defined in the project file (it extracts build configurations from there). So, including -Dtest in the JVM arguments will include the build of tests.

Server

Start a CLJ REPL (e.g. command line or IntelliJ):

lein repl

At the REPL, start the server:

(go)

Navigate to: http://localhost:3000/index.html

If you make changes to the source code hot code reload will re-render without a browser reload.

Changes to server code can be put into effect at the REPL (NOTE: will wipe database) with:

(restart)

Fulcro Inspect Integration

While in dev mode you should be able to press CTRL-F to pop open inspection tools. These let you view the network interactions from a Fulcro perspective, browse the app database, and other things.

Credit

Based on Tony Kay's TodoMVC implementation, of which this is a fork. Many thanks to him for writing Fulcro and for his personal time helping me use it.

Many thanks to Eric Normand for his mentoring and to guys on Clojurian #beginners and #fulcro for enduring my pestering.

But most of all:

               ___           ___           ___                                
              /\  \         /\  \         /\__\      ___                      
             /::\  \       /::\  \       /:/  /     /\  \                     
            /:/\ \  \     /:/\:\  \     /:/  /      \:\  \                    
           _\:\~\ \  \   /:/  \:\  \   /:/  /       /::\__\                   
          /\ \:\ \ \__\ /:/__/ \:\__\ /:/__/     __/:/\/__/                   
          \:\ \:\ \/__/ \:\  \ /:/  / \:\  \    /\/:/  /                      
           \:\ \:\__\    \:\  /:/  /   \:\  \   \::/__/                       
            \:\/:/  /     \:\/:/  /     \:\  \   \:\__\                       
             \::/  /       \::/  /       \:\__\   \/__/                       
              \/__/         \/__/         \/__/                               
                        ___           ___           ___                       
                       /\  \         /\  \         /\  \                      
                      /::\  \       /::\  \       /::\  \                     
                     /:/\:\  \     /:/\:\  \     /:/\:\  \                    
                    /:/  \:\__\   /::\~\:\  \   /:/  \:\  \                   
                   /:/__/ \:|__| /:/\:\ \:\__\ /:/__/ \:\__\                  
                   \:\  \ /:/  / \:\~\:\ \/__/ \:\  \ /:/  /                  
                    \:\  /:/  /   \:\ \:\__\    \:\  /:/  /                   
                     \:\/:/  /     \:\ \/__/     \:\/:/  /                    
                      \::/__/       \:\__\        \::/  /                     
                       ~~            \/__/         \/__/                      
      ___           ___       ___           ___                       ___     
     /\  \         /\__\     /\  \         /\  \          ___        /\  \    
    /::\  \       /:/  /    /::\  \       /::\  \        /\  \      /::\  \   
   /:/\:\  \     /:/  /    /:/\:\  \     /:/\:\  \       \:\  \    /:/\:\  \  
  /:/  \:\  \   /:/  /    /:/  \:\  \   /::\~\:\  \      /::\__\  /::\~\:\  \ 
 /:/__/_\:\__\ /:/__/    /:/__/ \:\__\ /:/\:\ \:\__\  __/:/\/__/ /:/\:\ \:\__\
 \:\  /\ \/__/ \:\  \    \:\  \ /:/  / \/_|::\/:/  / /\/:/  /    \/__\:\/:/  /
  \:\ \:\__\    \:\  \    \:\  /:/  /     |:|::/  /  \::/__/          \::/  / 
   \:\/:/  /     \:\  \    \:\/:/  /      |:|\/__/    \:\__\          /:/  /  
    \::/  /       \:\__\    \::/  /       |:|  |       \/__/         /:/  /   
     \/__/         \/__/     \/__/         \|__|                     \/__/