Better builds for Clojure
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
src/meyvn Move main-class to compiler options Jun 29, 2018
CHANGELOG.org Changes Jun 13, 2018
README.org Windows Oct 19, 2018
deps.edn No Lip Service 1.0.3 Jun 15, 2018
meyvn.edn No Lip Service 1.0.3 Jun 15, 2018

README.org

Meyvn

You Know Nothing, Jon Snow. — Ygritte

The word maven comes from the Yiddish meyvn, meaning one who understands. You can think of Meyvn as:

  • A build tool
  • A wrapper for Maven
  • A tools.deps extension.

Meyvn enables you to generate uberjars (executables) and jars (libraries), and to deploy them on remote servers, Clojars, etc.

Goals

  • Use deps.edn as single source of truth.
  • One command to do it all: compile, package, install on a remote server, etc.
  • Standalone and lightweight. It only depends on Clojure and Maven (with no external dependencies).
  • Ability to selectively shrink uberjars
  • Leverage the Maven ecosystem
  • Include a good Clojurescript story: official compiler, shared aot cache, all options available to user, etc.
  • Sustainable open source with dual licensing scheme.

Quickstart

A “Hello, World” workflow with Meyvn.

Interactive programming

Installation

http://clojars.org/org.danielsz/meyvn/latest-version.svg

  • Meyvn requires Maven. Make sure it is installed on your system.
  • Define an alias in $HOME/.clojure/deps.edn:
:aliases {:meyvn
          {:extra-deps {org.danielsz/meyvn {:mvn/version "x.x.x"}}}
          ...}
  • Create a new shell script in your path.
$ touch /usr/local/bin/myvn
  • Edit your newly created shell script with the following.
$ cat /usr/local/bin/myvn
#!/bin/sh
M2_HOME=/usr/share/maven clj -A:meyvn -m meyvn.core "$@"

clj is the standard runner that ships with the Clojure installer and CLI tools. $@ references the arguments passed to the script. M2_HOME points to the root of your Maven installation directory, replace the content with the prefix appropriate for your system.

Note: Maven can run without this environment variable on the command line, but the Maven Invoker APIs require it to be set explicitly.

  • To find the prefix for the Maven installation on your system.
$ readlink -f `which mvn` | awk '{gsub("bin/mvn", ""); print}'

Note: On Mac OS X, brew install coreutils and replace readlink with greadlink.

  • Set the executable bit.
$ chmod +x /usr/local/bin/myvn

Usage

The standard Maven lifecycle phases and goals are passed as arguments. There’s documentation, too.

For example:

myvn compile 

Or

myvn package

Or

myvn deploy

You can also chain publishing goals, just like in Maven:

myvn clean clojure:test install

Debugging the build

If you see errors with the build, run myvn -g. This will persist Meyvn’s pom file. You can now run mvn on it and debug as you normally would in Maven. You will need to specify the path to the pom file.

mvn -f meyvn-pom.xml <goal>

Configuration

Configuration is stored in meyvn.edn, which will be created in the root of your project on first run.

Here are the defaults. Aside from the :pom key which captures the project coordinates and is always used, the other keys can be enabled or disabled as needed.

{:pom {:group-id "com.changeme"
       :artifact-id "myproject"
       :version "1.0.0"
       :name "My project does a lot"}
 :packaging {:uberjar 
             {:enabled true
              :main-class "main.core"
              :remote-repository {:id "ssh-repository"
                                  :url "scpexe://user@domain:/home/.m2/repository"}
              :excludes {:sets ["org.clojure:google-closure-library"]
                         :filters ["META-INF/*.MF" "META-INF/*.SF" "META-INF/*.DSA" "META-INF/*.RSA"]}}             
             :jar
             {:enabled false
              :remote-repository {:id "clojars" ;; Username and password lives in ~/.m2/settings.xml
                                  :url "https://clojars.org/repo"}}}
 :cljs {:enabled false
        :compiler-opts {:main "main.core"
                        :optimizations :advanced
                        :output-wrapper true
                        :infer-externs true
                        :parallel-build true
                        :aot-cache true
                        :output-to "resources/js/main.js"}
        :tools-deps-alias :cljs}
 :scm {:enabled true} ; will autodetect git repository
 :testing {:enabled false
           :tools-deps-alias :test} ; only in commercial version
 :profiles {:enabled false
            :staging {:http-port "3000"}
            :production {:http-port "8000"}}} ; only in commercial version

How does it work?

tools.deps has the ability to translate a deps.edn file into a pom file (clj -Spom). Meyvn starts off from that pom file and augments it with features that make sense for Clojure workflows. Meyvn’s pom file is transient and does not interfere with POM files that may already be present in your project.

Maven is invoked via an API (Apache Maven Invoker) and can be passed all lifecycle phases or goal it supports.

Clojurescript sources are compiled and included in the final artifact. Clojurescript compilation is done in its own process with the official compiler.

Uberjars

Consider the following deps.edn file:

{:paths ["src/clj"]
 :deps {org.clojure/core.async {:mvn/version "0.4.474"}
        ring {:mvn/version "1.6.3"}
        compojure {:mvn/version "1.6.1"}}
 :aliases {:cljs {:extra-deps {org.clojure/clojurescript {:mvn/version "1.10.238"}
                               reagent {:mvn/version "0.8.1"} 
                               secretary {:mvn/version "1.2.3"}}
                  :extra-paths ["src/cljs"]}}}

The Clojurescript-side of the mixed project is cleanly segregated. The :cljs alias is used when compiling the *.cljs files, but not when assembling the uberjar, helping to keep the latter small. You tell Meyvn to use this alias in the meyvn.edn configuration, under the cljs -> tools-deps-alias keys.

If there is a resources folder in the base directory, it will be included in the build.

Meyvn uses the Apache Maven Shade Plugin in order to build uberjars.

Shading dependencies is the process of including and renaming dependencies (thus relocating the classes & rewriting affected bytecode & resources) to create a private copy that you bundle alongside your own code. But the shading part is actually optional: the plugin allows to include dependencies in your jar (fat jar), and optionally rename (shade) dependencies.

Meyvn gives you access to the exclusions facility provided by the Shade plugin, equivalent to Leiningen’s uberjar-exclusions or Boot’s standard-jar-exclusions.

:excludes {:artifacts ["org.clojure:google-closure-library"]
           :filters ["META-INF/*.MF" "META-INF/*.SF" "META-INF/*.DSA" "META-INF/*.RSA"]}

Note that you don’t need to exclude INDEX/LIST as this is built-in by the Shade plugin.

Additionally, Meyvn allows you to exclude artifacts. For example, sometimes the Closure library is pulled by a transitive dependency and lands in your final uberjar. With Meyvn you can prevent that.

Data readers are merged with a custom transformer that knows how to merge EDN maps.

Regular jars

Libraries uploaded to Clojars are typically non-aot, source-only jars. Uploading to Clojars follows standard procedure. Private repositories are supported as well. For example, to upload an artifact to deps.co, adjust the remote repository setting in the jar section of meyvn.edn.

:jar
{:enabled true
 :remote-repository {:id "releases"
                     :url "https://repo.deps.co/your-org/releases"}}

In all cases, use settings.xml for storing your credentials, or refer to Maven for password encryption.

Pom files

Meyvn works with its own set of pom files. It isn’t bothered with existing pom files in your project directory. This is by design. The single source of truth is deps.edn. Together with the configuration (in meyvn.edn), it knows all that it needs to know.

The added benefit is that you can continue to maintain a pom file if you are already using a Maven workflow.

Dependency mechanism

The transitive dependency mechanism used by Maven is guided by the nearest wins conflict resolution strategy. This allows for resolution of individual conflicts: for any particular conflicting dependency, you can specify its version within your own POM, and that version becomes the nearest.

Note that if two dependency versions are at the same depth in the dependency tree, until Maven 2.0.8 it was not defined which one would win, but since Maven 2.0.9 it’s the order in the declaration that counts: the first declaration wins.

Testing

Consider the following deps.edn file.

{:paths ["src"]
 :deps {
   clj-time {:mvn/version "0.14.2"}
 }
 :aliases {:test {:extra-paths ["test"]
                  :extra-deps {org.clojure/test.check {:mvn/version "0.9.0"}}}}}

Again, please note the best practice of segregating paths and dependencies with aliases. To run your tests with Meyvn, make sure the relevant section in meyvn.edn looks like this:

:testing {:enabled true
          :tools-deps-alias :test}

Then run:

$ myvn clojure:test

The build will abort in case of errors.

Note: This feature is found in the commercial version only.

Interactive coding

$ myvn clojure:nrepl

This will start a nREPL server with Cider middleware that you can connect to with nREPL clients.

Note: This feature is found in the commercial version only.

Profiles

This is an experimental feature meant to help setting up an environment when deploying your code.

In Maven, profiles are used to parameterize builds, but we leverage the fact that custom properties can be defined under any profile.

When you enable the profiles section, Meyvn will create a Maven profile in the transient POM, and under each profile (for example, staging and production), it will write a standard edn map describing your environment into standard java properties.

On your staging/production server, those properties will be accessible in the pom alongside your jar in the local repository.

Meyvn doesn’t want to force you to install clojure or Meyvn on your servers, but if you do, you can use it to list those properties and pipe into a script in typical UNIX style.

$ myvn aux list-profile -a org.bar:foo:1.0.0  -p production | dosomething.sh

The -a switch is for artifact (in Maven coordinates) and -p is for profile.

Said script can, for example, massage the properties into environment variables. How you use them depends on your final command output, really. The last mile is context-dependent.

In the absence of Meyvn on the server, you can get the properties via the Maven helper plugin.

$ mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:all-profiles "-Dartifact=org.company:myproject:1.0.0
$ mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.properties -Dartifact=org.company:myproject:1.0.0

Note: This feature is found in the commercial version only.

Will it work?

It should work for the typical Clojure workflows. Please feel free to contact me in private if you want help solving your company’s build workflow.

Please note that Windows is not supported (the Clojure command line tools are not available).

Feel free to open issues regarding the supported workflows. New workflows will be added under commercial agreements.

Roadmap

This is just the beginning. The release of the clj command line tools is still fresh, and we are just starting to see the possibilities.

The takeaway for Meyvn is that building on top of the Maven ecosystem is rewarding. It is a huge ecosystem, well documented and extremely mature. A lot of functionality just sits there, waiting to be tapped by our tooling (in areas such as continuous integration, generated documentation, testing, reporting, etc.)

The plan is to have more features as companies sponsor them. Those features will be fed back to the OSS version.

What about Boot and Leiningen?

Naturally, Boot and Leiningen can also produce artifacts, but their scope is wider, providing development-time workflows and extension mechanisms.

Meyvn delegates build tasks to Maven, and offers direct access to the Clojurescript compiler.

In other words, there is no competition, only complementary options.

Sustainable open source

We as a community know how to write open source software, but we are less knowledgeable in how to make that activity sustainable. With Meyvn, I’m attempting to lead a sustainable Open Source project. That means that Meyvn is dual licensed, with a commercial license available for sale.

The LGPLv3 licensed community version will always remain free and available to all parties. However, companies who use Meyvn in their operations are expected to acquire a commercial license.

In the coming months, I will experiment with two competing models:

  1. Commercial and community version have parity of features
  2. Commercial version has more features than community version

What enables the first model is analytics. By sending data home, I can approach companies with proposals to acquire a commercial license. The features I am adding to the commercial version are fed back to the OSS version.

Pros: The community benefits. Cons: Tracking.

The second model doesn’t need tracking, because the distinction between a basic and a feature-laden version is by itself an incentive to buy the “better version”.

Pros: No tracking. Cons: The community loses.

The first model I am putting to test is the first model (with opt-in tracking). When you opt-in, Meyvn will send the POM’s group ID and success result of each execution back to an analytics server. When you opt-out, the program quits. At this stage, I am interested in users who can relate with the mission statement, for whom finding ways to do sustainable OSS is a shared value and not mere lip service.

The NoLipService library is responsible for the reporting. To ensure transparency it is released as open source as well. It is still early days, and I welcome contributions and different implementation ideas.

License

Meyvn is released under a dual licensing scheme.

Meyvn is an Open Source project licensed under the terms of the LGPLv3 license. Please see http://www.gnu.org/licenses/lgpl-3.0.html for license text.

Meyvn Enterprise has a commercial-friendly license allowing private forks and modifications of Meyvn. Licensees get a build of Meyvn with commercial features, and devoid of NoLipService’s reporting. Additionally, licensees get access to email support.

Please contact me for more details.

Patron

Writing and maintaining Open Source Software takes time and effort. Be a mensch. Be a maven. Patronize Meyvn.

Literature