Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Use SNAPSHOT plugin dependency to deliver up-to-date ClojureScript/cljsbuild compatibility info #266

Closed
cemerick opened this Issue · 16 comments

7 participants

@cemerick
Collaborator

ClojureScript, and specifically its compiler API, is very volatile, such that various versions of lein-cljsbuild are only compatible with certain ranges of ClojureScript builds. However, because ClojureScript's release versioning isn't anything remotely semver, there's no way to determine (or even speculate about) compatibility based on its version number. This means that a user has no way to know whether ClojureScript 0.0-XXXX will work correctly with lein-cljsbuild A.B.C short of trying the combination, and either looking for errors, or potentially discovering incorrect compilation results long after changing dependencies. This all means that, even if maven version ranges were acceptable within the Clojure community, lein-cljsbuild could not use them to declare upstream compatibility.

(This is even worse than most other dependency conflicts / compatibility problems, since even the most hipster of libraries will bump the "minor" version number [in semver parlance] when breakage and/or major features are introduced. Because the only information in the ClojureScript version string is a monotonically-increasing commit number, it's effectively just dumb luck when a particular ClojureScript release works properly with a lein-cljsbuild release that wasn't "qualified" against it. Also, the ClojureScript dependency is utilized by both build-time components [like cljsbuild] and delivers runtime APIs, so users understandably want to upgrade it independent of cljsbuild releases to benefit from improvements in the latter without waiting for cljsbuild maintainers to act.)

My current idea is to use a plugin-side SNAPSHOT dependency to deliver compatibility information as it becomes available. So, when a new rev of ClojureScript is released, and problems are discovered/reported when using it with various lein-cljsbuild releases, that compatibility info can be updated via a new SNAPSHOT deployment, capping the ClojureScript releases can can be used with potentially many lein-cljsbuild releases.

The big downside of this is that that SNAPSHOT will provoke dependency resolution on every lein-cljsbuild run, though lein/aether defaults should ensure that that results in an actual network check only every 24 hours. lein-cljsbuild is a plugin, and so this SNAPSHOT will not affect users' projects' dependencies at all.

If this causes unforeseen problems, then rolling it back will be straightforward.

@cemerick
Collaborator

Quoting @mjg123, just to keep the conversation in one place:

Apologies if you've covered this elsewhere - but why not have a dependency from lein-cljsbuild to ClojureScript at a particular known-good version and let people use the normal dependency exclusion mechanism if they want to use something different?

Even the exclusion isn't necessary; any ClojureScript dependency you add locally will override what lein-cljsbuild provides.

The problem is, it's impossible for a user to know if a new ClojureScript version will work with their existing lein-cljsbuild plugin version. This issue exists elsewhere as well, but it's particularly painful in this case because the ClojureScript compiler API changes rapidly, and likely will continue to do so. It's a perfect storm of conflated tooling/library + high rate of churn/breakage + information-free version numbers. People get by by asking why things are busted for particular combinations in irc, via issues here, and so on, but that sucks for everyone: users, maintainers of cljsbuild and ClojureScript, and everyone else in the vicinity that has to filter out the attendant noise.

We could simply say, "You must use the version of ClojureScript that lein-cljsbuild provides" (and that is how it worked in the past), but that puts the onus of qualifying cljsbuild/ClojureScript compatibility on the cljsbuild maintainers, and gates access to the latest (maybe, hopefully?) non-breaking release of ClojureScript on whenever cljsbuild is released. I'm hoping to find a hack that let's people run ahead of cljsbuild as much as possible, and give them reasonable, actionable error messages when a particular combination falls outside of the range of known compatibility.

Hopefully that clarifies my thinking more…?

@tobias

I assume you've already explored having those in charge of ClojureScript releases actually use sane versions, so I won't suggest doing that :)

I like the idea of a SNAPSHOT that provides compatibility info, but I would be annoyed watching that SNAPSHOT resolve every day, especially when I use the lein-cljsbuild plugin rarely. An alternative to having the plugin depend on the SNAPSHOT is to have the plugin itself use pomegranate to resolve the SNAPSHOT only when you are using a cljsbuild task. It would still resolve only once a day at most, right?

@puredanger

Maven doesn't let you release something that depends on a SNAPSHOT - is this a problem?

Why not use some of the 0.0 parts for CLJS?

There are various enforcer plugins for Maven (and maybe for lein, haven't looked) - seems like lein-cljsbuild could at least enforce a min CLJS version it's known to support (presumably at the time of release max version is not known, so not sure that can be set).

Is there a compatibility page somewhere?

@jeluard

Great to see that you are planning to tackle that issue!

What about just concentrating on the compiler API part? An option could be to have a var defining an API number ClojureScript side which would be modified every time there is a breaking change. From what I saw last months the compiler API doesn't change that often (definitively much less often than ClojureScript is released).

Given this is an issue other tools will also face it looks interesting to spend some time making sure the API can be reliably manipulated.

@cemerick
Collaborator

@tobias Yeah, don't want to wade into the version number discussion upstream (again).

I think the resolution message will only appear if a new SNAPSHOT is available. If the compatibility matrix hasn't changed, then you shouldn't see anything.

@puredanger No one depends upon lein-cljsbuild; it's a plugin, so at worst, only other plugins would depend upon it (of which there are none AFAIK). Leiningen has the same deployment restriction, though it can be worked around by setting an environment variable, which the deployment script for lein-cljsbuild does with this commit.

What do you mean re: the 0.0 parts? Those are set by ClojureScript and its (ill-conceived IMO) versioning scheme. You'd have more leverage there than (almost) anyone, perhaps?

Yeah, there are enforcer-like plugins here and there. But, the main source of pain a.t.m. is the max ClojureScript version that a given cljsbuild release can work with, thus using the SNAPSHOT mechanism to be able to get data down to the plugin after it's been released.

There is no compatibility page, just tribal knowledge. The start of a compatibility table is here; it only contains information for releases that would be able to access the SNAPSHOT dep, tho.

@puredanger

For the 0.0, that is what I meant - I don't do anything with CLJS right now but imo @swannodette should be able to change that as he sees fit (or lobby the power that be if not). Seems like this is the source of the issue?

@mjg123

I'm not sure about lein, but maven would not let you release the lein-cljsbuild plugin without a SNAPSHOT qualifier if it depended on a SNAPSHOT of the compat jar.

@cemerick
Collaborator

@jeluard You're right that the compiler API doesn't change on every release, but it does change "often"; at least three times in the last 6 releases. Some of these have been impactful, some not. Also, there is the topic of compiler breakage; i.e. > 1845 and < 2014, the compiler's output was broken for projects that had more than one build. In any case, there are more compiler changes in the pipeline, not less.

@cemerick
Collaborator

@puredanger Not using something resembling semver is a huge contributing factor, but that was decided upon "long ago" (and has been carried over / embellished for core.async). I don't know who owns that lever, but I don't want to wait for a coordination of policy to address the problem.

In any case, I'm happy to let ClojureScript change as it has to without necessarily worrying about downstream consequences. Given what needs to be done there, I honestly don't know if there's any versioning scheme that can reasonably represent the impactfulness of each change, without completely poisoning the version number "space" for the org.clojure/clojurescript coordinates once things settle down. Once this becomes less of a problem (either because the compiler API / options / stability eventually settles out or the versioning scheme becomes sane), then cljsbuild can drop this scheme.

@cemerick
Collaborator

@mjg123 Yup; lein provides a workaround for such things.

@Chouser

I would vote against this proposal, for the normal release-depending-on-snapshot reasons, and instead suggest loud recommendations in cljsbuild docs that users who want a different clojurescript depend on it directly.

Everyone knows this already, but just to reiterate -- having cljsbuild depend on a specific version of clojurescript gives strong guarantees that a project depending on a specific version of cljsbuild will build the same today as it will a year from now (yes, bugs and all). It's a value, if you will. As soon as you start including SNAPSHOTs in the dep tree it has become a reference (if you will) and reproducing the same build becomes difficult or impossible.

That said, it wouldn't necessarily be catastrophic, but if cljsbuild starts using a SNAPSHOT build I'd want to make sure that my project (and book text) always depend on both a cljsbuild version and a specific non-snapshot clojurescript version.

@swannodette

It's going to take me a while to read through this thread, basing version changes on compiler changes is not going to happen. No one has made any promises about the analyzer, compiler, or closure and none are going to be made. If somebody wants to step forward and design a stable api, do it - and then we can start talking about considering the backend with respect to version numbers.

That said version numbers might be nice when we really actually break something in the semantics of the language - real keywords was one of those.

Even so the current approach was put in place by Rich and I think we need to check with him first.

@cemerick
Collaborator

@Chouser Two things:

  1. The SNAPSHOT in question never touches your project; it's strictly a way to get information about cljs compatibility into the plugin.
  2. cljsbuild has been (and will continue to) warn when you don't specify a local ClojureScript dependency. Even if you didn't, then your project will use the version that lein-cljsbuild provides, which will never have anything to do with the SNAPSHOT.

Perhaps that lightens your concerns?

@Chouser

@cemerick Ah, I had forgotten about the warning. It looks like my projects do indeed name a specific clojurescript version, no doubt because of the warning. Thanks for that.

@cemerick
Collaborator

I've triple-verified that this works as intended, and that the dependency resolution doesn't produce any console output unless there's an update to the cljs-compat SNAPSHOT dep. This whole arrangement should be transparent to users, until their build stops when they upgrade their cljs dep to one that cljsbuild is known to not support. :-)

Merging to master, and releasing shortly.

@cemerick cemerick closed this
@cemerick
Collaborator

Also, thanks everyone for your input, suggestions, and generally just helping verify that there's no other obvious solution being overlooked. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.