Move exact-dependencies.json out of elm-stuff

[thewoolleyman]

Hi,

(New to Elm but loving it so far! Good job!)

I always like to commit the file that specifies my entire exact dependency tree, so I can always see what changed in source control and revert if necessary. E.g. Gemfile.lock, npm-shrinkwrap.json.

So, I tried committing exact-dependencies.json in my Elm project, but since it was under elm-stuff, I had to .gitignore elm-stuff/packages and elm-stuff/build-artifacts so that I could commit it.

Which worked fine, but then I wanted to delete and regenerate /packages and /build-artifacts to debug something that looked like something was stale under those dirs.

However after deleting them but leaving elm-stuff there and empty except for exact-dependencies.json, I did elm-package install to recreate, but it didn't recreate ./packages! And of course things failed, because I didn't even have elm-core.

I ended up having to delete the entire elm-stuff dir, then packaging worked fine.

That may be a separate bug, but even aside from that, I think it would be better to move exact-dependencies to the same level as elm-package.json, just like Gemfile + Gemfile.lock, and package.json+npm-shrinkwrap.json are next to each other.

What do you think?

Thanks!
-- Chad

[thewoolleyman]

@evancz Would you be open to a Pull Request for this? If so, any pointers on what else I should fix? E.g., take a pass at all the docs in this org and update them accordingly? Anything else?

[JoeyEremondi]

I think small, focused PRs are a lot more likely to get accepted, I
wouldn't mix code changes with doc changes.

On Sat, May 28, 2016 at 9:32 AM, Chad Woolley notifications@github.com
wrote:

@evancz https://github.com/evancz Would you be open to a Pull Request
for this? If so, any pointers on what else I should fix? E.g., take a pass
at all the docs in this org and update them accordingly? Anything else?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#217 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ACzn0sPPrd6o4p1auEdMwfjmpJDuvxWUks5qGG4egaJpZM4IoJ5I
.

[thewoolleyman]

Right, but I wouldn't want to make a change that leaves the docs (or tooling) inconsistent or wrong.

[JoeyEremondi]

Sorry, I misunderstood. Yeah, if it's updating the docs to match the new
feature, that's probably ok. Can't say what Evan will think of the feature
itself.

On Sat, May 28, 2016 at 9:49 AM, Chad Woolley notifications@github.com
wrote:

Right, but I wouldn't want to make a change that leaves the docs (or
tooling) inconsistent or wrong.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#217 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ACzn0kljP86OtxqdqjbRlIS687mQeZ_jks5qGHIYgaJpZM4IoJ5I
.

[bbugh]

I would love to see Elm get rid of a separation between dependencies and exact-dependencies. There isn't a world where I ever want to install a loosely versioned dependency, especially when sharing code with other developers. Too much can go wrong, and that's why we end up with shrinkwrap and Gemfile.lock. Let's just have the exact versions specified when we install them and save the extra effort.

[Warry]

You can do this in your .gitignore file:

elm-stuff/*
!elm-stuff/exact-dependencies.json

[bbugh]

@Warry that works in theory, but there's a bug/unintended side effect of elm-make that prevents that from working as expected. If you build a project with that and only commit elm-stuff/exact-dependencies.json, then try to clone and build it elsewhere, it fails, or elm-package says everything is installed correctly but when you try to use elm-make it's missing libraries. This is what the @thewoolleyman was describing. Here's a demonstration in a project using that suggestion:

# everything looks good!
» elm-package install         
Packages configured successfully!

# but wait, what's in the packages folder?
» ls elm-stuff/packages/elm-lang
core        virtual-dom

# it's missing multiple libraries...!
» cat elm-stuff/exact-dependencies.json
{
    "elm-lang/virtual-dom": "1.0.2",
    "evancz/elm-http": "3.0.1",
    "evancz/elm-markdown": "3.0.0",
    "elm-lang/html": "1.0.0",
    "elm-lang/core": "4.0.1"
}

# what happens if we try to build:
» elm-make   
elm-make: elm-stuff/packages/elm-lang/html/1.0.0/elm-package.json: openBinaryFile: does not exist 
(No such file or directory)

# try to install that package and it says everything is fine...
» elm-package install elm-lang/html
Packages configured successfully!

# but it fails...!
» elm-make
elm-make: elm-stuff/packages/elm-lang/html/1.0.0/elm-package.json: openBinaryFile: does not exist (No such file or directory)

Hopefully this makes it clearer why this issue is important.

[thewoolleyman]

@bbugh In theory that works, but in practice it means you have to manually specify and resolve your entire dependency tree (which is huge for big projects), when you only care about specifying your top-level dependencies.

I know this firsthand, because I wrote and maintained the popular predecessor to Bundler, and ran into this exact problem with it.

Then, Bundler came out and added the concept of the auto-generated Gemfile.lock which does the right thing - allows you to commit it and ensure your entire dependency tree is locked down, but avoids making you manually resolve and maintain it (ditto for npm-shrinkwrap.json).

So, exact-dependencies.json is the correct and analogous approach, but putting it in elm-stuff is inconsistent with other dependency tools, and causes problems. Moving it up solves all that (other than the unrelated bug that you also described, but that would also then not matter).

[thewoolleyman]

Looking into this further, it appears that fixing #249 would be a prerequisite for this change.

[evancz]

FYI for everyone, issues like this are data. I understand the root problem here. After 0.18, the big focus will be on package management stuff. The goal would be to address everyone's concerns in a comprehensive and coherent way. You can't do that in a one off way.

[evancz]

In 0.19, your project metadata goes in a file called elm.json. There are two scenarios:

• If you are creating an application, you will list all of your dependencies with exact versions, even transitive dependencies. The elm binary will help you set this up and make it nice to upgrade.

• If you are creating a package, you will list only direct dependencies with version ranges. This is basically the same as how things are now in elm-package.json.

That means that application developers no longer have a separate "lock file" that they must remember to check in. Checking in elm.json will fully record all of the transitive dependencies with exact versions.