Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make lock files future-proof #1061

Merged
merged 1 commit into from
Feb 21, 2016
Merged

Conversation

ferd
Copy link
Collaborator

@ferd ferd commented Feb 16, 2016

Changes to how hex or packages may work in the future will necessarily
bring changes to the format of lock files.

This commit adds an optional framing for future lock files of the form:

{Version, LockList}.
<Whatever consultable attributes>

This format is supported such as the LockList is the current lockfile
contents, and will never have more information than it currently does.

Attributes can be whatever and are currently undefined.

Rebar copies will be able to:

  • Keep using the core locklist (which avoids breaking the last year or
    so of community libraries using rebar3)
  • Warn when it runs an outdated copy in comparison to the lock file
  • Automatically rewrite lock files in the format it supports
  • Augment or parse files in a version-specific manner.

This changes the usage interface slightly, but is backwards and
forwards compatible.

If we wanted to add, for example, a list of source indexes for given
dependencies, we could have a lockfile of the form:

{"3.7.2", [
   {<<"pkg1">>,{pkg,<<"name1">>,<<"0.1.6">>},0},
   {<<"pkg2">>,{pkg,<<"name2">>,<<"0.1.6">>},1},
   {<<"pkg3">>,{pkg,<<"name3">>,<<"0.1.6">>},2}
]}.
{pkg_indexes, [{<<"pkg2">>, "http://example.org"}]}.

Currently it wouldn't be expected that plugins or users write
arbitrary data to the lock file.

A conversion layer is added when reading and writing from/to
disk such that while the snippet above represents the on-disk
format, Rebar3 could internally work with:

[{<<"pkg1">>,{pkg,<<"name1">>,<<"0.1.6">>},0},
 {<<"pkg2">>,{pkg,<<"name2">>,<<"0.1.6">>, "http://example.org"},1},
 {<<"pkg3">>,{pkg,<<"name3">>,<<"0.1.6">>},2}]

And just do what it needs. All we need to do is maintain a set of
functions to map from past versions to future versions,
and to read what we can afford to read from future versions,
losing whatever information we must to keep going (while
displaying a big warning)

Changes to how hex or packages may work in the future will necessarily
bring changes to the format of lock files.

This commit adds an optional framing for future lock files of the form:

    {Version, LockList}.
    <Whatever consultable attributes>

This format is supported such as the LockList is the current lockfile
contents, and will never have more information than it currently does.

Attributes can be whatever and are currently undefined.

Rebar copies will be able to:
- Keep using the core locklist (which avoids breaking the last year or
  so of community libraries using rebar3)
- Warn when it runs an outdated copy in comparison to the lock file
- Automatically rewrite lock files in the format it supports
- Augment or parse files in a version-specific manner.

This changes the usage interface slightly, but is backwards *and*
forwards compatible.
@ferd
Copy link
Collaborator Author

ferd commented Feb 16, 2016

Even if we do not need the feature, we should probably include something of this kind ASAP so that when we do change the lockfile format, all rebar versions in the wild are ready to handle the lock file format change transparently.

@ferd
Copy link
Collaborator Author

ferd commented Feb 16, 2016

I was wondering otherwise if it should instead go with:

{version, "A.B.C"}.
[{<<"pkg1">>,{pkg,<<"name1">>,<<"0.1.6">>},0},
 {<<"pkg2">>,{pkg,<<"name2">>,<<"0.1.6">>},1},
 {<<"pkg3">>,{pkg,<<"name3">>,<<"0.1.6">>},2}].
some_attribute.
{or, whatever}.

I have no strong feelings about any of these.

@tsloughter
Copy link
Collaborator

I'm fine with this. Even though I don't think we'll need it for supporting mulitple registries because I think we should simply follow how Maven handles them :). But always good to have versions like this for the unknown unknowns.

@tsloughter
Copy link
Collaborator

Merging this. We should prepare for a way to lock plugins with this, but that doesn't need to happen before 3.0 now :)

tsloughter added a commit that referenced this pull request Feb 21, 2016
@tsloughter tsloughter merged commit 862486c into erlang:master Feb 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants