Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Versioned shared dependencies #262

Open
wants to merge 36 commits into from

5 participants

@thijsterlouw

see issue #257 for a full explanation. We have tested this with Git and mainly with tags and master/head. It works great for this. Furthermore it works very nicely together with the solution for issue 258 (next pull request). It depends on symlinks (supported in Unix + Windows after Vista)

Major improvements:

  • allow different rebar projects to share the same dependencies, without running into version-mismatches
  • allow to set the shared deps dir via REBAR_SHARED_DEPS_DIR env var (useful for moving all deps to a central location in a large environment without touching all rebar.config files)
src/rebar_deps.erl
@@ -111,6 +113,17 @@ setup_env(_Config) ->
end,
[{"REBAR_DEPS_DIR", DepsDir}, ERL_LIBS].
+%% Set symlinks from DEPS dir to SHARED_DEPS dir
+%% This only works for Linux-based OS!
@Motiejus
Motiejus added a note

Unix

Mac OS, *BSD

true :-)

@tuncer
tuncer added a note

According to erts sources file:make_link/2 works on newer Windows versions.

make_link seems to make a hard link, which does not work for directories. I did change my function to use file:make_symlink/2 and handle enotsup gracefully.

see next commit :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@Motiejus

Thijs,

Would you mind doing one of those:

  1. merge master in
  2. rebase on top of basho/master (probably more preferable for rebar maintainer)

I am using this feature in our CI environment, but need some patches from upstream rebar, too. Merging is not trivial, since it yields quite a bit of merge conflicts which I am not confident resolving myself.

Thanks!

@Motiejus

FYI, here is the merged version (with upstream master):

https://github.com/Motiejus/rebar/tree/merged_with_ver_deps

thijsterlouw and others added some commits
@thijsterlouw thijsterlouw Use file:make_symlink/2 and handle errors correctly d80e620
@Motiejus Motiejus Merge remote-tracking branch 'remotes/spil/master' into versioned-sha…
…red-dependencies
e53b66c
@Motiejus Motiejus Merge commit 'a46038c9393d5d21db0c21856422d6a262833a4c' into shared_deps
Conflicts:
	Makefile
	src/rebar_utils.erl
3567ba0
@yfyf yfyf Fix shared dependency deletion.
When using the shared deps directory, the deps in an application are actually
symlinked to the shared directory. This means that when the deps are being
deleted, only the symlinks are being removed. This change fixes it by following
the symlink and actually removing the shared dependency.
01bb9f3
@yfyf yfyf Clear extra whitespace. bd3ab55
@yfyf yfyf Fix get_shared_deps behaviour, refactor related code.
* rebar_deps:get_shared_deps(D) now properly returns {true, _}/{false, _}
  tuples instead of always {true, _}
* furthermore, it constructs the correct full path to the shared directory
  (with the branch/rev/tag suffix)
55d9d63
@yfyf yfyf Make rebar_deps:parse_version/1 handle undefined src. bb238ac
@thijsterlouw thijsterlouw New: option to copy instead of symlink shared deps (REBAR_COPY_FROM_S…
…HARED_DEPS_DIR)
a5183c2
@thijsterlouw thijsterlouw Set REBAR_COPY_FROM_SHARED_DEPS_DIR=1 for easier disabling 8392b9e
@Motiejus

Just to let you know I ported this functionality to a more recent version of rebar (71c717d or 2.1.0-pre-42-g71c717d), here:
https://github.com/spilgames/rebar/tree/shared_deps

There are minor problems which I am aware of. For example, get_shared_deps_dir has ambiguous meaning in the current context, so I had to temporarily rename our get_shared_deps_dir to get_spil_shared_deps_dir, which is terrible. If you have a vision how this behavior could fit to current rebar, please let me know, so I can update all config names, rename functions and update other references.

We have been using this functionality for half a year now, and it works well. Especially on CI machines, where zero compilation time of dependencies is a major advantage (all our dependencies are tagged, so we don't run into HEAD problems). If you want I can open a new pull request to rebar/rebar, but please let me know I should rebase the commits on top of new master first (shared_deps mentioned above is simply merged). If yes, I would probably squash them to a single commit.

@jaredmorrow

@thijsterlouw ping. We abandoned basho/rebar when rebar/rebar was created. We (basho) are now back to using basho/rebar, is this PR something you'd still like to see? Has it already been merged into rebar/rebar?

@tuncer tuncer referenced this pull request in rebar/rebar
Open

Implement real version constraint (sat) solver #240

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 26, 2012
  1. @thijsterlouw
  2. @thijsterlouw

    Example shared_deps_dir

    thijsterlouw authored
Commits on Jun 27, 2012
  1. @thijsterlouw
  2. @thijsterlouw
  3. @thijsterlouw
  4. @thijsterlouw
  5. @thijsterlouw
  6. @thijsterlouw
  7. @thijsterlouw
  8. @thijsterlouw
  9. @thijsterlouw
  10. @thijsterlouw
Commits on Jun 28, 2012
  1. @thijsterlouw
  2. @thijsterlouw
Commits on Jul 3, 2012
  1. @thijsterlouw
  2. @thijsterlouw

    Remove TODO comment

    thijsterlouw authored
  3. @thijsterlouw
Commits on Jul 4, 2012
  1. @thijsterlouw
  2. @thijsterlouw
  3. @thijsterlouw
  4. @thijsterlouw
  5. @thijsterlouw
  6. @thijsterlouw

    Fix small dialyzer error

    thijsterlouw authored
Commits on Jul 6, 2012
  1. @thijsterlouw
  2. @thijsterlouw
  3. @thijsterlouw
  4. @thijsterlouw
Commits on Jul 24, 2012
  1. @thijsterlouw
Commits on Dec 11, 2012
  1. @Motiejus
  2. @Motiejus

    Merge commit 'a46038c9393d5d21db0c21856422d6a262833a4c' into shared_deps

    Motiejus authored
    Conflicts:
    	Makefile
    	src/rebar_utils.erl
  3. @yfyf @Motiejus

    Fix shared dependency deletion.

    yfyf authored Motiejus committed
    When using the shared deps directory, the deps in an application are actually
    symlinked to the shared directory. This means that when the deps are being
    deleted, only the symlinks are being removed. This change fixes it by following
    the symlink and actually removing the shared dependency.
  4. @yfyf @Motiejus

    Clear extra whitespace.

    yfyf authored Motiejus committed
  5. @yfyf @Motiejus

    Fix get_shared_deps behaviour, refactor related code.

    yfyf authored Motiejus committed
    * rebar_deps:get_shared_deps(D) now properly returns {true, _}/{false, _}
      tuples instead of always {true, _}
    * furthermore, it constructs the correct full path to the shared directory
      (with the branch/rev/tag suffix)
  6. @yfyf @Motiejus

    Make rebar_deps:parse_version/1 handle undefined src.

    yfyf authored Motiejus committed
  7. @thijsterlouw @Motiejus

    New: option to copy instead of symlink shared deps (REBAR_COPY_FROM_S…

    thijsterlouw authored Motiejus committed
    …HARED_DEPS_DIR)
  8. @thijsterlouw @Motiejus
This page is out of date. Refresh to see the latest.
View
3  Makefile
@@ -17,6 +17,9 @@ check: debug xref dialyzer deps test
xref:
@./rebar xref
+eunit: all
+ ./rebar -v eunit
+
dialyzer: dialyzer_warnings
@diff -U0 dialyzer_reference dialyzer_warnings
View
1  ebin/rebar.app
@@ -34,6 +34,7 @@
rebar_templater,
rebar_upgrade,
rebar_utils,
+ rebar_version,
rebar_xref,
getopt,
mustache ]},
View
6 rebar.config.sample
@@ -108,6 +108,12 @@
%% == Dependencies ==
+%% Where to put shared dependencies. By default it is not used
+%% Shared dependencies contain versioned directories and result
+%% in symlinks from the deps_dir to the shared_deps_dir.
+%% This allows you to share deps between several projects.
+{shared_deps_dir, "../deps"}.
+
%% Where to put any downloaded dependencies. Default is "deps"
{deps_dir, "deps"}.
View
16 src/rebar_app_utils.erl
@@ -32,6 +32,7 @@
app_name/1,
app_applications/1,
app_vsn/1,
+ app_vsn_reset/1,
is_skipped_app/1]).
-export([load_app_file/1]). % TEMPORARY
@@ -105,6 +106,16 @@ app_vsn(AppFile) ->
[AppFile, Reason])
end.
+app_vsn_reset(AppFile) ->
+ case reload_app_file(AppFile) of
+ {ok, _, AppInfo} ->
+ AppDir = filename:dirname(filename:dirname(AppFile)),
+ rebar_utils:vcs_vsn_delete(get_value(vsn, AppInfo, AppFile), AppDir);
+ {error, Reason} ->
+ ?ABORT("Failed to extract vsn from ~s: ~p\n",
+ [AppFile, Reason])
+ end.
+
is_skipped_app(AppFile) ->
ThisApp = app_name(AppFile),
%% Check for apps global parameter; this is a comma-delimited list
@@ -131,6 +142,11 @@ is_skipped_app(AppFile) ->
%% Internal functions
%% ===================================================================
+reload_app_file(Filename) ->
+ AppFile = {app_file, Filename},
+ erlang:put(AppFile, undefined),
+ load_app_file(Filename).
+
load_app_file(Filename) ->
AppFile = {app_file, Filename},
case erlang:get(AppFile) of
View
7 src/rebar_config.erl
@@ -28,7 +28,7 @@
-export([new/0, new/1, base_config/1, consult_file/1,
get/3, get_local/3, get_list/3,
- get_all/2,
+ get_all/2, get_dir/1,
set/3,
set_global/2, get_global/2,
is_verbose/0, get_jobs/0,
@@ -103,6 +103,11 @@ get_local(Config, Key, Default) ->
get_all(Config, Key) ->
proplists:get_all_values(Key, Config#config.opts).
+get_dir(#config{dir=Dir}) ->
+ Dir;
+get_dir(_) ->
+ undefined.
+
set(Config, Key, Value) ->
Opts = proplists:delete(Key, Config#config.opts),
Config#config { opts = [{Key, Value} | Opts] }.
View
310 src/rebar_deps.erl
@@ -26,6 +26,7 @@
%% -------------------------------------------------------------------
-module(rebar_deps).
+-include_lib("kernel/include/file.hrl").
-include("rebar.hrl").
-export([preprocess/2,
@@ -38,6 +39,8 @@
'delete-deps'/2,
'list-deps'/2]).
+%% For testing
+-export([get_value/2, set_value/2, del_key/1]).
-record(dep, { dir,
app,
@@ -52,6 +55,8 @@ preprocess(Config, _) ->
%% Side effect to set deps_dir globally for all dependencies from
%% top level down. Means the root deps_dir is honoured or the default
%% used globally since it will be set on the first time through here
+ %% We also set a shared_deps_dir. If set, we use this to download
+ %% dependencies and then symlink from deps_dir to shared_deps_dir
set_global_deps_dir(Config, rebar_config:get_global(deps_dir, [])),
%% Get the list of deps for the current working directory and identify those
@@ -111,6 +116,41 @@ setup_env(_Config) ->
end,
[{"REBAR_DEPS_DIR", DepsDir}, ERL_LIBS].
+%% Set symlinks from DEPS dir to SHARED_DEPS dir
+%% This works most Unix systems and Windows beginning with Vista
+%% We need to make sure the deps_dir actually exists before
+%% we can symlink to it
+'symlink-shared-deps-to-deps'(DownloadDir, TargetDir) ->
+ {true, DepsDir} = get_deps_dir(),
+ ok = filelib:ensure_dir(DepsDir ++ "/"),
+ LinkResult = file:make_symlink(DownloadDir, TargetDir),
+ case LinkResult of
+ {error, enotsup} ->
+ ?ABORT("Shared deps require OS support for symlinks.\n", []);
+ ok ->
+ ?DEBUG("Symlinked ~1000p to ~1000p\n", [DownloadDir, TargetDir]);
+ _ ->
+ ?ERROR("Error symlinking ~1000p to ~1000p : ~1000p\n", [DownloadDir, TargetDir, LinkResult])
+ end,
+ LinkResult.
+
+%% Copy instead of symlink from SHARED_DEPS to DEPS dir
+'copy-shared-deps-to-deps'(DownloadDir, TargetDir) ->
+ {true, DepsDir} = get_deps_dir(),
+ ok = filelib:ensure_dir(DepsDir ++ "/"),
+ Result = rebar_file_utils:cp_r([DownloadDir], TargetDir),
+ ?DEBUG("Copied ~1000p to ~1000p\n", [DownloadDir, TargetDir]),
+ Result.
+
+
+setup_deps_dir(SharedTargetDir, TargetDir) ->
+ case rebar_config:get_global(copy_from_shared_deps_dir, undefined) of
+ true ->
+ 'copy-shared-deps-to-deps'(SharedTargetDir, TargetDir);
+ _ -> %undefined | false
+ 'symlink-shared-deps-to-deps'(SharedTargetDir, TargetDir)
+ end.
+
'check-deps'(Config, _) ->
%% Get the list of immediate (i.e. non-transitive) deps that are missing
Deps = rebar_config:get_local(Config, deps, []),
@@ -127,6 +167,9 @@ setup_env(_Config) ->
end.
'get-deps'(Config, _) ->
+ %% Make Config available
+ set_value(config, Config),
+
%% Determine what deps are available and missing
Deps = rebar_config:get_local(Config, deps, []),
{_AvailableDeps, MissingDeps} = find_deps(find, Deps),
@@ -137,6 +180,9 @@ setup_env(_Config) ->
%% Add each pulled dep to our list of dirs for post-processing. This yields
%% the necessary transitivity of the deps
erlang:put(?MODULE, [D#dep.dir || D <- PulledDeps]),
+
+ del_key(config),
+
ok.
'update-deps'(Config, _) ->
@@ -177,9 +223,29 @@ setup_env(_Config) ->
%% Added because of trans deps,
%% need all deps in same dir and should be the one set by the root rebar.config
%% Sets a default if root config has no deps_dir set
+%% shared_deps_dir by default is undefined.
+%% By default it will use OS environment value REBAR_SHARED_DEPS_DIR if set.
set_global_deps_dir(Config, []) ->
rebar_config:set_global(deps_dir,
- rebar_config:get_local(Config, deps_dir, "deps"));
+ rebar_config:get_local(Config, deps_dir, "deps")),
+
+ EnvSharedDepsDir = case os:getenv("REBAR_SHARED_DEPS_DIR") of
+ false ->
+ undefined;
+ Dir ->
+ Dir
+ end,
+ rebar_config:set_global(shared_deps_dir,
+ rebar_config:get_local(Config, shared_deps_dir, EnvSharedDepsDir)),
+
+ EnvCopyFromSharedDepsDir = case os:getenv("REBAR_COPY_FROM_SHARED_DEPS_DIR") of
+ "1" -> true;
+ "true" -> true;
+ _ -> false
+ end,
+ rebar_config:set_global(copy_from_shared_deps_dir,
+ rebar_config:get_local(Config, copy_from_shared_deps_dir, EnvCopyFromSharedDepsDir));
+
set_global_deps_dir(_Config, _DepsDir) ->
ok.
@@ -191,6 +257,20 @@ get_deps_dir(App) ->
DepsDir = rebar_config:get_global(deps_dir, "deps"),
{true, filename:join([BaseDir, DepsDir, App])}.
+get_shared_deps_dir(Dep) ->
+ BaseDir = rebar_config:get_global(base_dir, []),
+ SharedDepsDir = rebar_config:get_global(shared_deps_dir, undefined),
+ case SharedDepsDir of
+ undefined ->
+ {false, undefined};
+ _ ->
+ Version = parse_version(Dep#dep.source),
+ UnversionedAppDir = filename:join(
+ [BaseDir, SharedDepsDir, Dep#dep.app]),
+ VersionedAppDir = get_download_dir(UnversionedAppDir, Version),
+ {true, VersionedAppDir}
+ end.
+
get_lib_dir(App) ->
%% Find App amongst the reachable lib directories
%% Returns either the found path or a tagged tuple with a boolean
@@ -278,6 +358,13 @@ acc_deps(read, _, Dep, AppDir, Acc) ->
delete_dep(D) ->
case filelib:is_dir(D#dep.dir) of
true ->
+ ok = case get_shared_deps_dir(D) of
+ {true, SharedDepDir} ->
+ ?INFO("Deleting shared dependency: ~s\n", [SharedDepDir]),
+ rebar_file_utils:rm_rf(SharedDepDir);
+ {false, _} ->
+ ok
+ end,
?INFO("Deleting dependency: ~s\n", [D#dep.dir]),
rebar_file_utils:rm_rf(D#dep.dir);
false ->
@@ -288,24 +375,36 @@ require_source_engine(Source) ->
true = source_engine_avail(Source),
ok.
-is_app_available(App, VsnRegex, Path) ->
+is_app_available(App, VsnCheck, Path) ->
?DEBUG("is_app_available, looking for App ~p with Path ~p~n", [App, Path]),
case rebar_app_utils:is_app_dir(Path) of
{true, AppFile} ->
case rebar_app_utils:app_name(AppFile) of
App ->
Vsn = rebar_app_utils:app_vsn(AppFile),
- ?INFO("Looking for ~s-~s ; found ~s-~s at ~s\n",
- [App, VsnRegex, App, Vsn, Path]),
- case re:run(Vsn, VsnRegex, [{capture, none}]) of
- match ->
+ ?INFO("Looking for ~s ~1000p ; found ~s-~s at ~s\n",
+ [App, VsnCheck, App, Vsn, Path]),
+ case rebar_version:check(Vsn, VsnCheck) of
+ true ->
+ memoize_dependency(App, VsnCheck),
{true, Path};
- nomatch ->
- ?WARN("~s has version ~p; requested regex was ~s\n",
- [AppFile, Vsn, VsnRegex]),
- {false, {version_mismatch,
- {AppFile,
- {expected, VsnRegex}, {has, Vsn}}}}
+ _ ->
+ ?WARN("~s has version ~p; requested was ~1000p\n",
+ [AppFile, Vsn, VsnCheck]),
+ case can_resolve_dependency(App, VsnCheck) of
+ true ->
+ %% We need to clear the cache of the app_vsn
+ %% Because in a next round we probably have
+ %% a new version available in the directory
+ rebar_app_utils:app_vsn_reset(AppFile),
+ {false, resolvable_version_mismatch};
+ {false, Reason} ->
+ {false, {version_mismatch,
+ {AppFile,
+ {wanted, VsnCheck},
+ {has, Vsn},
+ Reason}}}
+ end
end;
OtherApp ->
?WARN("~s has application id ~p; expected ~p\n",
@@ -319,39 +418,132 @@ is_app_available(App, VsnRegex, Path) ->
{false, {missing_app_file, Path}}
end.
+
use_source(Dep) ->
- use_source(Dep, 3).
+ use_source(Dep, 3, false).
-use_source(Dep, 0) ->
+use_source(Dep, 0, _) ->
?ABORT("Failed to acquire source from ~p after 3 tries.\n",
[Dep#dep.source]);
-use_source(Dep, Count) ->
- case filelib:is_dir(Dep#dep.dir) of
+use_source(Dep, Count, Force) ->
+ case Force of
true ->
- %% Already downloaded -- verify the versioning matches the regex
- case is_app_available(Dep#dep.app,
- Dep#dep.vsn_regex, Dep#dep.dir) of
- {true, _} ->
- Dir = filename:join(Dep#dep.dir, "ebin"),
- ok = filelib:ensure_dir(filename:join(Dir, "dummy")),
- %% Available version matches up -- we're good to go;
- %% add the app dir to our code path
- true = code:add_patha(Dir),
- Dep;
- {false, Reason} ->
- %% The app that was downloaded doesn't match up (or had
- %% errors or something). For the time being, abort.
- ?ABORT("Dependency dir ~s failed application validation "
- "with reason:~n~p.\n", [Dep#dep.dir, Reason])
- end;
- false ->
- ?CONSOLE("Pulling ~p from ~p\n", [Dep#dep.app, Dep#dep.source]),
- require_source_engine(Dep#dep.source),
- {true, TargetDir} = get_deps_dir(Dep#dep.app),
- download_source(TargetDir, Dep#dep.source),
- use_source(Dep#dep { dir = TargetDir }, Count-1)
+ %% When forcing the retrieve, we need to make sure the directory
+ %% does not already exist
+ case filelib:is_dir(Dep#dep.dir) of
+ true ->
+ rebar_file_utils:rm_rf(Dep#dep.dir);
+ false ->
+ ok
+ end,
+ retrieve_source_and_retry(Dep, Count, Force);
+ _ ->
+ case filelib:is_dir(Dep#dep.dir) of
+ true ->
+ %% Already downloaded -- verify the versioning matches the regex
+ case is_app_available(Dep#dep.app,
+ Dep#dep.vsn_regex, Dep#dep.dir) of
+ {false, resolvable_version_mismatch} ->
+ ?WARN("Dependency dir ~s failed version-check, but able to resolve.\n",
+ [Dep#dep.dir]),
+ use_source(Dep, Count, true);
+ {true, _} ->
+ Dir = filename:join(Dep#dep.dir, "ebin"),
+ ok = filelib:ensure_dir(filename:join(Dir, "dummy")),
+ %% Available version matches up -- we're good to go;
+ %% add the app dir to our code path
+ true = code:add_patha(Dir),
+ Dep;
+ {false, Reason} ->
+ %% The app that was downloaded doesn't match up (or had
+ %% errors or something).
+ ?WARN("Dependency dir ~s failed application validation "
+ "with reason:~n~p.Will retry by removing previous deps dir.\n", [Dep#dep.dir, Reason]),
+ rebar_file_utils:rm_rf(Dep#dep.dir),
+ use_source(Dep, Count-1, true)
+ end;
+ false ->
+ retrieve_source_and_retry(Dep, Count, Force)
+ end
end.
+
+retrieve_source_and_retry(Dep, Count, Force) ->
+ %% The shared deps dir might already exist, in that case we only
+ %% need to symlink. So construct the download dir and check if it
+ %% already exists.
+ {true, TargetDir} = get_deps_dir(Dep#dep.app),
+ case get_shared_deps_dir(Dep) of
+ {false, _} ->
+ ok;
+ {true, SharedTargetDir} ->
+ %% If the (possibly versioned) downloads dir already exists, just
+ %% skip downloading the source
+ case filelib:is_dir(SharedTargetDir) of
+ false ->
+ download_dep(Dep, SharedTargetDir);
+ true ->
+ case Force of
+ true ->
+ rebar_file_utils:rm_rf(SharedTargetDir),
+ download_dep(Dep, SharedTargetDir);
+ false ->
+ ok
+ end
+ end,
+ rebar_file_utils:rm_rf(TargetDir),
+ setup_deps_dir(SharedTargetDir, TargetDir)
+ end,
+
+ case rebar_app_utils:is_app_dir(Dep#dep.dir) of
+ {true, AppFile} ->
+ rebar_app_utils:app_vsn_reset(AppFile);
+ _ ->
+ ?WARN("Failed to reload .app file for ~p.\n", [Dep#dep.app])
+ end,
+
+ use_source(Dep#dep { dir = TargetDir }, Count-1, false).
+
+%% Helper, downloads dependency from source
+download_dep(Dep, Destination) ->
+ ?CONSOLE("Pulling ~p from ~p\n", [Dep#dep.app, Dep#dep.source]),
+ require_source_engine(Dep#dep.source),
+ download_source(Destination, Dep#dep.source).
+
+%% Helper creates a versioned download
+get_download_dir(BaseAppDir, {branch, "HEAD"}) ->
+ BaseAppDir;
+get_download_dir(BaseAppDir, {branch, Branch}) ->
+ BaseAppDir ++ "-branch-" ++ Branch;
+get_download_dir(BaseAppDir, {tag, Tag}) ->
+ BaseAppDir ++ "-tag-" ++ Tag;
+get_download_dir(BaseAppDir, {rev, Rev}) ->
+ BaseAppDir ++ "-rev-" ++ Rev;
+get_download_dir(BaseAppDir, _) ->
+ BaseAppDir.
+
+%% Parse the version to a uniform format
+parse_version({hg, _, Rev}) ->
+ {rev, Rev};
+parse_version({git, Url}) ->
+ parse_version({git, Url, {branch, "HEAD"}});
+parse_version({git, Url, ""}) ->
+ parse_version({git, Url, {branch, "HEAD"}});
+parse_version({git, _, {branch, Branch}}) ->
+ {branch, Branch};
+parse_version({git, _, {tag, Tag}}) ->
+ {tag, Tag};
+parse_version({git, _, Rev}) ->
+ {rev, Rev};
+parse_version({bzr, _, Rev}) ->
+ {rev, Rev};
+parse_version({svn, _, Rev}) ->
+ {rev, Rev};
+parse_version(undefined) ->
+ undefined.
+
+
+%% Downloads the source and returns the directory containing the source.
download_source(AppDir, {hg, Url, Rev}) ->
ok = filelib:ensure_dir(AppDir),
rebar_utils:sh(?FMT("hg clone -U ~s ~s", [Url, filename:basename(AppDir)]),
@@ -434,7 +626,49 @@ update_source(AppDir, {rsync, Url}) ->
rebar_utils:sh(?FMT("rsync -az --delete ~s/ ~s",[Url,AppDir]),[]).
+%% Remember downloaded dependencies
+memoize_dependency(App, VsnCheck) ->
+ PreviousAppVersionRestrictions = get_value({dep,App}, []),
+ Config = get_value(config, undefined),
+ Dir = rebar_config:get_dir(Config),
+ NewAppVersionRestrictions = [{Dir, VsnCheck}] ++ PreviousAppVersionRestrictions,
+ set_value({dep,App}, NewAppVersionRestrictions).
+
+%% Check if we can resolve this new dependency
+can_resolve_dependency(App, VsnCheck) ->
+ AppDepConstraints = get_value({dep,App}, []),
+ VsnConstraints = [ Constraint || {_, Constraint} <- AppDepConstraints],
+ Res = check_dependencies(VsnCheck, VsnConstraints, true),
+ case Res of
+ false ->
+ {false, {reason, {cannot_satisfy, AppDepConstraints}}};
+ true ->
+ true
+ end.
+check_dependencies(_, _, false) ->
+ false;
+check_dependencies(_, [], Res) ->
+ Res;
+check_dependencies(Vsn, [H|T], _) ->
+ check_dependencies(Vsn, T, rebar_version:check(Vsn, H)).
+
+%% Remember some values using rebar_config.
+%% TODO: there should be a generic KV storage system
+get_value(Key, Default) ->
+ Config = rebar_config:get_global(rebar_deps_config, []),
+ proplists:get_value(Key, Config, Default).
+
+set_value(Key, Value) ->
+ Config = rebar_config:get_global(rebar_deps_config, []),
+ TempConfig = proplists:delete(Key, Config),
+ NewConfig = [{Key, Value}] ++ TempConfig,
+ rebar_config:set_global(rebar_deps_config, NewConfig).
+
+del_key(Key) ->
+ Config = rebar_config:get_global(rebar_deps_config, []),
+ NewConfig = proplists:delete(Key, Config),
+ rebar_config:set_global(rebar_deps_config, NewConfig).
%% ===================================================================
%% Source helper functions
View
6 src/rebar_utils.erl
@@ -43,6 +43,7 @@
expand_code_path/0,
expand_env_variable/3,
vcs_vsn/2,
+ vcs_vsn_delete/2,
deprecated/3, deprecated/4,
get_deprecated_global/3, get_deprecated_global/4,
get_deprecated_list/4, get_deprecated_list/5,
@@ -208,6 +209,11 @@ vcs_vsn(Vcs, Dir) ->
VsnString
end.
+vcs_vsn_delete(Vcs, Dir) ->
+ Key = {Vcs, Dir},
+ ets:delete(rebar_vsn_cache, Key).
+
+
get_deprecated_global(OldOpt, NewOpt, When) ->
get_deprecated_global(OldOpt, NewOpt, undefined, When).
View
183 src/rebar_version.erl
@@ -0,0 +1,183 @@
+%% -*- erlang-indent-level: 4;indent-tabs-mode: nil -*-
+%% ex: ts=4 sw=4 et
+%% -------------------------------------------------------------------
+%%
+%% rebar: Erlang Build Tools
+%%
+%% Copyright (c) 2012 Thijs Terlouw (Thijs.Terlouw@spilgames.com)
+%%
+%% Permission is hereby granted, free of charge, to any person obtaining a copy
+%% of this software and associated documentation files (the "Software"), to deal
+%% in the Software without restriction, including without limitation the rights
+%% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+%% copies of the Software, and to permit persons to whom the Software is
+%% furnished to do so, subject to the following conditions:
+%%
+%% The above copyright notice and this permission notice shall be included in
+%% all copies or substantial portions of the Software.
+%%
+%% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+%% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+%% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+%% AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+%% LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+%% OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+%% THE SOFTWARE.
+%% -------------------------------------------------------------------
+%% Partial implementation of Semantic Versioning (see http://semver.org/ )
+%% Currently only supports X.Y.Z where X, Y and Z are integers
+%% -------------------------------------------------------------------
+-module(rebar_version).
+
+-include("rebar.hrl").
+
+-export([check/2]).
+
+%% For testing:
+-export([compare/2, check_constraints/2]).
+
+-type version() :: {non_neg_integer(), non_neg_integer(), non_neg_integer()}. % {Major, Minor, Patch}
+-type comparator() :: string(). % ">" | ">=" | "=" | "<=" | "<" .
+-type constraint() :: {comparator(), version()}.
+-type constraints() :: [constraint()].
+
+
+%% ====================================================================
+%% Public API
+%% ====================================================================
+
+%%--------------------------------------------------------------------
+%% @doc
+%% New detailed version support
+%% example: [ ">= 1.0.0", "< 2.0.0" ]
+-spec check(string(), [string()] | string()) -> boolean().
+%% @end
+%%--------------------------------------------------------------------
+check(Vsn, [Constraint|_] = VsnConstraints) when not is_integer(Constraint) andalso length(Constraint) > 1 ->
+ case check_constraints(Vsn, VsnConstraints) of
+ true ->
+ true;
+ _ ->
+ false
+ end;
+%% Old version support with regex
+%% example: "0.5.*"
+check(Vsn, VsnRegex) ->
+ case re:run(Vsn, VsnRegex, [{capture, none}]) of
+ match ->
+ true;
+ nomatch ->
+ false
+ end.
+
+
+%% ====================================================================
+%% Internal functions
+%% ====================================================================
+
+%%--------------------------------------------------------------------
+%% @doc
+%% Check if all constraints are satisfied.
+-spec check_constraints(string(), [string()]) -> boolean().
+%% @end
+%%--------------------------------------------------------------------
+check_constraints(Vsn, Constraints) ->
+ ParsedVersion = parse_version(Vsn),
+ ParsedConstraints = parse_constraints(Constraints),
+ verify_constraints(ParsedVersion, ParsedConstraints, true).
+
+verify_constraints(_, _, false) ->
+ false;
+verify_constraints(_, [], Result) ->
+ Result;
+verify_constraints(Vsn, [{Comparator, ConstraintVsn}|Tail], _) ->
+ Comparison = compare(Vsn, ConstraintVsn),
+ Result = case Comparator of
+ ">" -> Comparison > 0;
+ ">=" -> Comparison >= 0;
+ "=" -> Comparison =:= 0;
+ "<=" -> Comparison =< 0;
+ "<" -> Comparison < 0
+ end,
+ verify_constraints(Vsn, Tail, Result).
+
+
+%%--------------------------------------------------------------------
+%% @doc
+%% Parsing each constraint into a comparator and a version.
+-spec parse_constraints([string()]) -> constraints().
+%% @end
+%%--------------------------------------------------------------------
+parse_constraints(VsnConstraints) ->
+ parse_constraints(VsnConstraints, []).
+
+parse_constraints([], Acc) ->
+ Acc;
+parse_constraints([H|T], Acc) ->
+ parse_constraints(T, [ parse_constraint(H) | Acc]).
+
+parse_constraint(Constraint) ->
+ case string:tokens(Constraint, " .") of
+ [Comparator, Major, Minor, Patch | _] ->
+ {Comparator, {list_to_integer(Major),
+ list_to_integer(Minor),
+ list_to_integer(Patch)
+ }};
+ _ ->
+ ?ABORT("Invalid version constraint : ~p .\n", [Constraint])
+ end.
+
+
+%%--------------------------------------------------------------------
+%% @doc
+%% Parsing the version (only allows X.Y.Z format)
+-spec parse_version(string()) -> version().
+%% @end
+%%--------------------------------------------------------------------
+parse_version(Vsn) ->
+ case string:tokens(Vsn, ".") of
+ [Major, Minor, Patch | _] ->
+ {list_to_integer(Major),
+ list_to_integer(Minor),
+ list_to_integer(Patch)
+ };
+ _ ->
+ ?ABORT("Invalid version for comparison with constraints : ~p .\n", [Vsn])
+ end.
+
+
+%%--------------------------------------------------------------------
+%% @doc
+%% Compare two versions,
+%% returns: 0 if equal
+%% -1 if left is smaller
+%% 1 if left is bigger
+-spec compare(version(), version()) -> -1 | 0 | 1.
+%% @end
+%%--------------------------------------------------------------------
+compare({MajorA,MinorA,PatchA}, {MajorB,MinorB,PatchB}) ->
+ if
+ MajorA =:= MajorB andalso MinorA =:= MinorB andalso PatchA =:= PatchB ->
+ 0;
+ true->
+ if
+ MajorA < MajorB ->
+ -1;
+ MajorA =:= MajorB ->
+ if
+ MinorA < MinorB ->
+ -1;
+ MinorA =:= MinorB ->
+ if
+ PatchA < PatchB ->
+ -1;
+ true ->
+ 1
+ end;
+ true ->
+ 1
+ end;
+ true ->
+ 1
+ end
+ end.
View
55 test/rebar_deps_eunit.erl
@@ -0,0 +1,55 @@
+%% -*- erlang-indent-level: 4;indent-tabs-mode: nil -*-
+%% ex: ts=4 sw=4 et
+%% -------------------------------------------------------------------
+%%
+%% rebar: Erlang Build Tools
+%%
+%% Copyright (c) 2012 Thijs Terlouw (Thijs.Terlouw@spilgames.com)
+%%
+%% Permission is hereby granted, free of charge, to any person obtaining a copy
+%% of this software and associated documentation files (the "Software"), to deal
+%% in the Software without restriction, including without limitation the rights
+%% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+%% copies of the Software, and to permit persons to whom the Software is
+%% furnished to do so, subject to the following conditions:
+%%
+%% The above copyright notice and this permission notice shall be included in
+%% all copies or substantial portions of the Software.
+%%
+%% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+%% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+%% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+%% AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+%% LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+%% OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+%% THE SOFTWARE.
+%% -------------------------------------------------------------------
+%% @author Thijs Terlouw <thijsterlouw@spilgames.com>
+%% @doc This tests functionality in the rebar_deps.
+%% -------------------------------------------------------------------
+-module(rebar_deps_eunit).
+
+-compile(export_all).
+
+-include_lib("eunit/include/eunit.hrl").
+
+kv_test_() ->
+ [
+ ?_assertEqual(default, rebar_deps:get_value(a, default)),
+ ?_assertEqual(ok, rebar_deps:set_value(a, b)),
+ ?_assertEqual(b, rebar_deps:get_value(a, default)),
+ ?_assertEqual(ok, rebar_deps:del_key(a)),
+ ?_assertEqual(default, rebar_deps:get_value(a, default))
+ ].
+
+kv_complex_key_test_() ->
+ Key1 = {a,1},
+ Key2 = {a,2},
+ [
+ ?_assertEqual(default, rebar_deps:get_value(Key1, default)),
+ ?_assertEqual(ok, rebar_deps:set_value(Key1, [{a,1}])),
+ ?_assertEqual(ok, rebar_deps:set_value(Key2, [{a,2}])),
+ ?_assertEqual([{a,1}], rebar_deps:get_value(Key1, default)),
+ ?_assertEqual(ok, rebar_deps:del_key(Key1)),
+ ?_assertEqual([{a,2}], rebar_deps:get_value(Key2, default))
+ ].
View
66 test/rebar_version_eunit.erl
@@ -0,0 +1,66 @@
+%% -*- erlang-indent-level: 4;indent-tabs-mode: nil -*-
+%% ex: ts=4 sw=4 et
+%% -------------------------------------------------------------------
+%%
+%% rebar: Erlang Build Tools
+%%
+%% Copyright (c) 2012 Thijs Terlouw (Thijs.Terlouw@spilgames.com)
+%%
+%% Permission is hereby granted, free of charge, to any person obtaining a copy
+%% of this software and associated documentation files (the "Software"), to deal
+%% in the Software without restriction, including without limitation the rights
+%% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+%% copies of the Software, and to permit persons to whom the Software is
+%% furnished to do so, subject to the following conditions:
+%%
+%% The above copyright notice and this permission notice shall be included in
+%% all copies or substantial portions of the Software.
+%%
+%% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+%% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+%% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+%% AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+%% LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+%% OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+%% THE SOFTWARE.
+%% -------------------------------------------------------------------
+%% @author Thijs Terlouw <thijsterlouw@spilgames.com>
+%% @doc This tests functionality in the rebar_version.
+%% -------------------------------------------------------------------
+-module(rebar_version_eunit).
+
+-compile(export_all).
+
+-include_lib("eunit/include/eunit.hrl").
+
+compare_test_() ->
+ [
+ ?_assertEqual(0, rebar_version:compare({1,2,3}, {1,2,3})),
+
+ ?_assertEqual(-1, rebar_version:compare({1,0,0}, {1,0,1})),
+ ?_assertEqual(-1, rebar_version:compare({1,0,0}, {1,1,0})),
+ ?_assertEqual(-1, rebar_version:compare({1,0,0}, {2,0,0})),
+
+ ?_assertEqual(1, rebar_version:compare({2,0,0}, {1,0,0})),
+ ?_assertEqual(1, rebar_version:compare({2,0,0}, {1,1,0})),
+ ?_assertEqual(1, rebar_version:compare({2,0,0}, {1,0,1}))
+ ].
+
+check_single_constraints_test_() ->
+ [
+ ?_assertEqual(true, rebar_version:check_constraints("1.2.3", [">= 1.2.3"])),
+ ?_assertEqual(true, rebar_version:check_constraints("1.2.3", ["> 1.2.2"])),
+ ?_assertEqual(true, rebar_version:check_constraints("1.2.3", ["= 1.2.3"])),
+ ?_assertEqual(true, rebar_version:check_constraints("1.2.3", ["<= 1.2.3"])),
+ ?_assertEqual(true, rebar_version:check_constraints("1.2.3", ["< 1.2.4"])),
+
+ ?_assertEqual(false, rebar_version:check_constraints("1.2.3", ["< 1.2.3"])),
+ ?_assertEqual(false, rebar_version:check_constraints("1.3.0", ["< 1.2.3"])),
+ ?_assertEqual(false, rebar_version:check_constraints("2.0.0", ["< 1.2.3"]))
+ ].
+
+check_multiple_constraints_test_() ->
+ [
+ ?_assertEqual(true, rebar_version:check_constraints("1.2.3", [">= 1.0.0", "<= 2.0.0"])),
+ ?_assertEqual(false, rebar_version:check_constraints("1.2.3", [">= 1.0.0", "< 1.2.0"]))
+ ].
Something went wrong with that request. Please try again.