Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profiles #31

Merged
merged 18 commits into from
Dec 2, 2014
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ And then we can implement the switch to figure out what to search:
do(State) ->
Apps = case discovery_type(State) of
project -> rebar_state:project_apps(State);
deps -> rebar_state:project_apps(State) ++ rebar_state:src_deps(State)
deps -> rebar_state:project_apps(State) ++ rebar_state:all_deps(State)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing why it's being done that way, but do the binary packages ship with the source code? Otherwise the demo app doing a search over source might look funny. Also this should require a patch on https://bitbucket.org/ferd/rebar3-todo-plugin to match the tutorial.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They do ship with source code. And yea, I'd have sent a PR if it was on github :P

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we should just take a note when merging it so that I can go and update it.

end,
lists:foreach(fun check_todo_app/1, Apps),
{ok, State}.
Expand Down
2 changes: 1 addition & 1 deletion priv/templates/release.template
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
{template, "app.erl.dtl", "{{name}}/{{apps_dir}}/{{name}}/src/{{name}}_app.erl"}.
{template, "sup.erl.dtl", "{{name}}/{{apps_dir}}/{{name}}/src/{{name}}_sup.erl"}.
{template, "otp_app.app.src.dtl", "{{name}}/{{apps_dir}}/{{name}}/src/{{name}}.app.src"}.
{template, "rebar.config.dtl", "{{name}}/rebar.config"}.
{template, "relx_rebar.config.dtl", "{{name}}/rebar.config"}.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this change for, and should it be adopted at a wider scale? Is this going to be breaking some compatibility, or does it require you to have a specific relx version installed?

If there's this config file, is there a reason to still have relx.config?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I didn't, I meant to remove relx.config. This works only for relx within rebar3. If you want to run your own version of relx you have to create the relx.config.

But without this you can't use profiles to build a dev vs prod release.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens to an existing project that swaps in rebar3 in lieu of rebar and used relx? It keeps working or relx-within-rebar3 starts breaking with this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeps working. This has worked for a long time just hasn't been the template. rebar3 checks if relx is defined in the config and passes that to relx if so. If it isn't then relx will look for relx.config because it doesn't get passed a config.

{template, "relx.config.dtl", "{{name}}/relx.config"}.
{template, "sys.config.dtl", "{{name}}/config/sys.config"}.
{template, "vm.args.dtl", "{{name}}/config/vm.args"}.
Expand Down
12 changes: 0 additions & 12 deletions priv/templates/relx.config.dtl

This file was deleted.

20 changes: 20 additions & 0 deletions priv/templates/relx_rebar.config.dtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{erl_opts, [debug_info]}.
{deps, []}.

{relx, [{release, {'{{name}}', "0.1.0"},
[{{name}},
sasl]},

{sys_config, "./config/sys.config"},
{vm_args, "./config/vm.args"},

{dev_mode, true},
{include_erts, false},

{extended_start_script, true}]
}.

{profiles, [{prod, [{relx, [{dev_mode, true},
{include_erts, false},]}]
}]
}.
11 changes: 1 addition & 10 deletions rebar.config
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,13 @@
debug_info,
warnings_as_errors]}.

%% Types dict:dict() and digraph:digraph() have been introduced in Erlang 17.
%% At the same time, their counterparts dict() and digraph() are to be
%% deprecated in Erlang 18. namespaced_types option is used to select proper
%% type name depending on the OTP version used.
{erl_opts,
[
{platform_define, "^[0-9]+", namespaced_types}
]}.

{deps, [
{erlware_commons, ".*",
{git, "https://github.com/erlware/erlware_commons.git",
{branch, "master"}}},
{providers, "",
{git, "https://github.com/tsloughter/providers.git",
{branch, "format_error1"}}},
{branch, "master"}}},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we depend on a tag or something?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I'll change it to a tag soon.

{erlydtl, ".*",
{git, "https://github.com/erlydtl/erlydtl.git",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So where does that leave us for backwards compat?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same. That platform_define is still in erl_opts above that line.

{tag, "0.9.4"}}},
Expand Down
3 changes: 1 addition & 2 deletions src/rebar.app.src
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@
rebar_prv_release,
rebar_prv_version,
rebar_prv_common_test,
rebar_prv_help,
rebar_prv_test_deps]}
rebar_prv_help]}
]}
]}.
10 changes: 6 additions & 4 deletions src/rebar.hrl
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@

-define(FMT(Str, Args), lists:flatten(io_lib:format(Str, Args))).

-define(DEFAULT_BASE_DIR, "").
-define(DEFAULT_BASE_DIR, "_build").
-define(DEFAULT_PROFILE_DIR, "default").
-define(DEFAULT_LIB_DIRS, ["_checkouts", "apps", "lib", "."]).
-define(DEFAULT_DEPS_DIR, "_deps").
-define(DEFAULT_PLUGINS_DIR, "_plugins").
-define(DEFAULT_TEST_DEPS_DIR, "_tdeps").
-define(DEFAULT_DEPS_DIR, "lib").
-define(DEFAULT_PLUGINS_DIR, "plugins").
-define(DEFAULT_TEST_DEPS_DIR, "test/lib").
-define(DEFAULT_RELEASE_DIR, "rel").
-define(DEFAULT_CONFIG_FILE, "rebar.config").
-define(LOCK_FILE, "rebar.lock").
-define(CONFIG_DIR, ".rebar3").
Expand Down
43 changes: 25 additions & 18 deletions src/rebar3.erl
Original file line number Diff line number Diff line change
Expand Up @@ -74,15 +74,15 @@ main(Args) ->
run(BaseState, Command) ->
_ = application:load(rebar),
BaseState1 = rebar_state:set(BaseState, task, Command),
run_aux(BaseState1, [Command]).
run_aux(BaseState1, [], [Command]).

%% ====================================================================
%% Internal functions
%% ====================================================================

run(RawArgs) ->
_ = application:load(rebar),
BaseConfig = init_config(),
{GlobalPluginProviders, BaseConfig} = init_config(),

case erlang:system_info(version) of
"6.1" ->
Expand All @@ -93,9 +93,9 @@ run(RawArgs) ->
end,

{BaseConfig1, _Args1} = set_options(BaseConfig, {[], []}),
run_aux(BaseConfig1, RawArgs).
run_aux(BaseConfig1, GlobalPluginProviders, RawArgs).

run_aux(State, RawArgs) ->
run_aux(State, GlobalPluginProviders, RawArgs) ->
%% Make sure crypto is running
case crypto:start() of
ok -> ok;
Expand All @@ -106,23 +106,28 @@ run_aux(State, RawArgs) ->
application:start(ssl),
inets:start(),

%% Process each command, resetting any state between each one
State2 = case rebar_state:get(State, base_dir, undefined) of
undefined ->
rebar_state:set(State, base_dir, filename:absname(rebar_state:dir(State)));
Dir ->
rebar_state:set(State, base_dir, filename:absname(Dir))
State2 = case os:getenv("REBAR_DEFAULT_PROFILE") of
false ->
State;
Profile ->
State1 = rebar_state:current_profile(State, list_to_atom(Profile)),
rebar_state:default(State1, rebar_state:opts(State1))
end,

{ok, Providers} = application:get_env(rebar, providers),
%% Process each command, resetting any state between each one
BaseDir = rebar_utils:base_dir(State2),
State3 = rebar_state:set(State2, base_dir,
filename:join(filename:absname(rebar_state:dir(State2)), BaseDir)),

{ok, PluginProviders, State3} = rebar_plugins:install(State2),
rebar_core:update_code_path(State3),
{ok, Providers} = application:get_env(rebar, providers),
{ok, PluginProviders, State4} = rebar_plugins:install(State3),
rebar_core:update_code_path(State4),

State4 = rebar_state:create_logic_providers(Providers++PluginProviders, State3),
AllProviders = Providers++PluginProviders++GlobalPluginProviders,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing this new difference between 'Providers', 'PluginProviders', and 'GlobalPluginProviders' means:

  • built-in providers
  • current profile plugin providers
  • all profile plugin providers

Am I right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost right. GlobalPluginProviders is plugins defined in your $HOME/.rebar3/config config file.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm eager to see how that one will work out. My fear is always that some people will start pushing apps that rely on plugins they didn't package in and stuff breaks near-randomly, or replace proper building by ad-hoc "you have to have plugin <plugin> v.a.b.c. for this to compile". There was some kind of cool benefit to forcing people to make plugins per-repo, though it makes for a crappier workflow.

I guess it's gonna be a question of discipline so that people who require plugins to build apps do ship them with them, and the plugins about workflow and analytics or whatever can be more global.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I agreed and had planned to always have them local until I made the auto compile plugin.

State5 = rebar_state:create_logic_providers(AllProviders, State4),
{Task, Args} = parse_args(RawArgs),

rebar_core:process_command(rebar_state:command_args(State4, Args), list_to_atom(Task)).
rebar_core:process_command(rebar_state:command_args(State5, Args), list_to_atom(Task)).

init_config() ->
%% Initialize logging system
Expand Down Expand Up @@ -150,9 +155,11 @@ init_config() ->
true ->
?DEBUG("Load global config file ~p",
[GlobalConfigFile]),
GlobalConfig = rebar_state:new(rebar_config:consult_file(GlobalConfigFile)),
rebar_state:new(GlobalConfig, Config1);
GlobalConfig = rebar_state:new(global, rebar_config:consult_file(GlobalConfigFile)),
{ok, PluginProviders, GlobalConfig1} = rebar_plugins:install(GlobalConfig),
rebar_state:new(GlobalConfig1, Config1);
false ->
PluginProviders = [],
rebar_state:new(Config1)
end,

Expand All @@ -168,7 +175,7 @@ init_config() ->

%% TODO: Do we need this still? I think it may still be used.
%% Initialize vsn cache
rebar_state:set(State1, vsn_cache, dict:new()).
{PluginProviders, rebar_state:set(State1, vsn_cache, dict:new())}.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason we don't return AllProviders or PluginProviders++GlobalPluginProviders ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of those are in scope... This is returning the globally found plugin providers.


%%
%% Parse command line arguments using getopt and also filtering out any
Expand Down
9 changes: 2 additions & 7 deletions src/rebar_base_compiler.erl
Original file line number Diff line number Diff line change
Expand Up @@ -258,10 +258,5 @@ format_error(AbsSource, Extra, {Mod, Desc}) ->
ErrorDesc = Mod:format_error(Desc),
?FMT("~s: ~s~s~n", [AbsSource, Extra, ErrorDesc]).

maybe_absname(Config, Filename) ->
case rebar_utils:processing_base_dir(Config) of
true ->
Filename;
false ->
filename:absname(Filename)
end.
maybe_absname(_Config, Filename) ->
Filename.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this suddenly need to go away as a need?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't sudden, dialyzer caught it. Since we no longer cd into directories but instead pass around full paths it isn't needed.

30 changes: 16 additions & 14 deletions src/rebar_core.erl
Original file line number Diff line number Diff line change
Expand Up @@ -40,40 +40,42 @@ process_command(State, Command) ->
not_found ->
{error, io_lib:format("Command ~p not found", [Command])};
CommandProvider ->
Profile = providers:profile(CommandProvider),
State1 = rebar_state:current_profile(State, Profile),
Opts = providers:opts(CommandProvider)++rebar3:global_option_spec_list(),
case Command of
do ->
do(TargetProviders, State);
do(TargetProviders, State1);
_ ->
case getopt:parse(Opts, rebar_state:command_args(State)) of
case getopt:parse(Opts, rebar_state:command_args(State1)) of
{ok, Args} ->
State2 = rebar_state:command_parsed_args(State, Args),
do(TargetProviders, State2);
State3 = rebar_state:command_parsed_args(State1, Args),
do(TargetProviders, State3);
{error, {invalid_option, Option}} ->
{error, io_lib:format("Invalid option ~s on task ~p", [Option, Command])}
end
end
end.

-spec do([atom()], rebar_state:t()) -> {ok, rebar_state:t()} | {error, string()}.
-spec do([{atom(), atom()}], rebar_state:t()) -> {ok, rebar_state:t()} | {error, string()}.
do([], State) ->
{ok, State};
do([ProviderName | Rest], State) ->
do([{ProviderName, Profile} | Rest], State) ->
State1 = rebar_state:current_profile(State, Profile),
Provider = providers:get_provider(ProviderName
,rebar_state:providers(State)),
case providers:do(Provider, State) of
{ok, State1} ->
do(Rest, State1);
,rebar_state:providers(State1)),
case providers:do(Provider, State1) of
{ok, State2} ->
do(Rest, State2);
{error, Error} ->
{error, Error}
end.

update_code_path(State) ->
true = rebar_utils:expand_code_path(),
BaseDir = rebar_state:get(State, base_dir, ?DEFAULT_BASE_DIR),
LibDirs = rebar_state:get(State, lib_dirs, ?DEFAULT_LIB_DIRS),
DepsDir = filename:join(BaseDir, rebar_state:get(State, deps_dir, ?DEFAULT_DEPS_DIR)),
PluginsDir = filename:join(BaseDir, rebar_state:get(State, plugins_dir, ?DEFAULT_PLUGINS_DIR)),
LibDirs = rebar_utils:lib_dirs(State),
DepsDir = rebar_utils:deps_dir(State),
PluginsDir = rebar_utils:plugins_dir(State),
_UpdatedCodePaths = update_code_path_([DepsDir, PluginsDir | LibDirs]).


Expand Down
12 changes: 9 additions & 3 deletions src/rebar_digraph.erl
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,15 @@ add(Graph, {PkgName, Deps}) ->
end,

lists:foreach(fun(DepName) ->
V3 = case digraph:vertex(Graph, DepName) of
case DepName of
{Name, _Vsn} ->
Name;
Name ->
Name
end,
V3 = case digraph:vertex(Graph, Name) of
false ->
digraph:add_vertex(Graph, DepName);
digraph:add_vertex(Graph, Name);
{V2, []} ->
V2
end,
Expand Down Expand Up @@ -94,5 +100,5 @@ names_to_apps(Names, Apps) ->
-spec find_app_by_name(atom(), [rebar_app_info:t()]) -> {ok, rebar_app_info:t()} | error.
find_app_by_name(Name, Apps) ->
ec_lists:find(fun(App) ->
ec_cnv:to_atom(rebar_app_info:name(App)) =:= ec_cnv:to_atom(Name)
binary_to_atom(rebar_app_info:name(App), utf8) =:= binary_to_atom(Name, utf8)
end, Apps).
71 changes: 32 additions & 39 deletions src/rebar_fetch.erl
Original file line number Diff line number Diff line change
Expand Up @@ -19,52 +19,45 @@
-spec lock_source(file:filename_all(), rebar_resource:resource()) ->
rebar_resource:resource() | {error, string()}.
lock_source(AppDir, Source) ->
case get_resource_type(Source) of
{error, _}=Error ->
Error;
Module ->
Module:lock(AppDir, Source)
end.
Module = get_resource_type(Source),
Module:lock(AppDir, Source).

-spec download_source(file:filename_all(), rebar_resource:resource()) -> true | {error, any()}.
download_source(AppDir, Source) ->
case get_resource_type(Source) of
{error, _}=Error ->
Error;
Module ->
TmpDir = ec_file:insecure_mkdtemp(),
AppDir1 = ec_cnv:to_list(AppDir),
ec_file:mkdir_p(AppDir1),
case Module:download(TmpDir, Source) of
{ok, _} ->
code:del_path(filename:absname(filename:join(AppDir1, "ebin"))),
ec_file:remove(filename:absname(AppDir1), [recursive]),
ok = ec_file:copy(TmpDir, filename:absname(AppDir1), [recursive]),
true;
{tarball, File} ->
ok = erl_tar:extract(File, [{cwd, TmpDir}
,compressed]),
BaseName = filename:basename(AppDir1),
[FromDir] = filelib:wildcard(filename:join(TmpDir, BaseName++"-*")),
code:del_path(filename:absname(filename:join(AppDir1, "ebin"))),
ec_file:remove(filename:absname(AppDir1), [recursive]),
ok = ec_file:copy(FromDir, filename:absname(AppDir1), [recursive]),
true
end
try
Module = get_resource_type(Source),
TmpDir = ec_file:insecure_mkdtemp(),
AppDir1 = ec_cnv:to_list(AppDir),
ec_file:mkdir_p(AppDir1),
case Module:download(TmpDir, Source) of
{ok, _} ->
code:del_path(filename:absname(filename:join(AppDir1, "ebin"))),
ec_file:remove(filename:absname(AppDir1), [recursive]),
ok = ec_file:copy(TmpDir, filename:absname(AppDir1), [recursive]),
true;
{tarball, File} ->
ok = erl_tar:extract(File, [{cwd, TmpDir}
,compressed]),
BaseName = filename:basename(AppDir1),
[FromDir] = filelib:wildcard(filename:join(TmpDir, BaseName++"-*")),
code:del_path(filename:absname(filename:join(AppDir1, "ebin"))),
ec_file:remove(filename:absname(AppDir1), [recursive]),
ok = ec_file:copy(FromDir, filename:absname(AppDir1), [recursive]),
true
end
catch
_:E ->
{error, E}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we catching here? I'm always wary of these catch alls, and may also prefer we at the very least catch R:E -> ... and return {error, {R,E}} to be able to later handle these cases better.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea.. It is a pain. So much could go wrong when dealing with the network, the users disk, moving files around. I can change it to {R, E} for now.

end.

-spec needs_update(file:filename_all(), rebar_resource:resource()) -> boolean() | {error, string()}.
needs_update(AppDir, Source) ->
case get_resource_type(Source) of
{error, _}=Error ->
Error;
Module ->
try
Module:needs_update(AppDir, Source)
catch
_:_ ->
true
end
Module = get_resource_type(Source),
try
Module:needs_update(AppDir, Source)
catch
_:_ ->
true
end.

get_resource_type({Type, Location}) ->
Expand Down
7 changes: 1 addition & 6 deletions src/rebar_git_resource.erl
Original file line number Diff line number Diff line change
Expand Up @@ -126,12 +126,7 @@ collect_default_refcount() ->
build_vsn_string(Vsn, RawRef, RawCount) ->
%% Cleanup the tag and the Ref information. Basically leading 'v's and
%% whitespace needs to go away.
RefTag = case RawRef of
undefined ->
"";
RawRef ->
[".ref", re:replace(RawRef, "\\s", "", [global])]
end,
RefTag = [".ref", re:replace(RawRef, "\\s", "", [global])],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I got how you got to make sure the reference can never be undefined anymore?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another catch by dialyzer. RawRef passed to build_vsn_string is the result of os:cmd("git log -n 1 --pretty=format:'%h\n' "), so will never be undefined.

Count = erlang:iolist_to_binary(re:replace(RawCount, "\\s", "", [global])),

%% Create the valid [semver](http://semver.org) version from the tag
Expand Down
2 changes: 1 addition & 1 deletion src/rebar_hooks.erl
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

run_compile_hooks(Dir, Type, Command, State) ->
Hooks = rebar_state:get(State, Type, []),
Env = [{"REBAR_DEPS_DIR", rebar_prv_install_deps:get_deps_dir(State)}],
Env = [{"REBAR_DEPS_DIR", rebar_utils:deps_dir(State)}],
lists:foreach(fun({_, C, _}=Hook) when C =:= Command ->
apply_hook(Dir, Env, Hook);
({C, _}=Hook) when C =:= Command ->
Expand Down