Add skip_deps=AppListSeparatedByCommas feature.
I agree it's a bit of a weird thing, but it's a reasonable and safe extension. When time comes to properly overhaul stuff, skip_deps should disappear entirely.
This allows `rebar ct suites=abc` to consider all suites when
you have a rebar setup with multiple sub_dirs.
Previously, rebar halted after it could not find the suite in the
first dir. But the suite might be present in another dir (when
sub_dirs contains multiple dirs).
This commit makes it so instead of halting, it prints a warning and
continues with looking for the suite in the other `sub_dir`s.
Note -- This uses try/catch to cause the test to be skipped because
otherwise I needed to adjust the return values of 4 functions, the
code path is pretty deeply nested here. Otherwise the whole call chain
needed to be adjusted for this return value:
`run_test -> make_cmd -> get_suites -> find_suite_path`
IMHO, I think for exceptional cases like this it is fine to use throw;
specially since only the {skip} is catched and nothing else.
When a DTL template includes other template files, those files don't need
to be compiled separately, and therefore can be given an extension different
from `source_ext` (such as `.dtli`) to avoid being compiled. This fix
allows rebar to find included dependencies with names `*.dtl*` rather
than `*.dtl` and properly determine if a template file needs to be recompiled.
Handle the case where the error didn't occur in the file being
compiled. That is, if there is an error on line 9 of bar.hrl,
instead of:
/path/to/foo.erl:9: type foo() already defined
print:
/path/to/bar.hrl:9: type foo() already defined
The printed message is made more terse.
Example before applying patch:
ERROR: Compiling template src/view/test.dtl failed:
{error,{"src/view/test.dtl",
[{{4,7},
erlydtl_parser,
["syntax error before: ",["\"\\\"HELLO_WORLD\\\"\""]]}]}}
Example after applying patch:
ERROR: Compiling template "src/cmp_html_error_template.dtl" failed:
(line:3, col:12): ["syntax error before: ",["trans"]]
Added a backward compartible feature to specify `erlydtl_opts' options
for the DTL template compiler to allow inclusion of templates in different
directories with different compilation settings for each. E.g.:
{erlydtl_opts, [
[{doc_root, "src/view"}, {module_ext, "_dtl_vw"}]
, [{doc_root, "src"}, {module_ext, ""}, {recursive, false}]
, {out_dir, "ebin"}
, {compiler_options, [verbose, debug_info]}
]}.
The definition above is identical to this (the last two options
are duplicated in each list):
{erlydtl_opts, [
[{doc_root, "src/view"}
,{module_ext, "_dtl_vw"}
,{out_dir, "ebin"}
,{compiler_options, [verbose, debug_info]}]
, [{doc_root, "src"}
,{module_ext, ""}
,{out_dir, "ebin"}
,{compiler_options, [verbose, debug_info]}
,{recursive, false}]
]}.
In this case "src/view" and "src" directories containing template files
will be compiled. A new `recursive' option tells rebar_erlydtl_compiler
to search files recursively from a given doc_root. In the example above
the "src" directory won't be scanned recursively, and the target template
name for target beam modules won't have "_dtl_vw" suffix.
Introduce a new 'raw' option for dependency specs in rebar.config file.
For example:
{deps,
{dependency_name, "1.0.*",
{git, "<...>", {branch, "master"}},
[raw]
}
]}.
When this option is specified, rebar does not require the dependency to
have a standard Erlang/OTP layout which assumes presence of either
"src/dependency_name.app.src" or "ebin/dependency_name.app" files.
'raw' dependencies can still contain 'rebar.config' and even can have
the proper OTP directory layout, but they won't be compiled.
Only a subset of rebar commands will be executed on the 'raw'
subdirectories:
get-deps, update-deps, check-deps, list-deps and delete-deps.
This allows an <app-name>.app.src.script to be defined and evaluated
when <app-name>.app.src or <app-name>.app are loaded. This allows the
user to add project specific manipulations to app metadata.
By default protobuffs doesn't create beams with debug info. This
causes issues when running dialyzer which requires debug info. Read
the `erl_opts` config and pass it down to protobuffs compiler.
Cover gets slower and slower for each application. This is due to the
cover_server internal state. Stopping the cover server between
eunit+cover runs, emptying the cover_server state, gives a ~5-6x speed
improvement when analyzing many Erlang modules. Stopping the cover
server replaces the earlier practice of doing a cover:reset before each
run. On a project consisting of 62 dependencies with a total of 1866
Erlang modules the running time of rebar eunit decreased from ~20
minutes to ~3 minutes.
1. manually insert directory entries to work around issues fixed in
R15B02 erl_prim_loader
2. put the escript archive's beams in appname/appname/ebin
Tested with R13B03 and R15B01.
When rebar ct executes with its default common test directory of "test",
it will generate a hardcoded "logs" directory in every application with
a test directory present, causing an overlap with eunit's test framework
so even test directories with only eunit tests will be processed by ct.
If running e.g. rebar generate using an OTP development build,
rebar will complain that reltool has the version "", which is
less than the required "0.5.2". This is because rebar_reltool
simply checks the path returned by code:which(reltool), which
doesn't yield version information if used in a development build.
This patch substitutes a more robust method (load reltool and
fetch the info from application:loaded_applications()).
As it happens, this will not be enough to make things work,
but now Reltool will explain that it cannot generate a spec
from a system that is not installed, giving a better hint
as to what needs to be done.
For now, this is just a proof of concept; would make sense to add a lot
of things, such as quick access to invoking rebar itself (with approproate
code reloading), as well as an eunit-aware version so that tests could be
run interactively.
By default, executables in paths such as /bin and /usr/local/bin have
the mode saying they can be executed by all.
The current version of escriptize only sets u+x, which creates problems
when copied directly in repositories and requiring other programs to
interact with them.
This change makes rebar follow the standard of linuxes and unixes by
setting the permission flag to a+x, allowing users, the group and others
to execute it.
'Erlang' projects that do not contain any erlang files (Joxa, LFE,
Elixir, etc) break the eunit task. It attempts to copy an empty list
of source files to the .eunit directory. This change makes copying an
empty list a simple no-op.
Make sure files from erl_opts->src_dirs are copied to .eunit for cover
inspection.
Without this patch cover failed to find source files which were in
src_dirs, and threw the following exception:
ERROR: eunit failed while processing
/path/to/project:
{'EXIT',{{badmatch,{error,no_source_code_found}},
[{rebar_eunit,'-cover_analyze/3-fun-0-',1,
[{file,"src/rebar_eunit.erl"},{line,280}]},
{lists,foreach,2,[{file,"lists.erl"},{line,1262}]},
{rebar_eunit,cover_analyze,3,
[{file,"src/rebar_eunit.erl"},{line,279}]},
{rebar_eunit,eunit,2,[{file,"src/rebar_eunit.erl"},{line,140}]},
{rebar_core,run_modules,4,[{file,"src/rebar_core.erl"},{line,425}]},
{rebar_core,execute,5,[{file,"src/rebar_core.erl"},{line,359}]},
{rebar_core,process_dir0,6,[{file,"src/rebar_core.erl"},{line,223}]},
{rebar_core,process_dir,4,
[{file,"src/rebar_core.erl"},{line,130}]}]}}
Now all source files are copied to .eunit.
Since you can't really do math with regexps and it's a pain to
repeatedly update the config for each new version or erlang, I wanted
to add support for minium OTP versions. This is a fix for
https://github.com/basho/riaknostic/issues/38
Not every system under test can be run with long names, and this makes
rebar's common_test support useless in those environments, as it
currently uses long names (test@hostname.domain) by default, without
recourse to change them.
This patch adds support for a {ct_use_short_names, boolean()} config
variable, which allows the user to specify whether short or long names
are required.
The custom queries are configured in rebar.config via the tuple
{xref_queries, [{query(), query_result()},...]}. The implementation
passes the query() string to xref:q and compares the return value with
query_result(). It will result in an error if they do not match.
The following configuration, for example, is the same as running the
xref check undefined_function_calls. It additionally filters
ejabberd_logger:*_msg/4 from the result as these functions are generated
on execution by ejabberd and not available at compile time.
{xref_queries, [{"(XC - UC) || (XU - X - B -
(\"ejabberd_logger\":\".*_msg\"/\"4\"))",[]}]}.
This patch also modifies the build process of this package by running a
custom query instead of doing a diff against a static xref_warning file.
The latest version of erlydtl requires that custom tag modules be in
the path when templates are compiled. Without this change rebar silently
exits when using the custom_tags_modules option.
Before copying the .erl files to the .eunit directory ensure that they
are deleted if they already exist. This prevents EACCES errors to happen
when trying to re-copy .erl files that are read-only.
a) make_symlink throws "not supported" on OS < Vista (e.g. 2003)
b) tarball creation stores windows symlinks as absolute paths
instead of relative, causing service to fail to start after upgrade
is made permanent.
Avoid error reports like this one:
prompt$ rebar eunit
==> dummy (eunit)
Test passed.
=ERROR REPORT==== 29-Dec-2011::23:22:11 ===
** Generic server inet_gethost_native_sup terminating
** Last message in was {'EXIT',<0.62.0>,killed}
** When Server state == {state,inet_gethost_native,undefined,<0.62.0>,
{local,inet_gethost_native_sup}}
** Reason for termination ==
** killed
Previously, it could happen if an eunit test did something that
required a name lookup, like in this example:
-module(dummy_tests).
-include_lib("eunit/include/eunit.hrl").
x_test() ->
{ok, _Hostent} = inet:gethostbyname(localhost).
The inet_gethost_native is a process, started on demand, under a
supervisor_bridge under the kernel_sup, but it is not a gen_server
process or the like, so it has nothing in '$ancestors' in its process
dictionary to indicate it is part of kernel.
This patch allows users to specify the directory in which common_test
source files can be found. Most common_test suites are integration,
rather than unit tests and keeping the sources apart from test sources
for other frameworks such as eunit and PropEr is a useful feature.
It is now possible to call rebar as:
rebar xref skip_app=Mod1,Mod2,...
This makes it easy to skip running xref on (e.g)
imported dependencies in your application.
The function rebar_utils:is_skipped_app/0 is added so
that other rebar commands may use it.
Previously if erl_interface could not be found {error, not_found} was
incorrectly used as a path. With this change we fail early with a
descriptive error message.
When generating a node using rebar generate, rebar always used
code:root_dir() to find the Erlang installation to clone into the node
tree. However, for anyone wishing to build a cross-compiled node, there
was no way to override this. This patch adds a new option to
reltool.config file to allow an alternate root_dir to be specified, for
example:
{root_dir, "/tmp/otp_R14B03_armv7l"}
This assumes that the contents of /tmp/otp_R14B03_armv7l has been
generated using the instructions found in the xcomp directory in your
$ERL_TOP structure. NOTE: you may have to add additional filters to
exclude files not present in the xcomp version, such as the .smp files
if you have disabled that.
This means that applications get a chance to shut down themselves,
before rebar will start killing their processes. This avoids ERROR
REPORTS from application supervisors.
The function may fail with a badarg exception because the first regex
returns an iolist() which is allowed to be a improper list. In this case
'++' cannot append to the iolist. The correct way to append something to
an iolist() is
[iolist(), "tail"]
because iolist's are allowed to be arbitrarily deep lists.
The introduction of setup_env as a global concept caused the rebar_port_compiler
implementation to start getting called a LOT. The expansion of environment variables
that happens in the port compiler was O(n^n), which means you could see upwards of
80k invocations of lists:foldl on a single app "./rebar clean". This commit reworks
the expansion to be O(n^2), and reduces the running time for the same operation by
60%+. On a large project like Riak, the end result is that a build went from 200
seconds to 73.
For parameterized modules, the beam code will have a compiler
generated new/1 and instance/1 function.
If while checking the beam, xref detects one of those is unused, the
rebars xref wrapper will try to find the location of the definition of
that function in the source code for the module (to give a more
specific warning to the user). Since the function was generated by the
compiler it does not actually exist in the source, and rebar crashes
at that stage. This patch works around that issue.
required for building on a different machine to what your live
config is kept on. This way you can build on one machine,
distribute to new machine, then run overlays against a live
config to prevent accidently running code against a live
config in a development or ci environment.
This patch modifies rebar_core to allow plugins to participate in the
pre and post processing steps, giving plugin authors more flexibility
and control.
Currently rebar_utils:sh/2 will invoke all commands through bash.exe if
found. Otherwise the command will be executed directly. Despite the fact
that the caller cannot know if the command is executed with Unix or
Windows semantics it leads to problems with MSYS's automatic path name
translation.
Therefore remove bash usage on Windows to get a consistent behavior and
to avoid the peculiarities of MSYS's automatic path conversion. Instead
use cmd.exe as its typically needed by most commands.
While building a upgrade package rebar will add new paths to the
internal erlang path, these paths and their order have effects on how
the package is built. This patch should fix some corner cases where a
user can receive a "undefined application" error.
While building an upgrade the sys.config file should be copied into the
upgrade tarball so release_handler:install_releases/1 does not clobber
the existing configuration from the application environment.
This patch adds support for customising the way in which rebar generates
version numbers for app.src files using the `{vsn,Spec}` approach.
Whilst the existing `{vsn,ScmName::atom()}` syntax will continue to
work, users can also pass `{vsn,{cmd,Cmd::string()}}` in which
case the provided *command* will be used. For example:
```erlang
{application, doodah,
[
{vsn, {cmd, "git rev-parse --short HEAD"}}]}.
```
This patch corrects the vm.args behavior while building upgrade tarballs
by copying the file from the release into the upgrade. Additionally it
patches the dummy runner script in the upgrade test project to work
properly.
app.config has been a long standing erroneous file in rebar. Erlang/OTP
documentation suggests a sys.config file instead. This file is stored in
the releases/VSN directory. This does a few things but most importantly
it ensures your config (contained in the application environment)
survives a hot upgrade. It also has the advantage of allowing the
configuration of the application to be versioned along side the
application code. This patch flips rebar to use sys.config rather than
app.config.
Additionally it makes this flip to vm.args as well, making them
versioned just like sys.config.
This patch also includes runner script changes to support the old
etc/app.config config file location and support for Windows.
Thanks to mokele for the initial work and kick in the pants to make this
finially happen.
This patch fixes the warning logging when the number of missing plugins
is greater than one. The current code only works by accident, when a
single plugin is all that is missing.
Export two extra environment variables when executing shell commands.
These variables are useful for rebar hooks that rely on Erlang
applications installed as rebar dependencies.
$REBAR_DEPS_DIR contains a fully-qualified name of the directory where
rebar stores dependencies.
$ERL_LIBS is set to $REBAR_DEPS_DIR or to "$REBAR_DEPS_DIR:$ERL_LIBS",
if $ERL_LIBS was defined before.
This patch updates rebar_core to look for missing plugins (i.e. those
that aren't found on the code path at runtime) in a configurable
plugin directory, and dynamically compile and load them at runtime.
By default, the directory "plugins" is searched, although this can be
overriden by setting the plugin_dir in your rebar.config.
Allows using tools like git-subtree and still maintaining proper
git-based vsns for those "imported" subtrees. Also helps understanding
what was the last change introduced to a specific application within a
multi-application repository (at this moment, all applications that
reside in the same repository, will share the same autogenerated git
vsn)
Currently the --config switch does not work because when loading
a new rebar config the global setting is ignored for all paths.
This patch provides a check when loading new rebar config to see
whether or not the current config path matches the `base_dir` set in
global conf, which produces the expected behaviour.
Added new property to `eunit_opts` option list:
reset_after_eunit::boolean() - default = true.
If true, try to "reset" VM state to approximate state prior to
running the EUnit tests:
* Stop net_kernel if it was started
* Stop OTP applications not running before EUnit tests were run
* Kill processes not running before EUnit tests were run
* Reset OTP application environment variables
Previous patch erroneously assumed that cover:start() returned
{already_started,Pid} in the cases where cover was already started. We
now turn {error,{already_started,Pid}} into {ok, Pid} and return
{error, Reason} if we encounter an error we do not know about, this
will cause a nice and violent badmatch to stop everything.
The cover module calls io:format and io:fwrite directly for several types of
informational messages and warnings. When using meck to mock covered modules
these warnings are triggered and can cause severe polution of the test output.
We can avoid this by starting cover explicitly then setting the group_leader
of that process to a file handle.
Generate .hrl files into include from SNMP MIB files.
Change the order of so .mib files are built before .erl
This is necessary since .hrl files are generated from
the .mib files.
The generated .bin and .hrl files are deleted by clean.
This is a cleaned up version of a patch originally
sent to the rebar mailing list by David Nonnenmacher.
The target_dir config in reltool allows you to put your release in
a directory other than in ./NAME, so we should look in the parent
directory of that to find the new and old versions instead of
simply looking in ./
Move untaring and retaring into a temporary path instead of in ./
to prevent name collisions with "releases" and "lib" that might
exist already. Having a subdirectory rel/releases/ can be useful.
