One could argue that rebar_eunit does some pretty dangerous stuff when
trying to clean up after a test run, but specifically, it tells the
Application Controller to delete everything returned by
application:get_all_env(App). Unfortunately, included_applications
also goes, which can lead to a crash in application_controller, if
the application is subsequently unloaded.
This patch attempts the smallest possible fix: remove all env
variables except included_applications.
The results returned by os:getenv() may contain unicode characters.
That said, we need to explicitly allow unicode when splitting the
environment information, otherwise badarg will be raised causing all
rebar commands to fail until the environment variable is removed.
Normally, Rebar runs eunit tests in the order the beam files are
stored in the file system (see rebar_utils:beams). However, sometimes
it is desirable to run the tests in a different order (e.g. to
reproduce an error found on a build server). For that case, it would
make sense to use the 'suites' parameter not just for selecting which
modules to consider, but also for choosing the order.
Add skip_deps=AppListSeparatedByCommas feature.
I agree it's a bit of a weird thing, but it's a reasonable and safe extension. When time comes to properly overhaul stuff, skip_deps should disappear entirely.
This allows `rebar ct suites=abc` to consider all suites when
you have a rebar setup with multiple sub_dirs.
Previously, rebar halted after it could not find the suite in the
first dir. But the suite might be present in another dir (when
sub_dirs contains multiple dirs).
This commit makes it so instead of halting, it prints a warning and
continues with looking for the suite in the other `sub_dir`s.
Note -- This uses try/catch to cause the test to be skipped because
otherwise I needed to adjust the return values of 4 functions, the
code path is pretty deeply nested here. Otherwise the whole call chain
needed to be adjusted for this return value:
`run_test -> make_cmd -> get_suites -> find_suite_path`
IMHO, I think for exceptional cases like this it is fine to use throw;
specially since only the {skip} is catched and nothing else.
When a DTL template includes other template files, those files don't need
to be compiled separately, and therefore can be given an extension different
from `source_ext` (such as `.dtli`) to avoid being compiled. This fix
allows rebar to find included dependencies with names `*.dtl*` rather
than `*.dtl` and properly determine if a template file needs to be recompiled.
Handle the case where the error didn't occur in the file being
compiled. That is, if there is an error on line 9 of bar.hrl,
instead of:
/path/to/foo.erl:9: type foo() already defined
print:
/path/to/bar.hrl:9: type foo() already defined
The printed message is made more terse.
Example before applying patch:
ERROR: Compiling template src/view/test.dtl failed:
{error,{"src/view/test.dtl",
[{{4,7},
erlydtl_parser,
["syntax error before: ",["\"\\\"HELLO_WORLD\\\"\""]]}]}}
Example after applying patch:
ERROR: Compiling template "src/cmp_html_error_template.dtl" failed:
(line:3, col:12): ["syntax error before: ",["trans"]]
Added a backward compartible feature to specify `erlydtl_opts' options
for the DTL template compiler to allow inclusion of templates in different
directories with different compilation settings for each. E.g.:
{erlydtl_opts, [
[{doc_root, "src/view"}, {module_ext, "_dtl_vw"}]
, [{doc_root, "src"}, {module_ext, ""}, {recursive, false}]
, {out_dir, "ebin"}
, {compiler_options, [verbose, debug_info]}
]}.
The definition above is identical to this (the last two options
are duplicated in each list):
{erlydtl_opts, [
[{doc_root, "src/view"}
,{module_ext, "_dtl_vw"}
,{out_dir, "ebin"}
,{compiler_options, [verbose, debug_info]}]
, [{doc_root, "src"}
,{module_ext, ""}
,{out_dir, "ebin"}
,{compiler_options, [verbose, debug_info]}
,{recursive, false}]
]}.
In this case "src/view" and "src" directories containing template files
will be compiled. A new `recursive' option tells rebar_erlydtl_compiler
to search files recursively from a given doc_root. In the example above
the "src" directory won't be scanned recursively, and the target template
name for target beam modules won't have "_dtl_vw" suffix.
Introduce a new 'raw' option for dependency specs in rebar.config file.
For example:
{deps,
{dependency_name, "1.0.*",
{git, "<...>", {branch, "master"}},
[raw]
}
]}.
When this option is specified, rebar does not require the dependency to
have a standard Erlang/OTP layout which assumes presence of either
"src/dependency_name.app.src" or "ebin/dependency_name.app" files.
'raw' dependencies can still contain 'rebar.config' and even can have
the proper OTP directory layout, but they won't be compiled.
Only a subset of rebar commands will be executed on the 'raw'
subdirectories:
get-deps, update-deps, check-deps, list-deps and delete-deps.
This allows an <app-name>.app.src.script to be defined and evaluated
when <app-name>.app.src or <app-name>.app are loaded. This allows the
user to add project specific manipulations to app metadata.
By default protobuffs doesn't create beams with debug info. This
causes issues when running dialyzer which requires debug info. Read
the `erl_opts` config and pass it down to protobuffs compiler.
Cover gets slower and slower for each application. This is due to the
cover_server internal state. Stopping the cover server between
eunit+cover runs, emptying the cover_server state, gives a ~5-6x speed
improvement when analyzing many Erlang modules. Stopping the cover
server replaces the earlier practice of doing a cover:reset before each
run. On a project consisting of 62 dependencies with a total of 1866
Erlang modules the running time of rebar eunit decreased from ~20
minutes to ~3 minutes.
1. manually insert directory entries to work around issues fixed in
R15B02 erl_prim_loader
2. put the escript archive's beams in appname/appname/ebin
Tested with R13B03 and R15B01.
When rebar ct executes with its default common test directory of "test",
it will generate a hardcoded "logs" directory in every application with
a test directory present, causing an overlap with eunit's test framework
so even test directories with only eunit tests will be processed by ct.
If running e.g. rebar generate using an OTP development build,
rebar will complain that reltool has the version "", which is
less than the required "0.5.2". This is because rebar_reltool
simply checks the path returned by code:which(reltool), which
doesn't yield version information if used in a development build.
This patch substitutes a more robust method (load reltool and
fetch the info from application:loaded_applications()).
As it happens, this will not be enough to make things work,
but now Reltool will explain that it cannot generate a spec
from a system that is not installed, giving a better hint
as to what needs to be done.
For now, this is just a proof of concept; would make sense to add a lot
of things, such as quick access to invoking rebar itself (with approproate
code reloading), as well as an eunit-aware version so that tests could be
run interactively.
By default, executables in paths such as /bin and /usr/local/bin have
the mode saying they can be executed by all.
The current version of escriptize only sets u+x, which creates problems
when copied directly in repositories and requiring other programs to
interact with them.
This change makes rebar follow the standard of linuxes and unixes by
setting the permission flag to a+x, allowing users, the group and others
to execute it.
'Erlang' projects that do not contain any erlang files (Joxa, LFE,
Elixir, etc) break the eunit task. It attempts to copy an empty list
of source files to the .eunit directory. This change makes copying an
empty list a simple no-op.
Make sure files from erl_opts->src_dirs are copied to .eunit for cover
inspection.
Without this patch cover failed to find source files which were in
src_dirs, and threw the following exception:
ERROR: eunit failed while processing
/path/to/project:
{'EXIT',{{badmatch,{error,no_source_code_found}},
[{rebar_eunit,'-cover_analyze/3-fun-0-',1,
[{file,"src/rebar_eunit.erl"},{line,280}]},
{lists,foreach,2,[{file,"lists.erl"},{line,1262}]},
{rebar_eunit,cover_analyze,3,
[{file,"src/rebar_eunit.erl"},{line,279}]},
{rebar_eunit,eunit,2,[{file,"src/rebar_eunit.erl"},{line,140}]},
{rebar_core,run_modules,4,[{file,"src/rebar_core.erl"},{line,425}]},
{rebar_core,execute,5,[{file,"src/rebar_core.erl"},{line,359}]},
{rebar_core,process_dir0,6,[{file,"src/rebar_core.erl"},{line,223}]},
{rebar_core,process_dir,4,
[{file,"src/rebar_core.erl"},{line,130}]}]}}
Now all source files are copied to .eunit.
Since you can't really do math with regexps and it's a pain to
repeatedly update the config for each new version or erlang, I wanted
to add support for minium OTP versions. This is a fix for
https://github.com/basho/riaknostic/issues/38
Not every system under test can be run with long names, and this makes
rebar's common_test support useless in those environments, as it
currently uses long names (test@hostname.domain) by default, without
recourse to change them.
This patch adds support for a {ct_use_short_names, boolean()} config
variable, which allows the user to specify whether short or long names
are required.
The custom queries are configured in rebar.config via the tuple
{xref_queries, [{query(), query_result()},...]}. The implementation
passes the query() string to xref:q and compares the return value with
query_result(). It will result in an error if they do not match.
The following configuration, for example, is the same as running the
xref check undefined_function_calls. It additionally filters
ejabberd_logger:*_msg/4 from the result as these functions are generated
on execution by ejabberd and not available at compile time.
{xref_queries, [{"(XC - UC) || (XU - X - B -
(\"ejabberd_logger\":\".*_msg\"/\"4\"))",[]}]}.
This patch also modifies the build process of this package by running a
custom query instead of doing a diff against a static xref_warning file.
The latest version of erlydtl requires that custom tag modules be in
the path when templates are compiled. Without this change rebar silently
exits when using the custom_tags_modules option.
Before copying the .erl files to the .eunit directory ensure that they
are deleted if they already exist. This prevents EACCES errors to happen
when trying to re-copy .erl files that are read-only.
a) make_symlink throws "not supported" on OS < Vista (e.g. 2003)
b) tarball creation stores windows symlinks as absolute paths
instead of relative, causing service to fail to start after upgrade
is made permanent.
Avoid error reports like this one:
prompt$ rebar eunit
==> dummy (eunit)
Test passed.
=ERROR REPORT==== 29-Dec-2011::23:22:11 ===
** Generic server inet_gethost_native_sup terminating
** Last message in was {'EXIT',<0.62.0>,killed}
** When Server state == {state,inet_gethost_native,undefined,<0.62.0>,
{local,inet_gethost_native_sup}}
** Reason for termination ==
** killed
Previously, it could happen if an eunit test did something that
required a name lookup, like in this example:
-module(dummy_tests).
-include_lib("eunit/include/eunit.hrl").
x_test() ->
{ok, _Hostent} = inet:gethostbyname(localhost).
The inet_gethost_native is a process, started on demand, under a
supervisor_bridge under the kernel_sup, but it is not a gen_server
process or the like, so it has nothing in '$ancestors' in its process
dictionary to indicate it is part of kernel.
This patch allows users to specify the directory in which common_test
source files can be found. Most common_test suites are integration,
rather than unit tests and keeping the sources apart from test sources
for other frameworks such as eunit and PropEr is a useful feature.
It is now possible to call rebar as:
rebar xref skip_app=Mod1,Mod2,...
This makes it easy to skip running xref on (e.g)
imported dependencies in your application.
The function rebar_utils:is_skipped_app/0 is added so
that other rebar commands may use it.
Previously if erl_interface could not be found {error, not_found} was
incorrectly used as a path. With this change we fail early with a
descriptive error message.
When generating a node using rebar generate, rebar always used
code:root_dir() to find the Erlang installation to clone into the node
tree. However, for anyone wishing to build a cross-compiled node, there
was no way to override this. This patch adds a new option to
reltool.config file to allow an alternate root_dir to be specified, for
example:
{root_dir, "/tmp/otp_R14B03_armv7l"}
This assumes that the contents of /tmp/otp_R14B03_armv7l has been
generated using the instructions found in the xcomp directory in your
$ERL_TOP structure. NOTE: you may have to add additional filters to
exclude files not present in the xcomp version, such as the .smp files
if you have disabled that.
This means that applications get a chance to shut down themselves,
before rebar will start killing their processes. This avoids ERROR
REPORTS from application supervisors.
The function may fail with a badarg exception because the first regex
returns an iolist() which is allowed to be a improper list. In this case
'++' cannot append to the iolist. The correct way to append something to
an iolist() is
[iolist(), "tail"]
because iolist's are allowed to be arbitrarily deep lists.
The introduction of setup_env as a global concept caused the rebar_port_compiler
implementation to start getting called a LOT. The expansion of environment variables
that happens in the port compiler was O(n^n), which means you could see upwards of
80k invocations of lists:foldl on a single app "./rebar clean". This commit reworks
the expansion to be O(n^2), and reduces the running time for the same operation by
60%+. On a large project like Riak, the end result is that a build went from 200
seconds to 73.
For parameterized modules, the beam code will have a compiler
generated new/1 and instance/1 function.
If while checking the beam, xref detects one of those is unused, the
rebars xref wrapper will try to find the location of the definition of
that function in the source code for the module (to give a more
specific warning to the user). Since the function was generated by the
compiler it does not actually exist in the source, and rebar crashes
at that stage. This patch works around that issue.
required for building on a different machine to what your live
config is kept on. This way you can build on one machine,
distribute to new machine, then run overlays against a live
config to prevent accidently running code against a live
config in a development or ci environment.
This patch modifies rebar_core to allow plugins to participate in the
pre and post processing steps, giving plugin authors more flexibility
and control.
