FAQ

This page contains answers to some common questions that people ask when they see ExMake.

General

Why yet another build tool?

Because the POSIX Make utility is antiquated. Once you start adding complex build logic to makefiles, they quickly turn into a maintenance nightmare because of the insane, inadequate scripting 'language' that Make sports. Make's syntax is also generally obscure; tabs have different semantics than spaces, rule lists are very hostile to paths with spaces, etc. Recursive Make invocation is a horrible hack that slows down the build and makes writing correct build scripts very error-prone. Finally, Make has next to no extension model beyond include directives and very limited templates.

There are other build tools that claim to solve these problems, but they all suffer from one or more of the following problems:

• They use ad-hoc, obscure languages with odd syntax and semantics.
• They generate makefiles rather than actually executing the build.
• They make unreasonable assumptions about the build environment.
• They only really work on a particular set of platforms.
• They are hostile to languages that are not blessed by the included libraries.

ExMake strives very hard to just be an improved version of Make. It adds features on top of what Make offers, but doesn't try to do anything fundamentally differently. ExMake uses Elixir for scripting and rule definitions, meaning that you have the power of a full-blown programming language with sane syntax at your disposal. Since it uses Elixir, standard modules like Path, File, IO, and String make it easy to write sane, portable build scripts. ExMake does not perform recursion the same way that Make does it; instead, recursion is a first-class citizen and script files in subdirectories are included in the main dependency graph. Finally, ExMake has built-in support for extension libraries that build scripts can load and use freely, and even provides an included library to help build them.

Why not just generate makefiles?

Doing so would mean that ExMake would have to use a configure-like phase to generate the 'real' makefiles, like what Autotools does. This goes against the philosophy of Make in the first place, and also means that ExMake would have to depend on Make. Worse yet, the generated makefiles would have to use recursive Make invocations for sanity (or insanity), slowing down the build for large projects. Finally, the Make job server is not particularly great since it has to use IPC to communicate with recursive Make processes.

Quite possibly one of the greatest advantages of ExMake is that it does everything in a single process (except for invoking external compilers, of course). For recipes that don't need to invoke external programs but just do some I/O or similar, you can avoid shelling out to any external program at all. This is a huge performance win, and also simplifies programming recipes in build scripts. This would not be possible if ExMake generated makefiles.

Why use a fringe language like Elixir?

The alternative would have been to use Lua, Python, Ruby, or Erlang (or similar). Lua, while a pretty good and ubiquitous language, has a rather lacking standard library. Python and Ruby have great standard libraries but are significantly lacking in the concurrency department. Erlang has fantastic support for concurrency but suffers from an inconsistent and somewhat lacking standard library.

Elixir, on the other hand, has all the good features of Erlang combined with a great, consistent standard library. It also has sane macros, which half of the ExMake infrastructure is based on.

Arguably, using a mostly-pure functional language for a build tool is odd, but the Erlang VM (that Elixir is based on) has just enough escape hatches to be usable in the real world. Being a functional language, it also encourages writing clean, understandable code.

One problem with Elixir is that it's not a very ubiquitous language, and that few people have Erlang installed by default. The first point is certainly true, and can't really be argued against. It is, however, the language that best fits the ExMake use case. On the bright side, Elixir is included in released ExMake binaries so users don't have to install it. The Erlang dependency, on the other hand, can seem unjustified to people who aren't using anything Erlang-related at all. Unlike Elixir, however, Erlang packages are available for pretty much any platform you can think of (see Installation). In the end, it's up to you to decide if you can live with the Erlang dependency.

Should I use ExMake?

It often happens that people curse at new, different build systems and hate anything that isn't whatever build system they are familiar with, such as Autotools. More often than not, this is because whoever implemented the build system for a particular project didn't properly weigh the advantages and disadvantages of the build tool used.

Consider what use cases you'll need to cater to with your build system. In particular, consider what use cases are common for other projects written in the programming language your project is written in. If ExMake cannot fulfill one of those use cases, or if doing so is extremely awkward, you probably should be using something else.

In short, to maintain the sanity of yourself and everyone else, use the build tool that fulfills your needs - not the build tool that you wish fulfilled your needs.

Features

In what areas is ExMake generally superior to Make?

There are pros and cons to the way ExMake does things, just as with Make's way of doing things. It's hard to call certain ExMake features objectively better than their Make equivalents, but the ExMake developers believe that the following things are, for most use cases, done better in ExMake:

• Scripting is done with a normal, general-purpose programming language.
• Full-blown libraries are supported, which can make as much use of Elixir as build scripts.
• Instead of invoking ExMake recursively, recursion is a first-class citizen.
• Compiled build scripts, the dependency graph, and the environment table are cached.
• Rules are specified with proper 'list of string' data types for targets and sources.
• Recipes can be expressed as any sequence of Elixir expressions.
• No output is printed by default; ExMake only prints output if explicitly told to be verbose, or if an error occurs.

What Make features are not supported by ExMake?

Most POSIX Make features are supported by ExMake, as far as we know. While some features are not directly supported, expressing them in terms of other features is entirely possible. So it's probably more accurate to ask what GNU Make features are not supported.

The GNU Make features that we know are not supported are:

• The VPATH variable and vpath directive and all functionality associated with them. These would make it hard to compute and cache the dependency graph.
• So-called "last resort" rules that attempt to create a target that no other rule covers. This has, empirically, little use in the real world.
• Built-in creation and updates of archive files. We believe this to be the job of proper ar invocations, not the build tool.
• The .LOW_RESOLUTION_TIME target. No modern operating system needs this.
• The .EXPORT_ALL_VARIABLES target. We believe this to be a recipe for disaster.
• The .NOTPARALLEL target. If you find yourself needing this, your build scripts are not written sanely.
• The .ONESHELL target. ExMake does not invoke shells in the same way that Make does, so it doesn't apply.

If you need any of these, you should probably reconsider what you're doing, or use another build tool entirely.

One other thing that may surprise Make users is that ExMake does not change directory when running recipes. Instead, recipes are expected to use full paths, or at least paths relative to the invocation directory. To help make this less painful, ExMake passes the directory of the current build script file to the recipe.

Does ExMake support configuration like configure.ac?

Yes and no; it's not supported in the traditional sense. Like Make, ExMake configures the environment on startup, which means setting CC, LD, AS, and whatever else might be needed. This is done whenever ExMake determines that the build cache in .exmake is stale. This is the case on a clean source control clone, after using -c to clear the cache, or after changing a build script file or some other manifest file.

In other words, the configure pass is executed whenever necessary, as opposed to being done explicitly as in e.g. Autotools. This means that affecting the configuration is somewhat different from the way it's done in Autotools.

Arguments can be passed to ExMake like so:

$ exmake my_program --args --prefix /opt/foo --libdir /usr/lib

These arguments are passed to all libraries that are loaded by the build scripts. They are interpreted by whichever libraries understand them (if any at all). For example, the Install library will interpret --prefix and --libdir.

Environment variables can be set to override the tools that ExMake libraries locate by default:

$ CC=arm-linux-gnueabihf-gcc exmake my_program

This will use arm-linux-gnueabihf-gcc as the C compiler and tell ExMake to not look for one by itself.

It's worth noting that ExMake remembers the last --args given to it, as well as the 'precious' environment variables (CC, LD, AS, etc). This means that if the cache is stale, ExMake will use the last configuration values to reconfigure.

How extensible is ExMake?

The core of ExMake (script loading, dependency calculation, job scheduling, etc) is not very extensible at the moment, simply because there hasn't been a good use case for opening up those components.

That aside, ExMake supports libraries that can be loaded dynamically by build scripts. These libraries can locate needed programs, discover configuration details, etc. They can also export macros and functions that can be used to simplify construction of build scripts.

Can ExMake work with other build systems?

Nothing stops you from invoking other build systems from ExMake, e.g. via the shell function. There's no built-in support, however, as mapping between build systems tends to be a mess to do correctly.

For Autotools-like build systems, the external build system's configure pass can be invoked from a library with an on_load function. This will ensure that it is only triggered when the ExMake cache is stale. This isn't necessarily perfect, but probably the best that can be done.

For simple build systems that use Make, the relevant rules can be translated into ExMake, but have their recipe simply be a shell call that invokes the Make-side recipe that has the actual build logic.

For other build systems, you'll have to experiment your way to the right solution. Mixing and matching build systems/tools is never easy.

What languages does ExMake support by default?

Libraries are currently available for the following languages:

• C#
• Elixir
• Erlang
• F#

Others will be added over time.

Performance

How does ExMake's serial performance compare to Make?

Since ExMake runs in the Erlang VM, it has a fixed startup overhead of around 150 to 200 milliseconds and a few milliseconds of shutdown overhead. This means that for small projects, Make is likely to perform better. However, scheduling of jobs and execution of recipes is roughly the same speed as Make, with ExMake only being around 5 milliseconds slower.

That being said, ExMake is not particularly optimized for serial workloads; rather, being based on the Erlang VM of all things, the idea is to perform well on parallel workloads (think -j 8).

How well does ExMake handle parallel jobs?

ExMake is specifically optimized for parallel workloads. It utilizes the Erlang VM's scheduler infrastructure to manage job runners and execute recipes.

In a test compile of Gproc, compiling with -j 1 takes roughly 2.6 seconds on average. Compiling with -j 4 takes 1.1 seconds. Compiling with -j 6 takes 0.9 seconds. All tests were done on a machine with an Intel i7-2600K CPU. So with parallel jobs, the build time is near a third of the serial speed. This scales as high as the machine being used has cores.

How much build information is cached?

The following major build artifacts are cached:

• The full dependency graph.
• The configuration environment table.
• Compiled modules built from script files.
• Precious system environment variables.
• Last-used --args arguments.

What this means in practice is that once ExMake has been invoked once, there is very little work for it to do before it can go ahead and invoke the requested rules on a subsequent invocation. In fact, the only work done is a cache staleness check, loading of cached data, and execution of recipes.

The cache is invalidated if any build scripts or manifest files are changed, or if --clear is passed. This will cause ExMake to reload and reconfigure everything (using the 'precious' environment variables and --args arguments last used).

Is recursion expensive in ExMake?

It is significantly cheaper than in Make and many other build tools. Recursing into a directory means that ExMake will have to load the script file (and possibly cache it). This means a slight fixed overhead of a few milliseconds - nothing special, and more or less the same situation as in any other build tool.

The reason ExMake shines when it comes to recursion is that all rules in subdirectories are added to the main dependency graph. This means that ExMake doesn't have to jump through hoops such as invoking itself recursively. In other words, rules in subdirectories are first-class members of the dependency graph just like the rules in the top-level directory.