Archive

Archive for October, 2011

Linq to C++ (or something much better)

October 31, 2011 9 comments

A really long time ago (26th Jan 2009 to be exact), I got excited about lambdas in C++0x, and the idea of bringing a Linq-like lazily-evaluated sequence filtering system to C++. I wrote a blog post about this idea partly as a pedagogical exercise in explaining the parallels/divergences in how an iteratable sequence is represented in C++ and C#.

Whenever I’m asked if this would make sense as a real library, I’ve replied that in the real world it would make much more sense to build on the stuff in Boost, especially the range library.

And guess what? At that very moment someone was already working on it (only with a far more elegant, extensible interface), and the following month a set of extensions to range were put up for review, and were accepted… then a long time passed (they’re very careful) until summer of 2010, when finally the new features were released in Boost 1.43.

I only discovered this just now. People still seem to be finding that old blog post, so it only seems proper to get up-to-date. My first stab looked like this:

#include <boost/range/algorithm/for_each.hpp>
#include <boost/range/adaptor/filtered.hpp>
#include <boost/range/adaptor/transformed.hpp>

#include <boost/lexical_cast.hpp>

#include <vector>
#include <iostream>
#include <string>

int main()
{
	using namespace boost::adaptors;

	std::vector<int> numbers;
	numbers.push_back(0);
	numbers.push_back(-1);
	numbers.push_back(4);
	numbers.push_back(-3);
	numbers.push_back(5);
	numbers.push_back(8);
	numbers.push_back(-2);
	
	auto quoted_positives = numbers 
		| filtered([] (int n) { return n >= 0; })
		| transformed([] (int x) { 
                    return "\"" + boost::lexical_cast<std::string>(x) + "\""; 
                  });

	boost::for_each(quoted_positives, [] (std::string n) { 
            std::cout << n << std::endl; 
        });
	return 0;
}

As you can see, the key is the use of operator| to make a piping syntax, so that we can naturally read from left to right (you may have seen the same approach used in F#). So I start with a vector of plain integers called numbers, I then pipe that through the filtered adaptor so as to get just the positives (corresponding to Where in Linq), and finally through the transformed adaptor to turn each positive into a quoted string (corresponding to map in most functional frameworks and to Select in Linq).

Except… it doesn’t work.

The problem is that boost is not yet entirely C++11 lambda-friendly. It’s unfortunate that the boost documentation (and other online information) is peppered with the word lambda signifying something else, so I may just be missing something. But the gist of the problem appears to be that transformed wants a functor with a member type called result_type. So you are supposed to write something like this:

struct quote
{
    typedef std::string result_type;

    std::string operator()(int x) const 
    {
        return "\"" + boost::lexical_cast<std::string>(x) + "\""; 
    }
};

C++11 lambdas don’t have that result_type inside them, so they don’t directly fit. It goes without saying that we don’t like this, because the whole point of lambdas is to avoid writing a separate class somewhere else.

It is surely possible to extend boost’s function_traits so that they can reflect on lambdas. In the past they have not been written to work on general functors because a functor can have multiple overloads of operator(), so you are required to manually pick one of those members before you ask questions. But this position is no longer sensible (if indeed it ever was). The majority of functors will now be lambdas, with a single overload of operator(), and it is (as we can see from this example) perfectly reasonable to want to know the return type of that single overload. A lambda is an alternative to a plain function, and whatever is useful for a function will be useful for a lambda.

But in the meantime, it is interesting to see how many hoops we have to jump through to make the above sample code work as expected.

If you read my last update on this sordid subject you’ll know that I don’t want to have to state the type of something that can be inferred, so I’m not going to be satisfied by a wrapper template that just adds a manually specified result_type member to my lambda. Ugh! It must be inferred automatically.

Fortunately this is just a matter of using a few simple steps in metaprogramming. Firstly, a template designed to match a member function:

template <class T>
struct mem_fun_traits { };

Of course in this general form it will match anything. But we’ll only bother to specialise it for one case: a pointer to a const member function that accepts one argument:

template <class R, class C, class A>
struct mem_fun_traits<R (C::*)(A) const> 
{
    typedef R result_type;
    typedef A argument_type;
};

So if you pass mem_fun_traits the type of a pointer to the operator() within a lambda, you know what it accepts and returns. But how do you get that type argument? With decltype that’s easy-peasy:

template<typename T>
struct functor_member {
    typedef decltype(&T::operator()) type;
};

Armed with these tools, we can write a traits class for our simple one-argument, single-overload functors (and hence, by an extraordinary coincidence, also for C++11 lambdas):

template <class F>
struct functor_traits
{
    typedef typename functor_member<F>::type member_t;
    typedef mem_fun_traits<member_t> traits;

    typedef typename traits::result_type result_type;
    typedef typename traits::argument_type argument_type;
};

This is the thing that should be integrated into boost’s function_traits, to make lambdas work as true replacements for functions. I’m sure someone’s working on it.

In the event that this doesn’t happen, we can shoehorn it together anyway by making a function template that adds type metadata to any single-argument functor:

template <class F>
class functor_with_traits_t
{
    F f;

public:
    functor_with_traits_t(F f_) : f(f_) {}

    typedef typename functor_traits<F>::result_type result_type;
    typedef typename functor_traits<F>::argument_type argument_type;

    result_type operator()(argument_type a) const { return f(a); }
};

template <class F>
inline functor_with_traits_t<F> functor_with_traits(F f) { return functor_with_traits_t<F>(f); }

So if you have a plain old lambda like this:

auto l = [] (int x) { 
    return "\"" + boost::lexical_cast<std::string>(x) + "\""; 
};

You can “fix” it to have type metadata like so:

auto l2 = functor_with_traits(l);

This makes it compatible with boost’s transformed! Although manually putting in that wrapper each time is pretty tedious. We can work around that by defining our own adapter (let’s call it selected in slavish imitation of Linq) that just plugs together functor_with_traits and transformed. Sounds simple, though as always the difficulty is in figuring out the types:

template <class F>
inline um... selected(F f)
{
    return boost::adaptors::transformed(functor_with_traits(f)); 
}

The um... is “whatever type we’re returning”. Wouldn’t it be nice if we could put auto there? Well, we almost can: decltype to the rescue as usual:

decltype(
    boost::adaptors::transformed(
        functor_with_traits(make_ref<F>())
    )
)

That is: give me the type that would be returned if I called transformed, passing it the result of a call to functor_with_traits, passing it an F. Of course we can’t assume that F can be default constructed, hence the use of that funny make_ref function that always comes in handy with decltype.

So our completed adaptor is:

template <class F>
decltype(
    boost::adaptors::transformed(
        functor_with_traits(make_ref<F>())
    )
)
selected(F f)
{
    return boost::adaptors::transformed(functor_with_traits(f)); 
}

We can use selected in place of transformed and everything is hunky dory. Full example:

#include <boost/range/algorithm/for_each.hpp>
#include <boost/range/adaptor/filtered.hpp>
#include <boost/range/adaptor/transformed.hpp>

#include <boost/lexical_cast.hpp>

#include <vector>
#include <iostream>
#include <string>

template <class T>
T *make_ptr() { return (T *)0; }

template <class T>
const T &make_ref() { return *(make_ptr<T>()); }

template <class T>
struct mem_fun_traits { };

template <class R, class C, class A>
struct mem_fun_traits<R (C::*)(A) const> 
{
  typedef R result_type;
  typedef A argument_type;
};

template<typename T>
struct functor_member {
    typedef decltype(&T::operator()) type;
};

template <class F>
struct functor_traits
{
	typedef typename functor_member<F>::type member_t;
	typedef mem_fun_traits<member_t> traits;

	typedef typename traits::result_type result_type;
	typedef typename traits::argument_type argument_type;
};

template <class F>
class functor_with_traits_t
{
	F f;

public:
	functor_with_traits_t(F f_) : f(f_) {}

	typedef typename functor_traits<F>::result_type result_type;
	typedef typename functor_traits<F>::argument_type argument_type;

	result_type operator()(argument_type a) const { return f(a); }
};

template <class F>
functor_with_traits_t<F> functor_with_traits(F f) { return functor_with_traits_t<F>(f); }

template <class F>
decltype(boost::adaptors::transformed(functor_with_traits(make_ref<F>()))) selected(F f)
{
	return boost::adaptors::transformed(functor_with_traits(f)); 
}

int main()
{
	using namespace boost::adaptors;

	std::vector<int> numbers;
	numbers.push_back(0);
	numbers.push_back(-1);
	numbers.push_back(4);
	numbers.push_back(-3);
	numbers.push_back(5);
	numbers.push_back(8);
	numbers.push_back(-2);
	
	auto quoted_positives = numbers 
		| filtered([] (int n) { return n >= 0; })
		| selected([] (int x) { 
                    return "\"" + 
                      boost::lexical_cast<std::string>(x) +
                      "\"";
                  });

	boost::for_each(quoted_positives, [] (std::string n) {
            std::cout << n << std::endl; 
        });
	return 0;
}

(Tested on MSVC++ 16.00.40219.01 and GNU G++ 4.5.3.)

Categories: Uncategorized Tags: , , , ,

Secrets of the GNU Make Ninjas: Part 2 – Introducing MORK

October 9, 2011 1 comment

Suppose you had written some make scripts that took advantage of all the fun metaprogramming possibilities to make life as simple as possible for you. Imagine what that might look like.

There would be a directory representing your whole project. (Everything else we discuss will be under that directory, so I’ll use full relative paths from now on.)

It would have a subdirectory containing some general-purpose makefiles that take care of all the messy work, which you usually never need to look at. Let’s call that subdirectory morkfiles, because mork sounds like make, only it’s funnier, and so is a good name for our fantasy makefile framework.

And there would be one actual makefile under the top level directory, acting as the entry point. It would contain only this:

include morkfiles/main.makefile

It’s just there to allow you to conveniently type make at the command line and kick off a build of all targets, or say make clean to wipe everything previously built.

Executables

So, how would you go about writing, say, an executable?

You’d start with a subdirectory called, say, myexe, where you’ll build your first executable (hence the boring example name).

You’ll need a main source file, myexe/myexe.cpp:

#include <iostream>

int main()
{
	std::cout << "Hello from myexe" << std::endl;
}

Not exactly world-changing, but good enough for now. If you typed make at this point, it would be weird and wrong if it went ahead and assumed anything. No, first we have to undertake the arduous task of writing a morkfile, which would be saved as myexe/morkfile and would contain this:

$(this)_type = exe

Now, there’s a lot of stuff to explain there, so I’ll go slowly… wait a second, no there isn’t. It’s literally one line long. It means, unsurprisingly, “This thing is going to be an executable”.

If you run make now, a mini-miracle occurs. Somehow mork figures out what to do. And you can run your new program:

myexe/bin/myexe

or on Windows:

myexe\bin\myexe.exe

Things to note:

  • We haven’t told mork where to find the myexe directory. It automatically finds all files called morkfile. You can reorganise your source tree willy-nilly, add or remove modules, and never have to update any central configuration file.
  • Neither did we have to add myexe/myexe.cpp to a list. An executable module is simply assumed to have source files directly inside it. All files with extension .cpp are automatically compiled and linked. We could add more source files under myexe if we wanted to, re-run make and they would be added to our executable.
  • And of course, we only recompile what we need to. The second time you run make after changing nothing, it immediately reports it has nothing to do (even if you have dozens of modules, it says this very quickly, because only one instance of make is executing).

If you’ve used make before you may be wondering about the business of using the preprocessor to generate include file dependencies. Don’t worry about it; that stuff is taken care of automatically. No need to say make depends or anything like that. No stupid error messages when you delete a header. It just works.

Libraries

But real projects involve multiple targets – some are executables, some are static libraries, some are shared libraries (DLLs on Windows). These all have to depend on one another, which typically means that they have to be built in the right order, and include each other’s header files.

So let’s make a static library. Create another directory called myutil – you can put it absolutely anywhere beneath the top-level project directory (even inside myexe though that would be unnecessarily confusing, so don’t do that).

This time the source file, myutil/myutil.cpp, has no main function. Instead it exposes a function intended to be called from another module:

#include <myutil.hpp>

std::string myutil_says()
{
	return "world.";
}

We also need to write that header file. It goes in a subdirectory of the module, as myutil/include/myutil.hpp:

#ifndef MYUTIL_INCLUDED
#define MYUTIL_INCLUDED

#include <string>

std::string myutil_says();

#endif//MYUTIL_INCLUDED

That include subdirectory is where you put all headers that may need to be included by other modules – it’s the “public interface” of your module. Note that you don’t have to use the extension .hpp; the only significant thing here is the exact name and location of the include subdirectory.

Finally we have to confront that terrifying task of writing myutil/morkfile:

$(this)_type = static

Wait a second, that looks vaguely familiar! The only difference from last time is that we’re giving this module the type static instead of exe.

At this point we can run make and, no surprises, a library is built. It’s created under myutil/lib, but as we’ll see, you don’t even need to know that.

Now let’s make myexe use myutil. Back in myexe/myexe.cpp we adjust it to:

#include <iostream>

#include <myutil.hpp>

int main()
{
	std::cout << "Hello, " + myutil_says() << std::endl;
}

So myexe requires myutil. Now, mork is helpful, but not telepathic (or smart/stupid enough to try to scan your source files looking for implied dependencies between modules). You have to tell it that myexe requires myutil, by amending myexe/morkfile like so:

$(this)_type = exe
$(this)_requires = myutil

Couldn’t be simpler, could it? If your module requires another other modules, you list them in the $(this)_requires variable, separated by spaces, and mork takes care of adding myutil/include to the compiler options for myexe. It also (of course) deals with the linker options.

But it goes further – requirements are automatically transitive. That is, if module A requires module B (including some of its headers), and module B requires module C (so some of B’s headers include some of C’s headers), then it follows that A will not be able to compile successfully unless it can include headers from both B and C. But this is handled automatically: you only have to configure A to require B, and A will implicitly require C also, and so “it just works”.

Note that you only give the name, not the full path, to the required modules. Hence modules need to have unique names within the entire source tree, not just unique paths. This is a good idea anyway because when you distribute your software it’s usually convenient to put all the binaries (executables and shared libraries) in one directory, so they’ll need to have unique names.

But it has an additional nice advantage here, which is that even if you have dozens of modules that require one another in complex ways, you can still rearrange the modules under neat organisational subdirectories. It makes no difference to mork.

Dynamic Libraries

What about dynamic libraries (shared objects on Linux or DLLs on Windows)? It’s as easy as amending myutil/morkfile to:

$(this)_type = shared

That’s it. Run make again and everything is taken care of – mork even copies the built DLLs or shared objects to the bin directory of any executable that requires them, so it can be instantly executed.

Java

Oh yes, for good measure mork can build Java .jar files as well. You just have the morkfile look like this:

$(this)_type = jar

You put your Java source files in a subdirectory called src. The requires works exactly the same – it controls build order, as with native code modules, but also figures out the classpath settings.

This can be useful if you’re writing a mixed native/Java project with the emphasis on the native code. (If it was mostly Java, you’d presumably use Java build tools… by the way, I have something a little bit like mork but layered on ant, and integrating very nicely with Eclipse, so I’ll write that up sometime soon.).

Other stuff

There are more topics to consider, my favourite being code generation, which is a massively important technique, sadly underused in many projects, and many build tools don’t seem designed to support it, but mork is ready. And how do you integrate unit testing? Also very easy. But this post is long enough already.

ME WANT MORK!

I almost forgot… here’s where mork lives:

http://earwicker.com/mork

It has some examples with it that are very similar to the ones suggested here, but have more complex interdependencies. If you want to use it on Linux, it should just work, assuming you have the usual Gnu development tools.

Windows Quirks

On Windows, you need to install Cygwin with the development tools, and Visual Studio (for maximum interoperability with standard Windows libraries, mork uses Visual C++ as the compiler but relies on Cygwin for make, the bash shell and related commands – it even uses gcc, purely to get make-compatible dependencies).

So you need to have the Visual C++ tools and Cygwin accessible in your environment. The ordering of the PATH environment variable is important. Visual C++ paths need to be at the front of the PATH, so the correct (Microsoft) link.exe is used, then cygwin\bin so that the correct (Cygwin) find.exe is used, and finally the rest of your standard Windows PATH.

Next time: Part 3, in which we delve into how mork works, how to extend it, why the morkfile settings begin with $(this)_ and so on.

Categories: Uncategorized Tags: , , , ,

Secrets of the GNU Make Ninjas: Part 1

October 1, 2011 2 comments

The title of this post is a lie, of course. I am not a Ninja in GNU Make. I started learning it last week.

However, in that time I’ve cooked up a tiny framework of makefiles that I’m excited about. To put it another way, I’ve been driven partially insane by the experience – how else could someone get excited about something as old-fashioned as makefiles?

But if you’ve seen typical makefiles before, and then you see my beautiful new* alternative way of using them, I guarantee** you’ll get excited too. You’ll throw away*** your Visual Studio, Eclipse, NetBeans, Ant, Nant, Maven, and so on. They’ll seem so clunky in comparison to the elegance of my makefile framework. Oh yes.

(* Not really new)
(** Not an actual guarantee)
(*** Except for editing code and debugging and all that other stuff)

Note that throughout I’m talking about GNU make, not any of the many other incompatible makes.

The main cause of the excitement is my surprise when I actually starting R-ingTFM. Turns out that make is no less than a functional metaprogramming environment.

Like C++ templates, it uses recursively expanded definitions to generate a program. That generation process is “phase one”. Then the generated program actually runs, and has imperative/procedural elements to it: that’s “phase two”. Broadly speaking.

This is extremely powerful. Most of the things that people complain about being impossible in make are in fact perfectly possible, even easy, as long as you treat it as what it is: a functional metaprogramming environment.

Also, by the way, because it’s declarative and functional and so on, it’s amenable to automatic concurrency: make looks at what you’ve told it, and figures out which parts can run in parallel, and takes care of it (on Linux, anyway.)

It’s example time. If you want to play along, start a text editor, save the file as makefile. On Windows you’ll need to get Cygwin or MinGW, on Mac OS X you’ll need to install Xcode. Then you can type snippets in and save them, and from the same directory at the command line, run make. (Note that some of these early examples won’t do anything as we’re just defining variables.)

somevar = a b c d

I’ve created a variable called samovar, and stored a string containing four letters separated by spaces. Space separated lists are special – as well as being used wholesale as a simple string, there are handy built-in functions that treat them as lists.

This leads to Important Conclusion 1: spaces in filenames are bad. Just don’t put spaces in any of your directory or source files that you want make to deal with, and you’ll be fine. Probably seems a bit backward to you at first, but it turns out to be a fine trade-off.

To get the value of a variable:

othervar = before$(somevar)after

Now othervar contains beforea b c dafter because I referred to samovar in parentheses with a dollar sign in front, which tells make to substitute the value of the named variable.

Do those two definitions have to be in a particular order? What if I gave make the othervar definition at the top of the file and then the samevar definition after that? Can you guess?

The answer is due to the way make expands variables. My definition of othervar is stored exactly as I wrote it: the dollar sign, the parentheses, the somevar, are not expanded. Only when I actually refer to othervar in some context will make actually carry out the expansion, and by that stage it will know all the variable definitions.

Hence this is a problem:

a = $(b)
b = $(a)

Torturing make like that will just get you error messages.

You can in fact cause the expansion to happen at the point where you define the variable, by using := instead of plain =. For example, if you wanted to find all the png files in the images directory, you could say (using the built-in wildcard function):

all_pngs := $(wildcard images/*.png)

(Important Conclusion (IC)2: use forward slashes for file paths. On Windows, Cygwin will deal with it.)

The advantage of using := is that the directory gets scanned once and then all_pngs will contain a space-separated list of filenames. If you used plain = then the directory would get scanned every time you referred (directly or indirectly) to all_pngs.

And hence, IC3: variables defined with plain = are in fact functions. When you refer to them, you’re really calling them. They’re functions that don’t take any arguments, but they are still functions. They are in a simple sense closures: they close over the whole single variable namespace.

IC4: choose variable names carefully. One big namespace => danger of accidental clashes.

Actually you can pass arguments to variables when you refer to them. There’s a built-in function called call:

square_brackets = [ $(1) ]

example := hello $(call square_brackets,world)

Now example contains hello [ world ] – note how you can get the argument’s value in the definition by referring to its position (one-based).

Back to our list of pngs. I want to get a list of jpgs that would have the same base names as those pngs:

image_basenames = $(basename $(all_pngs))

Note how I don’t have to say “for each thing on the list, do something to it”. The built-in basename is list-aware. That is, it breaks up the input according to spaces, and then operates on each piece, and finally joins them back into a single string with space separators. To finish the example:

jpg_names = $(addsuffix .jpg,$(image_basenames))

Or to do both steps in one line:

jpg_names = $(addsuffix .jpg,$(basename $(all_pngs)))

There’s also a built-in foreach, which is like a general list operator, analogous to Select in Linq, or map in most other functional libraries. If you’re up on your monads, you’ll already have realised something semi-cool.

The core of a monad is the bind operator. For list monads this goes by various names depending on the language: in Linq it’s SelectMany, in Scala it’s flatMap. But it always does the same thing: for each item of the list, create a new list (somehow), and then join all the little lists together into one big list. The input is a list, the output is a list: that makes it composable.

In make the built-in list operators are all automatically monadically composable in that way, because a list is a string and a string is a list. (If the string has no spaces, it’s a list of one item, right?) So if you have a list-of-lists-of strings, then you already have a list-of strings. They’re indistinguishable. I said semi-cool because what if you don’t want the list to be flattened? But mostly you do.

IC5: This make stuff has all kinds of deep analogical relationships to interesting things!

Everything we’ve done so far is actually useless because it just defines string (or lists, same thing). With some shell incantations we can run make and print out the final value of a variable:

make -p | grep 'my_variable_name = '

But to actually cause make to do something we have to define a rule. The simplest kind of rule is:

files-to-build : required-files
	shell command
	shell command
	and so on


The shell commands are executed as normal operating system shell programs. The required-files and files-to-build are of course space separated. You can refer to variables anywhere. Note that each shell command has a tab indentation, and it must be a proper tab character, not spaces.

IC6: Make depends on a shell, so for portable makefiles, use UNIX-style shells on all systems (it’s the default on Windows with Cygwin anyway).

IC7: Use an editor that can make tab characters visible in some way.

The commands under a rule will only be executed if any of files-to-build has a timestamp that is earlier than any of required-files (or doesn’t exist at all). If any of required-files doesn’t exist, then make will complain, unless there is another rule in which that missing file appears in the files-to-build list, in which case make will run that other rule first, in the hope that it will cause the missing file to miraculously appear.

The upshot is that if you put a non-existent filename on the left, and nothing on the right, and then you make sure your shell commands don’t create the non-existent file, then you have something that will run every time you execute make:

run_every_time:
	@echo Hello, world!

When you run make, it will notice that run_every_time doesn’t exist, and go ahead with the commands in an attempt to make it. Note that it is not a terrible crime to fail to generate the target file of a rule, but most rules should be designed to do that. Most problems people get into with make involve trying to use phony rules to do imperative programming.

IC8: This is a tool for generating output files from input files, doing the minimum amount of work each time it is run. Each rule should take in some input files and write out some output files. Avoid writing rules that try to cheat the system; otherwise you’ll eventually get into a mess.

Now, what we’ve seen so far comprises the standard feature set of make as used in most makefiles. But I mentioned metaprogramming before; that’s not possible with these features along. Consider:

a = b = won't work
$(a)

That is, in variable a I store an expression that would (if it appeared on its own) assign the value won't work to b. And then on the next line I refer to that variable, hoping that this will cause b to be defined. But it’s a syntax error.

However, with another built-in, eval, we can do this:

$(eval $(a))

And that works! The variable b gets defined. So we can generate statements and then get them evaluated. But some things in make are inherently multi-line: rules especially. Can we define multi-line variables and so automate the declaring of rules? You’re darn tootin’ we can:

define my_recipe
cake: flour eggs sugar butter spices
	preheat oven
	mix sugar and butter
	stir in eggs
	stir in flour
	bake
	oh, forgot to add spices
endef

Note that my_recipe is just a variable. And by the way, did I mention that we can pass parameters to eval? I didn’t? Well, consider it duly mentioned.

This is where we can experience the true joy of metaprogramming, because we’re using the $(syntax) of variable references to generate code that must then go on to declare variable references that will only make sense later on. Some variables should be expanded before eval interprets its input. Others should go through eval unmolested so that they can be expanded later. To arrange this, we have to tell make to ignore some of our variable references by writing two dollars together: $$ is the escape sequence for a dollar. So eval ignores those variable references and “outputs” one dollar as a result, which can then be seen by make when it actually tries to evaluate things later on.

So, IC9: anything you’d declare in a makefile can be auto-generated within make from a multi-line template stored in a variable. There is never any need to repeat yourself. Say it once, say it loud: I’m metaprogramming and I’m proud.

It can get quite hair-raising when you’re trying understand snippets of metaprogramming, because two levels are intermingled. It is code talking about code. If you like that kind of thing, you’re a proper nerd like me. Hi!

Anyway, with these tools of functional metaprogramming in our back pocket, we can easily get make to solve a huge range of problems. Yes, easily. And we can make it so it runs fast. Yes, fast!

And yet so many blogs are full of people complaining about how make wouldn’t let them do this or that, or it took ten minutes to compile each file, so they came up with something else entirely. It’s a tragedy. Like the LISP programmers would say, every non-trivial C program contains an ad hoc implementation of LISP. Well, every new build-directing tool is basically the same as make, except that the author didn’t bother to RTF-make-M, so they didn’t realise they didn’t need to write their own version.

The number one cause of irritations with make is a technique called recursive make. When you have to build five static libraries, three dynamic libraries and four executables, plus some code generation, its often suggested that you should have a main makefile that visits each module’s directory and there runs another instance of make. But this turns out to be a very bad idea, as noted in this famous paper. I tried recursive make last week, and I’m not going to use it ever again. It runs counter to your nerd instincts to reject anything recursive, of course, because recursion is such an important, primal nerd totem. But this isn’t the same as abandoning recursion altogether! All the techniques described above are powerfully recursive – much more so than the specific technique known as “recursive make”, where you actually launch a separate instance. That technique sucks.

In Part 2: a quick tutorial of my fantasy make framework, where everything is easy and fast. And it’s not just a fantasy!

Categories: Uncategorized Tags: