Home > Uncategorized > Secrets of the GNU Make Ninjas: Part 2 – Introducing MORK

Secrets of the GNU Make Ninjas: Part 2 – Introducing MORK

Suppose you had written some make scripts that took advantage of all the fun metaprogramming possibilities to make life as simple as possible for you. Imagine what that might look like.

There would be a directory representing your whole project. (Everything else we discuss will be under that directory, so I’ll use full relative paths from now on.)

It would have a subdirectory containing some general-purpose makefiles that take care of all the messy work, which you usually never need to look at. Let’s call that subdirectory morkfiles, because mork sounds like make, only it’s funnier, and so is a good name for our fantasy makefile framework.

And there would be one actual makefile under the top level directory, acting as the entry point. It would contain only this:

include morkfiles/main.makefile

It’s just there to allow you to conveniently type make at the command line and kick off a build of all targets, or say make clean to wipe everything previously built.

Executables

So, how would you go about writing, say, an executable?

You’d start with a subdirectory called, say, myexe, where you’ll build your first executable (hence the boring example name).

You’ll need a main source file, myexe/myexe.cpp:

#include <iostream>

int main()
{
	std::cout << "Hello from myexe" << std::endl;
}

Not exactly world-changing, but good enough for now. If you typed make at this point, it would be weird and wrong if it went ahead and assumed anything. No, first we have to undertake the arduous task of writing a morkfile, which would be saved as myexe/morkfile and would contain this:

$(this)_type = exe

Now, there’s a lot of stuff to explain there, so I’ll go slowly… wait a second, no there isn’t. It’s literally one line long. It means, unsurprisingly, “This thing is going to be an executable”.

If you run make now, a mini-miracle occurs. Somehow mork figures out what to do. And you can run your new program:

myexe/bin/myexe

or on Windows:

myexe\bin\myexe.exe

Things to note:

  • We haven’t told mork where to find the myexe directory. It automatically finds all files called morkfile. You can reorganise your source tree willy-nilly, add or remove modules, and never have to update any central configuration file.
  • Neither did we have to add myexe/myexe.cpp to a list. An executable module is simply assumed to have source files directly inside it. All files with extension .cpp are automatically compiled and linked. We could add more source files under myexe if we wanted to, re-run make and they would be added to our executable.
  • And of course, we only recompile what we need to. The second time you run make after changing nothing, it immediately reports it has nothing to do (even if you have dozens of modules, it says this very quickly, because only one instance of make is executing).

If you’ve used make before you may be wondering about the business of using the preprocessor to generate include file dependencies. Don’t worry about it; that stuff is taken care of automatically. No need to say make depends or anything like that. No stupid error messages when you delete a header. It just works.

Libraries

But real projects involve multiple targets – some are executables, some are static libraries, some are shared libraries (DLLs on Windows). These all have to depend on one another, which typically means that they have to be built in the right order, and include each other’s header files.

So let’s make a static library. Create another directory called myutil – you can put it absolutely anywhere beneath the top-level project directory (even inside myexe though that would be unnecessarily confusing, so don’t do that).

This time the source file, myutil/myutil.cpp, has no main function. Instead it exposes a function intended to be called from another module:

#include <myutil.hpp>

std::string myutil_says()
{
	return "world.";
}

We also need to write that header file. It goes in a subdirectory of the module, as myutil/include/myutil.hpp:

#ifndef MYUTIL_INCLUDED
#define MYUTIL_INCLUDED

#include <string>

std::string myutil_says();

#endif//MYUTIL_INCLUDED

That include subdirectory is where you put all headers that may need to be included by other modules – it’s the “public interface” of your module. Note that you don’t have to use the extension .hpp; the only significant thing here is the exact name and location of the include subdirectory.

Finally we have to confront that terrifying task of writing myutil/morkfile:

$(this)_type = static

Wait a second, that looks vaguely familiar! The only difference from last time is that we’re giving this module the type static instead of exe.

At this point we can run make and, no surprises, a library is built. It’s created under myutil/lib, but as we’ll see, you don’t even need to know that.

Now let’s make myexe use myutil. Back in myexe/myexe.cpp we adjust it to:

#include <iostream>

#include <myutil.hpp>

int main()
{
	std::cout << "Hello, " + myutil_says() << std::endl;
}

So myexe requires myutil. Now, mork is helpful, but not telepathic (or smart/stupid enough to try to scan your source files looking for implied dependencies between modules). You have to tell it that myexe requires myutil, by amending myexe/morkfile like so:

$(this)_type = exe
$(this)_requires = myutil

Couldn’t be simpler, could it? If your module requires another other modules, you list them in the $(this)_requires variable, separated by spaces, and mork takes care of adding myutil/include to the compiler options for myexe. It also (of course) deals with the linker options.

But it goes further – requirements are automatically transitive. That is, if module A requires module B (including some of its headers), and module B requires module C (so some of B’s headers include some of C’s headers), then it follows that A will not be able to compile successfully unless it can include headers from both B and C. But this is handled automatically: you only have to configure A to require B, and A will implicitly require C also, and so “it just works”.

Note that you only give the name, not the full path, to the required modules. Hence modules need to have unique names within the entire source tree, not just unique paths. This is a good idea anyway because when you distribute your software it’s usually convenient to put all the binaries (executables and shared libraries) in one directory, so they’ll need to have unique names.

But it has an additional nice advantage here, which is that even if you have dozens of modules that require one another in complex ways, you can still rearrange the modules under neat organisational subdirectories. It makes no difference to mork.

Dynamic Libraries

What about dynamic libraries (shared objects on Linux or DLLs on Windows)? It’s as easy as amending myutil/morkfile to:

$(this)_type = shared

That’s it. Run make again and everything is taken care of – mork even copies the built DLLs or shared objects to the bin directory of any executable that requires them, so it can be instantly executed.

Java

Oh yes, for good measure mork can build Java .jar files as well. You just have the morkfile look like this:

$(this)_type = jar

You put your Java source files in a subdirectory called src. The requires works exactly the same – it controls build order, as with native code modules, but also figures out the classpath settings.

This can be useful if you’re writing a mixed native/Java project with the emphasis on the native code. (If it was mostly Java, you’d presumably use Java build tools… by the way, I have something a little bit like mork but layered on ant, and integrating very nicely with Eclipse, so I’ll write that up sometime soon.).

Other stuff

There are more topics to consider, my favourite being code generation, which is a massively important technique, sadly underused in many projects, and many build tools don’t seem designed to support it, but mork is ready. And how do you integrate unit testing? Also very easy. But this post is long enough already.

ME WANT MORK!

I almost forgot… here’s where mork lives:

http://earwicker.com/mork

It has some examples with it that are very similar to the ones suggested here, but have more complex interdependencies. If you want to use it on Linux, it should just work, assuming you have the usual Gnu development tools.

Windows Quirks

On Windows, you need to install Cygwin with the development tools, and Visual Studio (for maximum interoperability with standard Windows libraries, mork uses Visual C++ as the compiler but relies on Cygwin for make, the bash shell and related commands – it even uses gcc, purely to get make-compatible dependencies).

So you need to have the Visual C++ tools and Cygwin accessible in your environment. The ordering of the PATH environment variable is important. Visual C++ paths need to be at the front of the PATH, so the correct (Microsoft) link.exe is used, then cygwin\bin so that the correct (Cygwin) find.exe is used, and finally the rest of your standard Windows PATH.

Next time: Part 3, in which we delve into how mork works, how to extend it, why the morkfile settings begin with $(this)_ and so on.

Advertisements
Categories: Uncategorized Tags: , , , ,
  1. No comments yet.
  1. October 9, 2011 at 2:35 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: