GUB the Grand Unified Builder



directory contents and use
. (GNUmakefile, *.make) make drivers (try: make help)
bin/ programs (gub, gpkg, gib, ...; try: bin/gub --help)
gub/ python library
gub/specs/ build specifications (gub/specs/,,,
patches/ patch files used in build specifications
target/ build trees
log/ (gub.log, ...) (build) log files
uploads/ installers
target/<platform>/ working tree for <platform>
target/linux-x86/src/ linux-x86 unpacked sources
target/linux-64/build/ linux-64 build output
target/darwin-x86/log/lilypond.log darwin-x86 most recent lilypond build log
target/mingw/packages/ .gup packages for mingw
target/darwin-ppc/log/status/ status of partial darwin-ppc builds
target/freebsd-x86/root/ freebsd-x86 build and install root
target/tools/root/usr/bin/ native tools used for build
sourcefiles/ non-patch files used in build specifications
nsis/ nsis windows installer scripts


Checksums and rebuilds

GUB seeks to minimise unnecessary rebuilds while being strict about important changes. Before starting the actual building of a package, the build recipe is executed as a dry-run and a checksum is calculated from these build commands. This checksum includes the environment variables DISPLAY, FTP_PROXY, GUB_TOOLS_PREFIX, HOME, HTTP_PROXY, LIBRESTRICT, PATH [no other environment variables are allowed through].

The packages' build checksum is matched with the previous build and if it changes, the package is rebuilt. Otherwise, only the binary package is reinstalled, if necessary. This means that the .py build specs can be changed and refactored at will and as long as no actual build commands change, nothing will be rebuilt.

This makes that forcing rebuilds can can be tricky: gub refuses to do so if it deems rebuilding unnecessary.


GUB uses an ABI environment variable to work around some build bugs in some packages. This variable is not checked by the environment-changed function, so problems will probably occur if you share your GUB build directory between a 32-bit and 64-bit OS. If you change your operating system, delete the target/ directory of GUB before trying to build it again.


A common problem with [making reproducible cross] builds is using files or features that are not controlled by the [GUB] build process, but just happen to be on the build host and happen to differ between most build hosts.

GUB uses the LD_PRELOAD feature with the gub/specs/ package and code from ./librestrict/restrict.c to disallow reading from build host files. This makes for much better reprocucibility, especially between different build hosts.

The downside to this LIBRESTRICT feature is that quite a few packages are not prepared for cross building. LIBRESTRICT will catch these errors and they have to be fixed in the package's spec file.

This is what such a LIBRESTRICT assertion could like
librestrict:error:/home/janneke/vc/gub/target/mingw/root/usr/cross/libexec/gcc/i686-mingw32/4.1.1/cc1plus: tried to open () file /home/janneke/vc/lilypond/lily/
cc1plus: internal compiler error: Aborted
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
[1]10:42:08 janneke@peder:~/vc/gub
you often find these in config.log files when package-dependant code blondly looks in /usr/include or /usr/lib.


LIBRESTRICT has an experimental strict mode enabled by setting the environment variable
This will even disallow any STATting outside the target's build tree. It would be nice to get this to work and some effort has been done. However, this is potentially a lot of work as not only upstream packages but also build system authors [autoconf, automake, libtool etc.] look at the build host root file system while cross building.

Simplifying package builds or avoiding code duplication

Another idea of GUB is to identify, fix and generalise build quircks and move them from a package's spec file into one of the generic build classes. This increases maintainability by avoiding code duplication, makes spec files smaller, simpler and also increases the chances of a new package's spec file to only need a source url to specify the build.


Build package hello for platform linux-x86
bin/gub linux-x86::hello
Examine why package hello wants to rebuild
less target/linux-x86/log/build.log search from end back to `mismatch:' [type: FC-c?mismatch:RET]
Force a rebuild of package hello for platform linux-x86
rm -rf target/linux-x86/*/hello-*
Force a rebuild of package hello for all platforms
rm -rf target/*/*/hello-*
Prepare to rebuild entire linux-x86 platform
rm -rf target/linux-x86
Prepare to rebuild everything, all platforms
rm -rf target


Plain url builds

<--no spec-->

Well behaved packages can be built without spec file, straight from an url. For example, to build bison-2.2 as a build tool, do
bin/gub tools::
This produces
Check files installed under target/tools/root/ by doing
bin/gpkg -p tools files bison
To build the very latest findutils for your architecture, do
bin/gub git://

Simple spec

from gub import target

class Hello (target.AutoBuild):
    source = ''
build it for mingw by doing
bin/gub mingw::hello

__TOOLS package with dependencies

from gub import tools

class Autoconf__tools (tools.AutoBuild):
    source = ''
    parallel_build_broken = True
    dependencies = [
build it by doing
bin/gub tools::autoconf

configure flags

from gub import tools class Icoutils__tools (tools.AutoBuild): dependencies = ['libpng-devel'] configure_flags = (tools.AutoBuild.configure_flags + ' --with-libintl-prefix=%(system_prefix)s' + ' --disable-nls')

__<platform>__<arch>-specific instructions

For platform specific instructions, append "__<platform>" to the class name. For platform+architecture-specific instructions, "__<platform>__<arch>" to the class name.

from gub import target class Glib (target.AutoBuild): ... class Glib__freebsd (Glib): dependencies = Glib.dependencies + ['libiconv-devel'] configure_variables = Glib.configure_variables + ' CFLAGS=-pthread' class Glib__freebsd__x86 (Glib__freebsd): # Must include -pthread in lib flags, because our most beloved # libtool (2.2.6a) thinks it knows best and blondly strips -pthread # if it thinks it's a compile flag. # FIXME: should add fixup to update_libtool () compile_flags = ' G_THREAD_LIBS=-pthread G_THREAD_LIBS_FOR_GTHREAD=-pthread '

String expansion

In all build commands, standard Python string expansion
can be used. These strings are taken from class Settings in gub/ and from the package's build class and its parent classes.

Adding string expansion variables to a spec build class can be done in several ways

Pre-defined strings

Typical variables defined by settings are
build_architecture=x86_64-linux build_os=linux build_platform=linux-64 platform=target target_architecture=x86_64-linux target_bits=64 target_cpu=x86_64 target_os=linux target_platform=target
Typical variables defined by the build class
compile_flags= builddir=.../target/build/... config_cache_overrides= configure_binary= .../configure configure_command= sh .../configure ... configure_flags= --prefix=... --enable-shared --enable-static configure_variables= CFLAGS=-I.../target/root/usr/include install_command= install_flags= install_prefix= install_root= make_flags= srcdir=.../target/src/...

More examples

See gub/specs/*.py for some more examples.