9.5 Pixel-based regtest comparison

As an alternative to the make test method for regtest checking (which relies upon .signature files created by a LilyPond run and which describe the placing of grobs) there is a script which compares the output of two LilyPond versions pixel-by-pixel. To use this, start by checking out the version of LilyPond you want to use as a baseline, and run make. Then, do the following:

cd $LILYPOND_GIT/scripts/auxiliar/
./make-regtest-pngs.sh -j9 -o

The -j9 option tells the script to use 9 CPUs to create the images - change this to your own CPU count+1. -o means this is the "old" version. This will create images of all the regtests in


Now checkout the version you want to compare with the baseline. Run make again to recreate the LilyPond binary. Then, do the following:

cd $LILYPOND_GIT/scripts/auxiliar/
./make-regtest-pngs.sh -j9 -n

The -n option tells the script to make a "new" version of the images. They are created in


Once the new images have been created, the script compares the old images with the new ones pixel-by-pixel and prints a list of the different images to the terminal, together with a count of how many differences were found. The results of the checks are in


To check for differences, browse that directory with an image viewer. Differences are shown in red. Be aware that some images with complex fonts or spacing annotations always display a few minor differences. These can safely be ignored.

LilyPond Contributor’s Guide v2.25.16 (development-branch).