[ << Regression tests ]	[Top][Contents]	[ Programming work >> ]
[ < Pixel-based regtest comparison ]	[ Up : Regression tests ]	[ MusicXML tests > ]

---

9.6 Finding the cause of a regression

Git has special functionality to help tracking down the exact commit which causes a problem. See the git manual page for git bisect. This is a job that non-programmers can do, although it requires familiarity with git, ability to compile LilyPond, and generally a fair amount of technical knowledge. A brief summary is given below, but you may need to consult other documentation for in-depth explanations.

Even if you are not familiar with git or are not able to compile LilyPond you can still help to narrow down the cause of a regression simply by downloading the binary releases of different LilyPond versions and testing them for the regression. Knowing which version of LilyPond first exhibited the regression is helpful to a developer as it shortens the git bisect procedure.

Once a problematic commit is identified, the programmers’ job is much easier. In fact, for most regression bugs, the majority of the time is spent simply finding the problematic commit.

More information is in Regression tests.

git bisect setup

We need to set up the bisect for each problem we want to investigate.

Suppose we have an input file which compiled in version 2.13.32, but fails in version 2.13.38 and above.

1. Begin the process:

   git bisect start
   

2. Give it the earliest known bad tag:

   git bisect bad release/2.13.38-1
   

   (you can see tags with: git tag )

3. Give it the latest known good tag:

   git bisect good release/2.13.32-1
   

   You should now see something like:

   Bisecting: 195 revisions left to test after this (roughly 8 steps)
   [b17e2f3d7a5853a30f7d5a3cdc6b5079e77a3d2a] Web: Announcement
   update for the new “LilyPond Report”.
   

git bisect actual

1. Compile the source:

   make
   

2. Test your input file:

   out/bin/lilypond test.ly
   

3. Test results?
   • Does it crash, or is the output bad? If so:

     git bisect bad
     

   • Does your input file produce good output? If so:

     git bisect good
     

4. Once the exact problem commit has been identified, git will inform you with a message like:

   6d28aebbaaab1be9961a00bf15a1ef93acb91e30 is the first bad commit
   %%% ... blah blah blah ...
   

   If there is still a range of commits, then git will automatically select a new version for you to test. Go to step #1.

Recommendation: use two terminal windows

• One window is open to the build/ directory, and alternates between these commands:

  make
  out/bin/lilypond test.ly
  

• One window is open to the top source directory, and alternates between these commands:

  git bisect good
  git bisect bad