[ << Regression tests ][Top][Contents][ Programming work >> ]
[ < Compiling regression tests ][ Up : Regression tests ][]

9.4 Regtest comparison

Before modified code is committed to master, a regression test comparison must be completed to ensure that the changes have not caused problems with previously working code. The comparison is made automatically upon compiling the regression test suite twice.

  1. Before making changes to the code, establish a baseline for the comparison by checking out the current git master, going to the ‘$LILYPOND_GIT/build/’ directory and running: 
    make clean # whenever any files in mf/ are modified
make test-baseline
  2. Make your changes, or apply the patch(es) to consider.
  3. Check for unintentional changes to the regtests: 
    make check

    After this has finished, a regression test comparison will be available (relative to the current ‘build/’ directory) at: 

    out/test-results/index.html

    For each regression test that differs between the baseline and the changed code, a regression test entry will be displayed. Ideally, the only changes would be the changes that you were working on. If regressions are introduced, they must be fixed before committing the code.

  4. If you are happy with the results, then skip to the final step.

    If you want to continue programming, then make any additional code changes, and continue.

  5. Finally, you should verify that make doc completes successfully.

Advanced note: Once a test baseline has been established, there is no need to run it again unless git master changed. In other words, if you work with several branches and want to do regtests comparison for all of them, you can make test-baseline with git master, checkout some branch, make check it, then switch to another branch, make test-clean and make check it without doing make test-baseline again.

[ << Regression tests ][Top][Contents][ Programming work >> ]
[ < Compiling regression tests ][ Up : Regression tests ][]

<< Back to Documentation Index

LilyPond — Contributor’s Guide v2.23.82 (development-branch).