4. GOP2-3 - GLISS

Summary

Let’s start stabilizing portions of the LilyPond input syntax. We will guarantee that selected elements of the syntax will not change (even with convert-ly) during the 3.x releases. This will be a slow process, and the first phase (2012) will not even cover the entire “single staff notation” section in the tutorial.

Motivation

Some “computer languages” are fairly stable. A TeX or C++ program written 10 years ago will probably still compile with no modifications (notwithstanding the g++ 4.3 header and namespace changes). The same is not true of LilyPond; even after using convert-ly, there are still bits that require manual updating.

Given that, LilyPond is not suitable as an archival format for music. It can produce a great pdf when you first write the file, but the .ly files require regular maintenance if you want them to compile in the latest stable version of lilypond. This is a problem for projects such as mutopia – a large fraction of their .ly files don’t compile with current lilypond. That means that they can’t benefit from recent bugfixes; users wanting the sheet music in a different size (say, printing a choral score as an A5 booklet) must delve into the ly code and make manual changes.

A stable input syntax should also make it easier to write converters to/from lilypond, and should also make it easier to write GUIs for lilypond. Currently, any program which exports lilypond code needs to support multiple versions (e.g., 2.12 vs. 2.16). Hopefully making it easier to output lilypond code will lead to more/better programs which do this, either in terms of converting from alternate formats into lilypond, or in terms of GUIs calling lilypond as the backend.

On a personal note, this is one of the biggest reasons I’ve given up on using lilypond personally. From 2001 to 2004 I got a minor in music composition. I carefully kept all my .ly files but foolishly did not preserve the pdfs. And now, 10 years later, I’m left with a bunch of music that I cannot generate sheet music for. It’s true that I could dig out old lilypond binaries to process the ly files (and I’ll probably tdo that at some point), but it would be much nicer if I could benefit from the past ten years of bugfixes in lilypond. Manually updating the .ly files would take hours or days; I’ve started this process a few times but always lost interest after a few files, since there’s no guarantee that I wouldn’t need to go through the same process in another few years.

Why disallow convert-ly?

• it forces us to take the process seriously by removing the "safety net". Any poor decisions from the process will be enthroned in the syntax for years to come[1]. Hopefully this will make us proceed cautiously, take a more serious look at the syntax proposals for potential problems, etc.
• it signals to other projects that we’re serious about this. This makes tasks such as writing importers/exporters to/from lilypond much less undesirable. It also might help people doing musicology (or music theory) research with lilypond files.
• it makes lilypond more suited to being an "archival" format (or at least less unsuited). convert-ly only converts files in a forward direction. Granted, there aren’t many instances where somebody might have a corpus of music they want to render in both lilypond 3.0 and 3.2, but it’s not impossible. For example, suppose there was a team of a dozen Russian musicologists archiving folk tunes, but lilypond 3.2 doesn’t work on OSX 11.4 because Apple broke their own API again. It would be nice if the team could share lilypond files between lilypond 3.0 and 3.2. (assuming that there were no special tweaks happening – i.e. the team was first getting the notes and rhythms written down, and are not planning to do a great deal of tweaking).

Will this help the parser?

Straightening out the parser is going to lead to some breakage in complicated and/or badly written scores. That may lead to some understandable frustration from some users, but if we’re running GLISS at the same time, that gives them some hope that things will settle down. Also, simply discussing the notation we wish to support will give rise to questions about precisly what the current system already supports, and can clarify our thoughts on it.

Not necessarily any changes

GLISS will not necessarily involve any change of notation; in fact, the first portion of “syntax stabilization” could just end up approving the existing syntax exactly as it stands. I think we should discuss each notation element separately without simply rubber-stamping the existing syntax. If there are any changes in the basic notation, then of course it would be extremely bad if convert-ly couldn’t handle it. But the end result of such discussions could lead to the conclusion that any disadvantages of the current notation outweigh the pain of changing. The important thing is informing users (and programmers) about what elements are guaranteed to work for every 3.x release.

Subset, not complete definitions

When we discuss something like accidentals, we’re not committing that the finalized syntax will be the only way of achieving the relevant goal. For example, if we standardize on cis to indicate a C sharp, then at a later date we can still introduce syntax such as c+1/2 to indicate the same C sharp (the latter possibilities are to allow more greater flexibility for microtonal notation).

Multiple rounds of standardization

It’s really easy to under-estimate the work that goes into such discussions and implementing the changes. I think we should start very small and expand gradually. At the present time, we will decide on what to do in the first round only, but I’ll add a very tentative suggestions for rounds 2. There will be another GOP discussion to settle on the exact range of notation tackled in round 2.

The basic idea is to spend approximately 3 months discussing some changes, then spend a few months implementing the changes (with no syntax discussions). Then we’ll have a stable release, and wait for at least 6 months to see of there’s any complaints with the input syntax; if there’s no complaints, then we’ll declare that part of the input syntax to be “finalized”.

• Round 1: note pitches (absolute and relative, common 12-tone accidentals only), durations (powers-of-two and non-nested tuplets), key signatures, time signatures (non-compound), bar checks, dynamics, partial measures and grace notes. Also setting a title and composer, and outputting in pdf and midi. Expected to be implemented by the end of 2012 and finalized by the end of 2013.
• Round 2: ties, slurs, articulations, adding text, manual beaming, chords, multiple voices on a staff, multiple staves, lyrics, variables/identifiers.

Subset for first phase

In greater detail: I’m suggesting that we have multiple rounds of syntax stabilization. The proposed elements of current lilypond notation which we will stabilize is captured by these two files:

\version "2.16.0"
\header {
  title = "don't overwrite this title"
  composer = "the lilypond GLISS team"
}
\score {
  \new Staff {
    \new Voice {
      \partial 8 d8 |
      c4 d' e, f'' |
      \times 2/3 {a4 b c} \grace {d16} d2 |
      \acciaccatura {b16} c2 \appoggiatura {b16} c2 |
    }
  }
  \layout {}
  \midi {}
}

\version "2.16.0"
\score {
  \new Staff {
    \new Voice {
      \relative c, {
        \clef "bass"
        \time 3/4
        \tempo "Andante" 4 = 120
        c2\mp e8 c' |
        g'2. |
        \time 6/8
        \key d \major
        \tempo "Allegro" 4. = 120
        f4.\f eisis8 eis r |
        deses,8 des r e'8 c c,8 |
      }
    }
  }
}

and then we guarantee that these files will compile, completely unmodified (no convert-ly allowed), for every lilypond 3.x version. This seem like a really small step, but I really don’t think that we can overestimate how much time, energy, and arguments this will require.

Example questions

Here’s a few sample questions that we’d encounter even with a really small subset.

PLEASE DO NOT DISCUSS THESE RIGHT NOW.

• do we keep dutch as the default language, or switch to english?
• do we finally make that \times -> \tuplet change that’s been discussed for years?
• \score \staff vs. \new score \new staff.
• what’s the canonical input structure? what shorthands do we commit to supporting?

This document was generated by Graham Percival on September 22, 2012 using texi2html 1.82.