As a point of vocabulary, we often use “rests” and “rest events” interchangeably. A rest event can either be a single rest, such as a quarter note rest, or a series of rests, such as a quarter note rest followed by one or more rests of any duration. If there is a note between two rests, they would be counted as two rests or two rest events.
A respectable argument can be made that we should not include rests when training an AI model on music.
Arguments Against Including Rests
First, melodies are defined primarily by pitch and rhythm, not silence. A melody remains recognizable if every rest is removed and the notes are played legato. “Happy Birthday” is still identifiable if you stretch out the note to fill the duration of any rest that follows it. Removing pitch or duration, however, and the melody quickly becomes unrecognizable. So if one focuses on melody only, rests are not that important.
Second, rests are often editorial. In published notated music, rests frequently reflect breathing for singers or wind and brass players, bowing for string instruments, phrasing hints, engraving style, pickup conventions, and splitting long notes across barlines. Indeed, different editions and different publishers often place rests differently while preserving the same melody.
Third, rests introduce unnecessary vocabulary. Every rest requires additional token types to include rest durations, note-to-test transitions, rest-to-note transitions, and leading or trailing rests. These increases vocabulary size without adding new melodic intervals.
Fourth, rests create statistical sparsity. Instead of learning that an A transitions to a B that then transitions to a C, the model learns that an A transitions to a rest, that rest transitions to B, and B transitions to C. You need more tokens for that, and if the rest is merely a place for a singer to take a breath, it adds no new information.
Fifth, phrase boundaries can be learned from cadence, so the rests that provide that service are not necessary. Good melodies often imply phrase endings without explicit rests. For example, G A B C followed by E F G may naturally sound like two phrases simply because of harmonic resolution. Transformers are capable of learning these boundaries from context. A rest between the C and E is not necessary if that’s its only purpose
Sixth, to the extent rests act like stylistic markers, we have already made plenty of decisions to discard many of them. For instance, we ignore (that is, we have no representation in the database for) the different spellings of the same note (C# vs Db, aka enharmonic choice), ties vs. dotted notes, stem direction, beams, and slurs. We did that because they don’t change the sound of the melody, merely its representation on the page, and in some cases its style. Ignoring rests would be philosophically consistent with those choices.
Frequency of Rest Events
There are counters to at least some of these arguments, but before we get into them, let’s see whether rests are even numerous enough to worry about. First, internal rests (that is, any rest that is not a leading or trailing rest) are relatively uncommon. Internal rests number about 145,000 out of a corpus of 4.9 million notes, so around three percent.
Given that 97 percent of what happens in the database is something other than rests, and assuming any decent AI model would learn that, the model should generate rests only occasionally. We would want to make sure that any music generated as a result of the AI’s learning produces tunes with roughly the same frequency of rests.
Almost half the tunes in the database contain at least one rest. So while resets are relatively uncommon with respect to the number of note events, they are common when considered from a tune perspective. The number of tunes with rests are about 37, 540 out of 83,400 tunes, so around 45 percent of them have at least one rest. In other words, many tunes contain rests, but most tunes contain only a few rests if any.
Another way to consider the frequency is to note that the median internal rests per tune is 0.5 (the median is the number such that half the tunes have a greater number of rests than the median, and half have fewer). The total distribution of rests looks like this:
- 50 percent of tunes have no internal rests
- 40 percent have 1–5 rests
- 5 percent have 6–7 rests
- 4 percent have 8–17 rests
- 1 percent have more than 17 rests
That’s a highly skewed distribution concentrated near zero. From a machine-learning standpoint, this means the model spends most of its time learning from melodies that never encounter a rest, yet it must nonetheless allocate parameters and vocabulary to model the complexities that rests interject into the melody. All that for an event that is absent from over half the training examples, and is a low single-digit percentage of all events in the database.
Next week we’ll conclude the discussion on how to treat rests by finishing up the discussion on rest durations, provide arguments for including rests, and reach a decision on how to treat them in the database.