Last week we stopped because we realized we needed more analysis done on more search lengths to provide a bit more certainty to our tentative conclusions. Our initial set of runs happened to settle on an inflection at our last search length of 20 tuples. This week we run the search algorithm with 22-, 24-, 26-, 28-, and 30-tuple lengths. If Mixed continues downward while Combinatorial crosses Bottleneck and keeps rising, then the little decline at 20 isn’t noise at all. It marks the turning point in the mechanism transition, which is considerably more interesting than a monotonically increasing Mixed curve.
The bottom line from the latest run is that the earlier result was not a plateau. The combinatorial mechanism keeps strengthening and becomes dominant:
|
Requested tuples (search length) |
Qualified tunes |
Zero inter-section |
Zero % |
Bottleneck % of zeroes |
Robust combina-torial % |
Mixed % |
|
20 |
21,398 |
7,752 |
36.23% |
43.61% |
41.83% |
14.55% |
|
22 |
20,724 |
8,301 |
40.06% |
39.93% |
45.83% |
14.24% |
|
24 |
20,032 |
8,744 |
43.65% |
35.81% |
49.84% |
14.35% |
|
26 |
19,331 |
9,065 |
46.89% |
32.83% |
52.95% |
14.22% |
|
28 |
18,482 |
9,253 |
50.06% |
30.82% |
55.43% |
13.75% |
|
30 |
17,476 |
9,273 |
53.06% |
28.24% |
58.48% |
13.28% |
Let’s look at a few points that jump out from this table of results.
Crossing the 50 Percent Threshold
At 28 tuples, 50.06 percent of all qualifying passages have zero intersection with another tune. At 30, it is 53.06 percent. We’re asking the question whether some other tune contains every distinct pitch/duration tuple occurring in the selected passage. Yet by about 28 or 30 events, a majority of passages contain a tuple vocabulary that no other tune in this large folk corpus can collectively supply.
The paired analysis confirms that this isn’t primarily a changing-population artifact. Using exactly the same 17,476 tunes capable of reaching length 30, zero intersection progresses:
20 → 35.43%
22 → 39.11%
24 → 42.81%
26 → 46.21%
28 → 49.61%
30 → 53.06%
That is clearly a monotonic curve. The all-qualified and paired curves are remarkably close.
The Reason Passages Become Unique Changes
At length 20, bottleneck and combinatorial explanations are almost tied: 43.6 percent bottleneck vs. 41.8 percent robust combinatorial. By length 22, they cross: 39.9 percent bottleneck vs. 45.8 percent combinatorial.
And thereafter they separate steadily. By 30, 28.2 percent bottleneck vs. 58.5 percent robust combinatorial. So among zero-intersection passages at length 30, more than twice as many are robustly combinatorial as are explained by the rarest-tuple bottleneck.
That substantially strengthens our interpretation of the Search Engine Paradox. If zeroes were principally caused by one peculiar musical event regardless of search length, the experiment would essentially say, “Rare things are rare.” But at 30 tuples, almost 60 percent of the zeroes remain zero even after any single constituent tuple is removed. No individual tuple is responsible for the uniqueness. It is the combination of all the search tuples that makes them unique.
The explanatory power of the rarest individual tuple is deteriorates as passage length increases, while the power of the combinatorial explanation grows.
Clear Musical Interpretation
We can now state our emerging result more strongly than last week: Melodic individuality appears to arise primarily from combinations of common musical materials rather than from the use of individually unusual or rare note combinations.
Even zero-intersection passages are composed of tuples that, considered individually, occur extremely often in the corpus. What becomes rare is their joint presence in another tune.
That is the paradox in its cleanest form: The parts are common. The whole is rare.
Importantly, we are not requiring the parts to appear in the same order. The search deliberately throws away ordering and asks only whether another tune possesses the same collection of tuple types. Yet uniqueness still emerges.
Therefore, actual sequential melody should be more distinctive than this experiment measures. This is a deliberately conservative test of melodic individuality.
What the Search Engine Paradox Shows
The evidence points to three tranches:
- At short lengths, 4 to 8 tuples, the musical search space is saturated. Almost all search terms are drawing from the same small collection of interval/rhythm relationships, so intersections are abundant. When uniqueness occurs, it usually results from a rare tuple.
- At intermediate lengths, roughly where our earlier 10- to 20-length experiment lived, the two forces compete. Rare events still matter, but ordinary elements are beginning to combine into configurations that aren’t replicated elsewhere.
-
By 22 to 30 events, we enter a genuinely combinatorial regime. Increasing passage length isn’t simply increasing the opportunity to encounter a rare feature. It is assembling enough individually ordinary features that their conjunction becomes distinctive.
Let’s be clear on what we’re not saying. One early hypothesis was that long passages become unique because you eventually hit one weird interval/rhythm pair. The data say no. At 30 tuples, that explanation accounts for only 28 percent of zeroes (no tunes with the same tuples), while almost 59 percent survive removal of every individual tuple. That is a qualitatively different phenomenon.
Let’s tie all this together with a final graphic that visualizes the results of this search analysis:
Here we clearly see the crossover around 21-tuple length search terms. At that length and beyond the Combinatorial explanation holds the majority of searches, and below that point the Bottleneck explanation is dominant.
This resolves the Search Engine Paradox. Next week we turn to a new topic: Cluster analysis.