Song to MIDI: what a full mix comes back as

A finished song does not come back as parts. It comes back as one undifferentiated list of notes, with nothing in the file that can say which instrument any of them came from. We built a 16-second song with known answers — 32 kicks, 16 snares, 64 hi-hats, 16 bass notes, 24 chord notes and 32 melody notes, so 72 written pitched notes — and ran it through this converter’s own dsp.js and midi.js. It returned 197 notes. Of those, 141 (71.6%) match no written pitch at all. The bass came back complete (16 of 16) and the melody nearly complete (30 of 32); the chords came back at 10 of 24. That holds when the bass sits in a readable register; this converter reads pitch from the whole signal and cannot pull one voice out of a mix on its own — Can audio to MIDI pull out just the bass line? covers the exact floor. The drums, which contain no pitch whatsoever, supplied most of the difference. The exported file has 2 tracks, 1 channel, 0 program-change events and 0 track names — there is no field in it where “this is the snare” could be recorded.

1. What we fed it

Six stems synthesised at 44.1 kHz, 16.0 s, 120 BPM, eight bars of 4/4. Every note has a known start time and a known pitch, so “correct” is not a matter of opinion. They were then mixed at the levels a rough demo would sit at and normalised to 0.9000 peak.

2. What came back

Of the 72 written pitched notes, here is what survived the mix:

That last column is the practical result of this whole page: the same three parts, converted one at a time, came back complete. Mixed together, 16 of them disappeared. Everything that follows is an explanation of why.

3. Where the notes that were never played land

141 of the 197 output notes match no written pitch. They are not spread evenly — they pile into the middle of the keyboard, exactly where the chords and the lower half of the melody live, which is what makes them hard to spot and tedious to delete.

Sorting them by what caused them:

They are also visibly shorter than real notes: 156 ms average for unmatched notes against 341 ms for the ones that matched. That is the single quickest way to find them in a piano roll — sort by length and the debris clusters at the short end. Here is the actual opening of the export, first four seconds, where the written music is one bass note, one three-note chord and eight melody notes:

4. There is nowhere in the file to write “this is the drum”

Separating a mix into parts would require somewhere to put the answer. We parsed the exported file event by event rather than scanning raw bytes, because a raw byte scan mistakes velocity values for status bytes. This is what the 197-note export actually contains:

One note track, one channel, no program change, no track name. The writer in midi.js emits 0x90 and 0x80 for every event — there is no code path that writes any other channel or any instrument assignment. So the question “which track is the bass?” has no answer in the file, not because the answer is wrong but because the file has no slot for it. Any separation has to happen before conversion, by converting stems, or after, by hand.

5. The drums are read as pitched notes

Each drum stem was then converted on its own. Nothing in this audio has a pitch — a hi-hat is filtered noise — and yet the converter returns notes for all three.

The kick is the most instructive one. In polyphonic mode it is read as a bass note between 73 and 87 Hz — F2, D2 and E2 — which is inside the range a bass guitar actually uses, so it does not look wrong in a piano roll. In monophonic mode all 32 kicks come back as G1, a note an octave and a half below low E on a bass. Sixty-four hi-hats produced 360 notes, most of them above C7, because a broadband burst has no fundamental and the peak picker simply finds whichever bins happen to be loudest that frame.

The three drum stems together returned only 33 notes, far fewer than the hi-hat stem’s 360 on its own. That is the loudness gate doing its job: it is computed once over the whole file as floor + (loudest frame − floor) × 0.14 at the default sensitivity, and with the kick present the floor rises far enough to discard most of the hat. Inside a full mix the gate sits higher still, which is why the same drums behave differently alone and in context.

6. Why the chords are the part that disappears

In polyphonic mode a peak must reach a share of the frame’s loudest peak before it counts: relThr = 0.14 − 0.11 × sens, which at the default sensitivity of 60 is 0.074. The chord stem is mixed at 0.42 while the kick peaks at 0.90. The chord partials are still in the spectrum; they are simply not loud enough relative to the frame’s peak to clear 7.4% of it. The bass, at 0.70, and the melody, at 0.58, clear it easily.

Two things are worth reading off this table. The chords are lost the moment anything else joins them — 24 of 24 alone, 11 of 24 as soon as the bass is under them. And the drums add 89 notes (108 → 197) without changing a single pitched recall figure. They contribute volume, not information. If you are trying to identify chords specifically — not just transcribe a mix — the chord identifier page measures what the engine finds on isolated triads, what harmonic suppression eats, and when a chord below 181 Hz stops being readable.

7. Monophonic mode gives up on a full mix entirely

Monophonic mode gates on harmonicity: the harmonic-comb score of the frame must clear 0.35 − 0.25 × sens. On the full mix it never gets near that.

Not one frame of the full mix clears the default threshold, so monophonic mode returns zero notes — not a reduced melody, an empty file. Dragging sensitivity to the top is the only way back:

If someone tells you “switch it to monophonic and it will just follow the vocal”, that only holds for a solo line with little else on it. On a full mix the noise floor from the drums drags the harmonicity score below every usable threshold.

8. What you can actually change

Sensitivity is the wrong direction. Raising it lowers all three thresholds, so more of everything gets in — including more debris. On the full mix, sensitivity 100 returned 242 notes at 76.4% unmatched, worse than the default’s 197 at 71.6%.

Between 60 ms and 200 ms the debris share falls from 71.6% to 47.1% while the melody loses exactly one note — that is the useful range. Past that the control starts eating the music: at 250 ms the melody drops to 13 of 32. Note that it never recovers the chords, because the chords are lost to the relative-to-peak threshold, not to note length.

Filtering the mix first does not rescue them either. We low-passed the whole mix with a one-pole filter before converting — at 8 kHz the output was 194 notes, 71.1% unmatched; at 800 Hz, 163 notes, 65.6% unmatched. Bass, chords and lead recall were identical at every setting. The extra notes are not simply “bright cymbals leaking through”.

The one change that measurably recovers everything is the one that happens before the converter: convert the stems separately. Bass alone 16 of 16, chords alone 24 of 24, lead alone 32 of 32 — 72 of 72, against 56 of 72 for the same music mixed.

9. How to check this on your own song

These are our numbers on a synthetic mix, not a promise about your track. Real releases add reverb, compression and stereo width, none of which we tested here. Four steps that take a couple of minutes and give you your own figures:

1. Convert a short section, not the whole track. Thirty seconds near the chorus is enough and keeps the note count readable.
2. Open the CSV, not just the MIDI. The converter exports index,midi_note,note_name,start_sec,end_sec,duration_sec,velocity. Sort by duration_sec ascending: on our mix the debris averaged 156 ms against 341 ms for real notes, so the short end of that list is where the junk collects.
3. Count notes per second. Our pitched material ran at 6.75 notes per second and the full mix at 12.31. If your export is running at double the density of the music you played, something other than the music is being detected.
4. Convert one stem and compare. If a solo instrument gives you complete, clean output and the full mix does not, the loss is happening in the mix, not in the converter’s ability to hear that instrument.

If you intend to share or release the MIDI you make from a song, the rights question rides on the source, not on the conversion — Who owns the MIDI you converted from a song sets out the plain facts.

Questions people actually ask

Can I turn one finished mix into separate MIDI tracks for each instrument?

No, not with this converter. We parsed the export event by event: format 1, 2 track chunks, 480 ticks per quarter, 197 note events, one channel (channel 0), zero program-change events, zero track-name events. The writer emits only 0x90 and 0x80. There is no field in a file like that for “this note is the bass”. If you need parts, convert the stems separately — on our test that returned 72 of 72 written notes against 56 of 72 from the mix.

Why does converting a drum track give me so many stray notes?

Because the detector looks for spectral peaks, and a drum hit is full of them. Sixty-four synthesised hi-hats returned 360 notes, most above C7; sixteen snares returned 156. Even a kick, which is nearly a sine wave, came back as F2, D2 and E2 in polyphonic mode and as 32 G1s in monophonic mode. Nothing in that audio has a pitch. In the full mix, the drums added 89 notes without improving a single pitched recall figure.

Why did my chords vanish while the bass and the melody came through?

Relative level. In polyphonic mode a peak has to reach 0.14 − 0.11 × sens of the frame’s loudest peak — 7.4% at the default sensitivity. Our chord stem sat at 0.42 against a kick peaking at 0.90, so most chord partials fell under that line: 24 of 24 when converted alone, 10 of 24 in the mix. The bass at 0.70 and the melody at 0.58 cleared it and came back at 16 of 16 and 30 of 32. Turning the shortest-note control down does not fix this; only raising the chords in the mix or converting them alone does.

Does turning the sensitivity down clean up the result?

It does the opposite on a dense mix. The slider lowers all three thresholds at once (loudness gate, harmonicity, relative peak), so more of everything gets through. On our full mix, sensitivity 0 gave 119 notes at 53.8% unmatched, the default 60 gave 197 at 71.6%, and sensitivity 100 gave 242 at 76.4%. You trade away real notes along with the junk — at sensitivity 0 the melody fell to 29 of 32 and at 300 ms shortest note the whole file collapsed to 3 notes. Use the shortest-note control instead: 60 ms → 200 ms cut the unmatched share from 71.6% to 47.1% and cost one melody note.

Should I convert the whole song in one go?

Only if you want a rough harmonic sketch and are prepared to delete a lot. Measured on our material: converting the whole mix returned 197 notes of which 141 matched nothing written, at 12.31 notes per second. Converting bass, chords and melody separately returned all 72 written notes, complete. The mix is not additive — drums alone gave 33 notes and pitched material alone gave 108, but together they gave 197, because the loudest thing in each frame re-sets the reference everything else is judged against. If the part matters, convert it on its own.

Can I clean up the stray notes by EQing or filtering the audio first?

Not usefully. We ran the full mix through a one-pole low-pass before converting: at 8 kHz it still returned 194 notes (71.1% matching nothing written), 3 kHz gave 188, 1.5 kHz gave 177, 800 Hz gave 163. The debris shrank only a little, and the pitched recall did not move at all — bass 16 of 16, chords 10 of 24 and melody 30 of 32 were identical at every cutoff. The junk is not just bright cymbal leakage you can slice off; it is how the detector turns every surviving spectral peak into a note, and that happens below the filter frequencies too. To actually lose the stray notes, convert the instrument on its own stem rather than filtering the whole mix.

Can audio to MIDI handle an orchestra recording?

It can, but an orchestral mix is exactly the dense material this converter struggles with. It holds up to six notes per frame and tends to keep the loudest lines — the melody and the bass — while quieter inner voices and percussion fall away. On a full mix we measured the converter returning many more notes than were written, most of them unmatched to anything played. For an orchestral source, convert sections or separate stems one at a time to recover the parts that matter.

Can audio to MIDI separate multiple instruments at once?

It hears several instruments together, up to six voices at a time, but it does not assign notes to instruments. A full mix is not additive: the loudest part in each frame sets the reference everything else is judged against, so quieter instruments get buried under the dominant one. We found converting a whole mix returned far more notes than were written, almost none of them matching, while converting the parts separately recovered every note. For clean results, separate the instruments onto stems first.

Related guides

• How many notes a converter can hold at once — the six-voice ceiling and the harmonic suppression rule.
• What a full-length soundtrack comes back as — how file length changes the gate.
• What a bend does to your notes — measured on pitch-moving material.
• What is actually inside a MIDI file — the bytes, offset by offset.
• Convert a file now and read the CSV — every number on this page can be checked against your own export.
• Mono vs poly — What each detection mode misses
• Pitch bend detection — What a glide adds to your MIDI, and when it matters
• Sample rate & MIDI — Whether 44.1k, 48k or 22.05k changes the notes
• Clean up a converted MIDI — The 5 passes that turn a raw export into a usable part