Vocal to MIDI: why lyrics don't become notes cleanly

The short answer: because the converter never hears words. It measures two things per frame — a pitch and a loudness — and it re-measures them every 11.610 ms. There is no phoneme model, no syllable model and no text output anywhere in the chain: when we wrote a two-note .mid with this site’s own writer, the file came to 71 bytes and contained exactly three meta events — tempo, time signature and end-of-track. No lyric event, no text event, no track name. So the words cannot be in the result, even in principle. What survives is the part of the voice that behaves like a pitch: the vowels and other voiced sounds. The rest is treated as noise or as silence. A 200 ms “s” measured 0 notes in monophonic mode and 14 spurious notes in polyphonic mode, all of them in the top two octaves. One syllable can give you at most one note, several syllables can collapse into one, and the lyric is gone.

1. Two numbers per frame, and nothing else

Audio is resampled to 22 050 Hz and cut into 2048-sample windows stepped forward 256 samples at a time. Each frame is 11.610 ms long, which is 86.13 frames a second. Inside a frame the analyser does one of two things depending on the mode:

• Monophonic: scores every candidate fundamental by summing its harmonics — up to 8, weighted by 1/k^0.7 — and keeps the winner. Then it tests that winner against two gates: a loudness gate and a harmonicity gate.
• Polyphonic: keeps up to 6 spectral peaks per frame, discarding any peak that is an integer multiple (within 5%) of a stronger one.

Neither path has any notion of a phone, a syllable, a word or a phrase. A sung sentence and a bowed cello line are the same kind of input to this code: a series of frames with a pitch and a level. That is the root of every failure below — not a bug, a scope.

2. The exported file contains no text at all

This is worth stating separately, because it settles the question. The MIDI writer builds two tracks: one holding tempo and time signature, one holding note-on and note-off events. We dumped the bytes of a two-note export at 120 BPM:

The whole file was 71 bytes for two notes. A lyric meta event is a standard part of the MIDI specification and a karaoke file uses it, so this is not a format limitation — the writer simply never emits one. If you need the words attached to the notes, you add them in a DAW on a lyric or text track after the import. Nothing in the conversion will supply them. What is actually inside a MIDI file covers the rest of that structure.

3. Voiced and unvoiced sounds are opposites to this engine

Speech sounds split into two families, and the split maps exactly onto what the analyser can and cannot use.

The fricative row is the one that surprises people. A 200 ms noise burst at a moderate level returned zero notes in monophonic mode on all three test seeds, because broadband noise has no harmonic series to sum and the harmonicity gate rejects the frame. The same burst in polyphonic mode returned 14, 14 and 15 notes across those three seeds — and because noise peaks land wherever the spectrum happens to be loudest, they scattered across the top two octaves, from D4 up to B7. Nothing about that cluster is stable between runs.

4. In polyphonic mode, a longer consonant means more ghost notes

We held the fricative level fixed and varied only its length. The count of spurious notes rose with it:

So the choice of mode decides whether your consonants are silent or loud and wrong. For a solo vocal take, monophonic mode is the one that keeps consonants out of the result. If you use polyphonic mode on a voice, expect to delete a spray of high notes on every sibilant — and expect the exact pitches to differ each time you convert, because they come from noise, not from the singer.

5. Why one syllable is not one note

The intuitive model — each syllable becomes a note — is wrong in both directions: some syllables produce nothing, and some sequences of syllables produce a single note.

A vowel has to last about 0.12 s

Holding everything else constant and shortening a vowel, the output falls off a cliff rather than fading:

The shortest-note control was set to 60 ms, which the code turns into 5 frames — 58.05 ms — so the 100 ms vowel should have cleared it. It did not, because the frames at each end of a short vowel are still ramping up and down, and the ones that survive fall below the minimum frame count. The practical floor is about 0.12 s of steady vowel. That sets a ceiling on syllable rate: at 0.12 s per vowel plus 0.12 s of consonant, a syllable occupies 0.24 s, so roughly four syllables a second is the most a lyric can be sung and still be transcribed. A fast phrase in a pop vocal runs above that, and the fast syllables simply do not appear.

Vowels on a harmonic series collapse into one note

This one is counter-intuitive and it is the strongest reason a sung line comes back shorter than it was sung. We sang four vowels on C4, E4, G4 and C5 — a C major arpeggio — and monophonic mode returned exactly one note, C4. It did so whether the vowels were contiguous, whether each was followed by a 120 ms fricative, and whether each was followed by a 150 ms gap. The consonants and the gaps changed nothing.

The cause is in the scoring. E4, G4 and C5 are the 5th, 3rd and 2nd harmonics of C4, so a frame containing only E4 still scores C4 highly, because C4’s harmonic comb covers that frequency. The common fundamental keeps winning. Change the pitches to ones that are not harmonics of each other — C4, F#4, B4, D#5 — and the same test returns three notes instead of one. The collapse is a property of the pitch set, not of the singing.

Practical reading: melodies built on wide, non-harmonic intervals transcribe far better than melodies that outline a chord. An arpeggiated vocal hook is close to the worst case for monophonic mode.

6. A vowel can vanish without being quiet

The harmonicity gate is not about level. A frame must score above 0.35 − 0.25 × sensitivity, so at the default sensitivity of 60 the threshold is 0.200. We measured what a clean synthesised vowel actually scores:

Three of those five vowels produce no note at all at the default setting, and they are not the quiet ones or the badly sung ones — they are simply the ones whose harmonic comb happens to score just under the line. Raising sensitivity to 70 recovers F#4 and G4; F4 needs 80. That is the single most useful troubleshooting step for a vocal that comes back with holes in it, and it is why the fix is a slider rather than a re-record.

One caveat, stated plainly: these scores come from synthesised vowel-like tones whose harmonic balance we control exactly, not from recordings of singers. A real voice has breath noise, moving formants and a different harmonic rolloff, so its score moves around more. What the numbers pin down is the mechanism — a fixed gate against a score that sits near it — and the reason the same phrase can lose one note and keep its neighbours.

7. Two voices at once produce nothing in monophonic mode

If you sing a harmony with yourself, or a second voice is present in the recording, monophonic mode does not pick the louder one and carry on. It returns zero notes. Two fundamentals in the same frame break the harmonic-comb assumption badly enough that the frame fails the gate entirely. Polyphonic mode on the same audio returns the two pitches as separate notes — G4 and C5 in our test — which is the correct answer, and the reason poly mode exists.

That gives a clean rule: one voice, monophonic; two or more voices, polyphonic and expect to clean up. The middle case — one voice with reverb, or one voice over a quiet instrument — is where people get caught, because it looks like one voice but behaves like two. Monophonic vs polyphonic works through that decision in more detail.

8. What actually helps

• Sing the vowels long. 0.12 s is the floor for a vowel to register at all; 0.2 s and up is comfortable. If a phrase must be fast, accept that the fast syllables are gone and add them by hand.
• Use monophonic mode for a solo voice. It is the mode that keeps a 200 ms “s” out of the file. Polyphonic mode will turn every sibilant into a cluster of high notes that you then have to delete.
• Delete everything above the vocal range in poly mode. Ghost notes from consonants and formants land high. If you sang below C6, everything above it is almost certainly debris.
• Raise sensitivity before you re-record. A missing vowel is usually a gate problem, not a performance problem. Try 70 and 80 and watch which notes appear; the ones that appear at 80 were sitting just under the line.
• Avoid arpeggiated melodies if you can choose the take. Vowels outlining a chord collapse to the root. Stepwise lines on non-harmonic intervals survive far better.
• Add the lyric yourself. No text is written, so a lyric track in the DAW is the only way to attach words. Do it after import, not before.
• Expect to clean up. Even a good vocal conversion needs the velocity pass and the ghost-note pass; how to clean up a converted MIDI lists them in order.

Two related failures are covered on their own pages: a note that comes back exactly an octave high is a harmonic-sum artefact, explained in why a sung note comes back an octave up, and the loudness-to-velocity mapping is in what a single-line converter can and cannot follow. If there are no words at all — just a hummed melody — humming to MIDI is the closer match.

Convert a vocal take now — it runs in your browser, nothing is uploaded, and every threshold above is yours to reproduce on your own file.

Related guides

• Vocal to MIDIWhy a sung note comes back an octave up
• Voice to MIDIWhat a single-line converter can and cannot follow
• Humming to MIDIWhat wordless vocals come back as
• Monophonic vs polyphonicWhich mode to use, and what each one costs
• MIDI converterWhat is actually inside a MIDI file
• Clean up a converted MIDIThe 5 passes that turn a raw export into a usable part
• MP3 to MIDIWhy MP3 loses notes that WAV keeps
• Microphone to MIDIWhat this converter does with a recording
• Live vs studio audioWhich recording transcribes cleaner, measured