Disfluency types

Following [9], DFs can be classified based on how the actual utterance must be modified to obtain the intended fluent utterance, i.e., the utterance a speaker would produce if asked to repeat his or her utterance. The types can be characterized by the type of editing required.

Filled pauses (FP)

The pause filler (typically ``uh'' or ``um'') must be excised.

SHE UH GOT REAL LUCKY THOUGH
--> SHE GOT REAL LUCKY THOUGH

Repetitions (REP)

Contiguous repeated words must be removed.

IT'S A IT'S A FAIRLY LARGE COMMUNITY
--> IT'S A FAIRLY LARGE COMMUNITY

Deletions (DEL)

Words without correspondence in the repaired word sequence must be deleted.

I DID YOU HAPPEN TO SEE ...
--> DID YOU HAPPEN TO SEE ...

We know from prior work [9] that these three types of DF are the most frequent across a variety of spontaneous speech corpora, accounting for over 85% of DF tokens in the Switchboard corpus. See [9] for a description of other, less frequent, types of DF that are not modeled explicitly in our LM. For example, we are not modeling word substitutions or speech errors.