To account for the hidden DF events potentially occurring between any
two words, a forward computation is carried out to find
the probability of a sentence prefix
.
Conditional word probabilities are then computed as
If the underlying N-gram model is a trigram, it is sufficient to keep
eight states for each word position, according to whether the DF prior to
was NODF (none), FP (filled pause), SDEL,
DEL1, DEL2, REP1, REP2, or the second position
after a REP2 event. To illustrate, the partial computation
involving just the NODF and REP1 states is shown here.
where
if
, and 0 otherwise.
Trigram probabilities are denoted by
;
these are obtained through the usual backoff procedure [5].
The total prefix probability is then computed as
where X ranges over the hidden states representing the disfluency types (including NODF).