add-ppls[ppl-file... ]subtract-pplsppl-file1[ppl-file2... ]ppl-from-log[ppl-file... ]hits-from-log[ppl-file... ]compare-ppls[mindelta=D]ppl-file1ppl-file2compute-best-mix[lambda='l1 l2...'] [precision=P] \ppl-file1[ppl-file2... ]compute-best-sentence-mix[lambda='l1 l2...'] [precision=P] \ppl-file1[ppl-file2... ]filter-event-counts[order=N] [escape=\fIstring\P ] [counts... ]

** add-ppls **
takes several ppl output files and computes an aggregate perplexity and
corpus statistics.
Its output is suitable for subsequent manipulation by
** add-ppls **
or
**subtract-ppls**.

** subtract-ppls **
similarly computes an aggregate perplexity by removing the
statistics of zero or more
* ppl-file2 *
from those in
*ppl-file1*.
Its output is suitable for subsequent manipulation by
** add-ppls **
or
**subtract-ppls**.

** ppl-from-log **
recomputes the total perplexities and statistics from individual
lines in
** ngram -debug 2 -ppl **
output.
Combined with some filtering of that output this allows computing
perplexities on interesting subsets of words.

** hits-from-log **
computes N-gram hit rates from
** ngram -debug 2 -ppl **
output.

** compare-ppls **
tallies the number of words for which two language models produce the same,
higher, or lower probabilities.
The input files should be
** ngram -debug 2 -ppl **
output for the two models on the same test set.
The parameter
* D *
is the minimum absolute difference for two log probabilities to be
considered different (the default is 0).

** compute-best-mix **
takes the output of several
** ngram -debug 2 -ppl **
runs on the same test set and computes the optimal interpolation
weights for the corresponding models,
i.e., the weights that minimize the perplexity of an interpolated model.
Initial weights may be specified as
*l1 l2 ...*.
The computation is iterative and stops when the interpolation weights
change by less than
* P *
(default 0.001).

** compute-best-sentence-mix **
similarly optimizes the weights for sentence-level interpolation of LMs.
It requires input files generated by
**ngram -debug 1 -ppl**.
(Sentence-level mixtures can be implemented using the
** ngram -hmm **
option, by constructing a suitable HMM structure.)

** filter-event-counts **
prepares a count file for for perplexity computation.
It removes counts that do not represent events to the LM.
The
**order=**N
option specifies the maximal N-gram order to use.
The effect of filtering is such that

ngram -orderandN-lmLM-pplTEXT

ngram-count -orderyield the same result. TheN-textTEXT-write - | \ filter-event-counts order=N| \ ngram -orderN-lmLM-counts -

Copyright 1995-2009 SRI International