add-ppls [ ppl-file ... ] subtract-ppls ppl-file1 [ ppl-file2 ... ] ppl-from-log [ ppl-file ... ] hits-from-log [ ppl-file ... ] compare-ppls [ mindelta=D ] ppl-file1 ppl-file2 compute-best-mix [ lambda='l1 l2 ...' ] [ precision=P ] \ ppl-file1 [ ppl-file2 ... ] compute-best-sentence-mix [ lambda='l1 l2 ...' ] [ precision=P ] \ ppl-file1 [ ppl-file2 ... ] filter-event-counts [ order=N ] [ escape=\fIstring\P ] [ counts ... ]
add-ppls takes several ppl output files and computes an aggregate perplexity and corpus statistics. Its output is suitable for subsequent manipulation by add-ppls or subtract-ppls.
subtract-ppls similarly computes an aggregate perplexity by removing the statistics of zero or more ppl-file2 from those in ppl-file1. Its output is suitable for subsequent manipulation by add-ppls or subtract-ppls.
ppl-from-log recomputes the total perplexities and statistics from individual lines in ngram -debug 2 -ppl output. Combined with some filtering of that output this allows computing perplexities on interesting subsets of words.
hits-from-log computes N-gram hit rates from ngram -debug 2 -ppl output.
compare-ppls tallies the number of words for which two language models produce the same, higher, or lower probabilities. The input files should be ngram -debug 2 -ppl output for the two models on the same test set. The parameter D is the minimum absolute difference for two log probabilities to be considered different (the default is 0).
compute-best-mix takes the output of several ngram -debug 2 -ppl runs on the same test set and computes the optimal interpolation weights for the corresponding models, i.e., the weights that minimize the perplexity of an interpolated model. Initial weights may be specified as l1 l2 .... The computation is iterative and stops when the interpolation weights change by less than P (default 0.001).
compute-best-sentence-mix similarly optimizes the weights for sentence-level interpolation of LMs. It requires input files generated by ngram -debug 1 -ppl. (Sentence-level mixtures can be implemented using the ngram -hmm option, by constructing a suitable HMM structure.)
filter-event-counts prepares a count file for for perplexity computation. It removes counts that do not represent events to the LM. The order=N option specifies the maximal N-gram order to use. The effect of filtering is such that
ngram -order N -lm LM -ppl TEXTand
ngram-count -order N -text TEXT -write - | \ filter-event-counts order=N | \ ngram -order N -lm LM -counts -yield the same result. The escape= option specifies a string that causes all input lines beginning with that string to be passed through (useful in combination with ngram -escape).