Factored LMs and interpolated models

[Tanel Alumäe]

> Let me know if this helps or if I have misunderstood your question...
> 

Hello,

First, thanks to everybody for help.

My goal was, as Katrin correctly assumed, "to interpolate a
traditional class-based model and a standard n-gram model but you want
to express this within a single FLM file". This is currently not
possible, but it's not very important because I learned that I can
use:

ngram -factored -lm <FLM1> -mix-lm <FLM2>

The above really works.

Still, I noticed a strange thing with perplexity calculation. Namely,
the perplexity figures calculated by fngram and ngram are slightly
different.  I used the following options and got following results:

fngram -ppl <testtext> -factor-file tmp/fngram_m.conf

Result: 
61 sentences, 1009 words, 26 OOVs 
0 zeroprobs, logprob= -2760.87 ppl= 441.076 ppl1= 643.604

ngram -factored -ppl <testtext> -lm tmp/fngram_m.conf 61 sentences, 1009
words, 

Result:
26 OOVs 0 zeroprobs, logprob= -2761.16 ppl= 441.359 ppl1= 644.042


-- 

The above is for a FLM that in fact is standard word trigram. The
difference is very small.

However, when I test a FLM that is a word-given-two-previous-classes
trigram, the difference is much larger:

fngram -ppl <testtext> -factor-file tmp/fngram_c.conf 

61 sentences, 1009 words, 26 OOVs 
0 zeroprobs, logprob= - 2826.73 ppl= 510.034 ppl1= 750.963

And the same with ngram:

ngram -factored -lm tmp/fngram_c.conf -ppl <testtext>

61 sentences, 1009 words, 26 OOVs 
0 zeroprobs, logprob= -2863.71 ppl= 553.378 ppl1= 818.917


As you see, here the difference (ppl1= 750 vs 818) is significant. Could
this be a configuration issue, a bug or have I understood smth wrong?

Regards,

Tanel Alumäe