<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><DIV></DIV>
<DIV><BR> </DIV>
<DIV style="FONT-FAMILY: times new roman, new york, times, serif; FONT-SIZE: 12pt">
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman">Hi,</FONT></FONT><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman"></FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman">Thanks for your reply.</FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman">I need to compare two lm file by perplexity evaluation.</FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman"></FONT> </DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman">1. i) ngram -lm general.lm -lambda .5 -mix-lm l1.lm -ppl test1.txt </FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman"> ii) ngram -lm general.lm -lambda .5 -mix-lm l1.lm -ppl test1.txt -bayes 0</FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman"> in both commands it gives same perplexity </FONT></FONT><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman">but when</FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman">2. i) ngram -lm general.lm -lambda .5 -mix-lm l2.lm -ppl test1.txt </FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman"> ppl=460</FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman"> ii)ngram -lm general.lm -lambda .5 -mix-lm l2.lm -ppl test1.txt -bayes 0</FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman"> ppl=148</FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman"> the 2(ii) command gives lower perplexity.</FONT></FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman"></FONT> </DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman">could you please tell me why the second one gives lower perplexity? </FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman"></FONT> </DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman">thanks</FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=3 face="Times New Roman">akmal</FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><FONT size=2 face=Tahoma><FONT size=3 face="Times New Roman"> </FONT></DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px">
<HR SIZE=1>
</DIV>
<DIV style="FONT-FAMILY: arial, helvetica, sans-serif; FONT-SIZE: 13px"><B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Andreas Stolcke <stolcke@speech.sri.com><BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> Md. Akmal Haidar <akmalcuet00@yahoo.com><BR><B><SPAN style="FONT-WEIGHT: bold">Cc:</SPAN></B> srilm-user <srilm-user@speech.sri.com><BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Friday, August 28, 2009 1:39:45 PM<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B> Re: [SRILM User List] different perplexity<BR></FONT><BR>Md. Akmal Haidar wrote:<BR>> Hi,<BR>> i faced a problem in perplexity calculation..<BR>> when i used the commands: 1) ngram -lm l1.lm -ppl t.txt 2) ngram -lm l2.lm -lambda 0 -mix-lm l1.lm -ppl t.txt<BR>> the first
gives lowest perplexity that the second one.<BR>> Should the above commands give the different perplexity?<BR>They may, though not by much.<BR><BR>Realize that ngram -mix-lm WITHOUT the -bayes option performs an "ngram merging" that APPROXIMATES the result of interpolating the two LMs according to the classical formula. This is describe in the the SRILM paper:<BR>> The ability to approximate class-based and interpolated Ngram<BR>> LMs by a single word N-gram model deserves some discussion.<BR>> Both of these operations are useful in situations where<BR>> other software (e.g., a speech recognizer) supports only standard<BR>> N-grams. Class N-grams are approximated by expanding class labels<BR>> into their members (which can contain multiword strings) and<BR>> then computing the marginal probabilities of word N-gram strings.<BR>> This operation increases the number of N-grams combinatorially,<BR>> and is therefore
feasible only for relatively small models.<BR>> An interpolated backoff model is obtained by taking the union<BR>> of N-grams of the input models, assigning each N-gram the<BR>> weighted average of the probabilities from those models (in some<BR>> of the models this probability might be computed by backoff), and<BR>> then renormalizing the new model. We found that such interpolated<BR>> backoff models consistently give slightly lower perplexities<BR>> than the corresponding standard word-level interpolated models.<BR>> The reason could be that the backoff distributions are themselves<BR>> obtained by interpolation, unlike in standard interpolation, where<BR>> each component model backs off individually.<BR>So the result may differ because because the merging process introduces new backoff nodes into the LM and that may change some probabilities arrived at through backing off. However, if you use<BR><BR> ngram -lm
l2.lm -lambda 0 -mix-lm l1.lm -ppl t.txt -bayes 0<BR><BR>you get exact interpolation and then the perplexities should be identical.<BR>But you cannot save such an interpolated model back into a single ngram LM.<BR><BR>In practice the difference should not matter (at least in my experience).<BR><BR>Andreas<BR><BR><BR>> thanks<BR>> Akmal<BR>> <BR>> <BR>> ------------------------------------------------------------------------<BR>> <BR>> _______________________________________________<BR>> SRILM-User site list<BR>> <A href="mailto:SRILM-User@speech.sri.com" ymailto="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</A><BR>> http://www.speech.sri.com/mailman/listinfo/srilm-user<BR><BR></DIV></DIV></div><br>
</body></html>