<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:宋体,simsun,serif;font-size:10pt"><div style="font-family: 宋体,simsun,serif; font-size: 10pt;"><div style="font-family: 宋体,simsun,serif; font-size: 12pt;">Hi,Thanks for your concern!<br>I do know that back-off weight is not a probability,but in the interpolated mod-kn smoothing method,bows are not supposed to be greater than 1.<br>In the man document of srilm ngram-discount.7.html,I've got this:<br>For back-off smoothing,there is<br><i> (1) p</i>(<i>a</i>_<i>z</i>) = (<i>c</i>(<i>a</i>_<i>z</i>) > 0) ? <i>f</i>(<i>a</i>_<i>z</i>) : bow(<i>a</i>_) <i>p</i>(_<i>z</i>)
<br>where <i>f</i>(<i>a</i>_<i>z</i>) depends on the smoothing method and the bow(<i>a</i>_) is calculated below:<br> Sum_<i>Z</i> <i>p</i>(<i>a</i>_<i>z</i>) = 1
Sum_<i>Z1</i> <i>f</i>(<i>a</i>_<i>z</i>) + Sum_<i>Z0</i> bow(<i>a</i>_) <i>p</i>(_<i>z</i>) = 1
<br>(2) bow(<i>a</i>_) = (1- Sum_<i>Z1</i> <i>f</i>(<i>a</i>_<i>z</i>)) / Sum_<i>Z0</i> <i>p</i>(_<i>z</i>)
<br> = (1 - Sum_<i>Z1</i> <i>f</i>(<i>a</i>_<i>z</i>)) / (1 - Sum_<i>Z1</i> <i>p</i>(_<i>z</i>))
<br> = (1 - Sum_<i>Z1</i> <i>f</i>(<i>a</i>_<i>z</i>)) / (1 - Sum_<i>Z1</i> <i>f</i>(_<i>z</i>))
<br>but for interpolated smoothing, there is<br><i>(3) f</i>(<i>a</i>_<i>z</i>) = <i>g</i>(<i>a</i>_<i>z</i>) + bow(<i>a</i>_) <i>p</i>(_<i>z</i>)
<br> <i>(4) p</i>(<i>a</i>_<i>z</i>) = (<i>c</i>(<i>a</i>_<i>z</i>) > 0) ? <i>f</i>(<i>a</i>_<i>z</i>) : bow(<i>a</i>_) <i>p</i>(_<i>z</i>)
<br>and<br> Sum_<i>Z</i> <i>p(</i><i>a_</i><i>z)</i> = 1
<br> Sum_<i>Z1</i> <i>g(</i><i>a_</i><i>z</i>) + Sum_<i>Z</i> bow(<i>a</i>_) <i>p</i>(_<i>z</i>) = 1
<br>(5) bow(<i>a</i>_) = 1 - Sum_<i>Z1</i> <i>g</i>(<i>a</i>_<i>z</i>)
<br><div> (Where <i>Z </i>be the set of all words in the vocabulary, <i>Z0 </i>be the set of
all words with <i>c</i>(<i>a</i>_<i>z</i>) = 0, and <i>Z1 </i>be the set of all
words with <i>c</i>(<i>a</i>_<i>z</i>) > 0)<br><br>However in the srilm sourse codes ,it seems that the interpolated bows is calculated using (5) and then the probs and bows is trasfered into back-off model using (3) ,then the back-off version of the bows are recomputed using (2).I just don't understand why srilm do not use the bow calculated using (5)directedly.<br>Besides,I used to use the entropy-prune method to construct a language model:<br><font size="2">~ngram-count -read merge_counts_1994-2003.gz -gt1min 0 -gt2min 0 -gt3min 0 -kndiscount -interpolate -prune 0.000000001 -order 3 -vocab ChWord.lexno -lm 1994-2003_lm_pruned1e-9.lm</font><br>and there is definitely no bow greater than 1.<br>So this problem is wired and I wonder if anyone of you knows that.And was the command I used to build the mod-kn discount language model(where I want to exclude the 3-grams with the count of 1) correct?<br><font><font style="font-family: 宋体,simsun,serif;
font-size: 13px; color: rgb(0, 0, 0);" color="#000000" face="宋体, simsun, serif" size="2">~
ngram-count -read merge_counts_1994-2003.gz -gt1min 0 -gt2min 0 -gt3min
2 -kndiscount -interpolate -order 3 -vocab ChWord.lexno -lm
1994-2003_lm_all_pruned.lm<br></font></font><br>Thank you very much!<br><br><br><br></div>史海龙<br>Hailoon Shi<br>w63,EE Dpt.Tinghua.Unv.Beijing.China<br>分享快乐，加倍快乐<a rel="nofollow" target="_blank" href="http://cn.webmessenger.yahoo.com/index.php?t=1&to=eWlkPXNobC50aGNu&sig=d7706c860a0ed09b9dc617e5476ae7892d50baaa"><img alt="op3 5" src="http://opi.yahoo.com/online?u=shl.thcn&t=5&l=cn" name="pimg" border="0"></a><br><div><br></div><div style="font-family: 宋体,simsun,serif; font-size: 12pt;"><br><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"><font face="Tahoma" size="2"><hr size="1"><b><span style="font-weight: bold;">发件人：</span></b> Yannick Estève <yannick.esteve@lium.univ-lemans.fr><br><b><span style="font-weight: bold;">收件人：</span></b> 海龙 史 <shl.thcn@yahoo.com.cn><br><b><span style="font-weight: bold;">抄送：</span></b>
srilm-user@speech.sri.com<br><b><span style="font-weight: bold;">已发送：</span></b> 2009/8/27(周四), 下午4:19:44<br><b><span style="font-weight: bold;">主题：</span></b> Re: [SRILM User List] A confusion of the interpolated language model<br></font><br>Hi,<div><br></div><div>Back-off weights are not probabilities: they can be greater than 1.</div><div>So, your values are normal. You can have some explanations about back-off weight computation here, particularly for the use of the modified Kneser-Ney discounting method:</div><div><a rel="nofollow" target="_blank" href="http://www.speech.sri.com/projects/srilm/manpages/pdfs/chen-goodman-tr-10-98.pdf">http://www.speech.sri.com/projects/srilm/manpages/pdfs/chen-goodman-tr-10-98.pdf</a></div><div><br></div><div>Regards,</div><div>Yannick Estève</div><div>LIUM - University of Le Mans</div><div>France</div><div><br></div><div><div><div>Le 27 août 09 à 09:21, 海龙 史 a écrit :</div><br
class="Apple-interchange-newline"><blockquote type="cite"><div style="margin: 0px; font-family: 宋体,simsun,serif; font-size: 12pt;"><div style="margin: 0px; font-family: 宋体,simsun,serif; font-size: 12pt;"><div style="margin: 0px; font-family: 'times new roman','new york',times,serif; font-size: 12pt;"><br class="Apple-interchange-newline"><br><div style="margin: 0px; font-family: 宋体,simsun,serif; font-size: 12pt; color: rgb(0, 0, 0);"><table width="100%" align="center" cellpadding="0" cellspacing="0"><tbody><tr><td valign="top" width="100%" bgcolor="#ffffff" height="100%"><table style="table-layout: fixed;" width="100%" align="center" cellpadding="0" cellspacing="0" height="100%"><tbody><tr><td valign="bottom" width="170" align="left"><br></td><td style="white-space: nowrap;" valign="top"><table style="white-space: normal;" width="100%" cellpadding="0" cellspacing="0"><tbody><tr><td height="10"><br></td></tr><tr><td><font style="font-family:
宋体,simsun,serif; font-size: 13px; color: rgb(0, 0, 0);" color="#000000" face="宋体, simsun, serif" size="2"><div style="margin: 0px;">I am a new student user of srilm from Asia.Here I used the command below to construct a interpolated mod-kn discount language model:<br>~ ngram-count -read merge_counts_1994-2003.gz -gt1min 0 -gt2min 0 -gt3min 2 -kndiscount -interpolate -order 3 -vocab ChWord.lexno -lm 1994-2003_lm_all_pruned.lm<br><br><div style="margin: 0px;"> However in my model several N-grams' back-off werght(bow) appears to be greater than 1.That is ,in the text LM file,I've got a line:<br>-6.457229 <s> 1635 0.1270406<br>(Here we just use a kind of index to represent a chinese word)<br>in whitch the 1og10(bow) is greater than 0.We don't think a normal interplotate discount method can produce an N-gram bow greater than 1,besides this circumstance only occured to several(less than 5) different
N-grams.So I am confused and would like to ask if there is someyone who encounterd this circumstance or happens to know what is wrong.<br>Thank you very much!<br><br></div>史海龙<br>Hailoon Shi<br>w63,EE Dpt.Thu Univ.PRC<br><br><div style="margin: 0px;"><br></div></div></font></td></tr><tr><td height="100"><br></td></tr></tbody></table></td><td width="10"><br></td></tr></tbody></table></td></tr></tbody></table></div><a rel="nofollow" target="_blank" href="http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/"></a></div></div></div><br>__________________________________________________<br>赶快注册雅虎超大容量免费邮箱?<br><a rel="nofollow" target="_blank" href="http://cn.mail.yahoo.com">http://cn.mail.yahoo.com</a>_______________________________________________<br>SRILM-User site list<br><a rel="nofollow" ymailto="mailto:SRILM-User@speech.sri.com" target="_blank"
href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a><br><a rel="nofollow" target="_blank" href="http://www.speech.sri.com/mailman/listinfo/srilm-user">http://www.speech.sri.com/mailman/listinfo/srilm-user</a></blockquote></div><br></div></div></div></div></div></div><br>
<hr size=1><a href="http://cn.rd.yahoo.com/mail_cn/tagline/card/*http://card.mail.cn.yahoo.com/"> 好玩贺卡等你发，邮箱贺卡全新上线！</a></body></html>