<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:10pt"><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 10pt;"><span>Dear </span><span style="font-size: 12.727272033691406px;">Mohsen,</span></div><div style="background-color: transparent;"><span><span style="font-size: 12.727272033691406px;">I do not understand your question. perhaps, </span></span><span>the group members can help.</span></div><div style="background-color: transparent; color: rgb(0, 0, 0); font-size: 12.727272033691406px; font-family: 'times new roman', 'new york', times, serif; font-style: normal;"><span><br></span></div><div style="background-color: transparent; color: rgb(0, 0, 0); font-size: 12.727272033691406px; font-family: 'times new roman', 'new york', times, serif; font-style: normal;"><span>best regards,</span></div><div style="background-color:
transparent; color: rgb(0, 0, 0); font-size: 12.727272033691406px; font-family: 'times new roman', 'new york', times, serif; font-style: normal;"><span>Yasser</span></div><div style="font-family: 'times new roman', 'new york', times, serif; font-size: 10pt;"><br></div> <div style="font-family: 'times new roman', 'new york', times, serif; font-size: 10pt;"> <div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> mohsen jadidi <mohsen.jadidi@gmail.com><br> <b><span style="font-weight: bold;">To:</span></b> yasser hifny <yhifny@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> "SRILM-User@speech.sri.com" <SRILM-User@speech.sri.com> <br> <b><span style="font-weight: bold;">Sent:</span></b> Friday, March 8, 2013 11:10 PM<br> <b><span style="font-weight: bold;">Subject:</span></b>
Re: [SRILM User List] c++ sample for building language model<br> </font> </div> <br>
<div id="yiv1699939155"><div dir="ltr">Thanks for your reply. Very helpful. But don't we need to use ssIndex and seIndex to build lexicon language model and then use it for next step? <div><br></div><div><br></div></div><div class="yiv1699939155gmail_extra">
<br><br><div class="yiv1699939155gmail_quote">On Fri, Mar 8, 2013 at 9:49 PM, yasser hifny <span dir="ltr"><<a rel="nofollow" ymailto="mailto:yhifny@yahoo.com" target="_blank" href="mailto:yhifny@yahoo.com">yhifny@yahoo.com</a>></span> wrote:<br><blockquote class="yiv1699939155gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><div style="font-size: 10pt; font-family: 'times new roman', 'new york', times, serif;"><div><span><font><div style="font-size: 13px;">Hi Mohsen,</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">code sample that I use in my work is</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">//
gloable variables</div><div style="background-color:transparent;"><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;background-color:transparent;">Vocab*</span><span style="font-size:12.727272033691406px;background-color:transparent;white-space:pre-wrap;"> </span><span style="font-size:12.727272033691406px;background-color:transparent;">g_srilm_vocab;</span><br>
</div><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;">Ngram*<span style="white-space:pre-wrap;"> </span>g_model;</span></div></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">double GetLogWordProb(const std::string& strCurrWord,const vector<std::string>& vstrHistory)</div>
<div style="font-size:small;">{</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>float<span style="white-space:pre-wrap;"> </span>fResult;</div><div style="font-size:small;"><br></div><div style="font-size:small;">
<span style="white-space:pre-wrap;"> </span>size_t len = 1 + vstrHistory.size()+1;</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>VocabIndex* WordIDs=new VocabIndex[len];</div><div style="font-size:small;">
<span style="white-space:pre-wrap;"> </span>WordIDs[0] = g_srilm_vocab->getIndex((char*)(strCurrWord.c_str()), g_srilm_vocab->unkIndex());</div><div style="font-size:small;"><span style="white-space:pre-wrap;">
</span>for(size_t i=0;i<vstrHistory.size();i++)</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>WordIDs[i+1] = vstrHistory[i]!="<s>"?</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>g_srilm_vocab->getIndex((char*)(vstrHistory[i].c_str()), g_srilm_vocab->unkIndex()):Vocab_None;</div>
<div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>WordIDs[vstrHistory.size()+1] = Vocab_None;</div><div style="font-size:small;"><br></div><div style="font-size:small;"><br></div><div style="font-size:small;">
<span style="white-space:pre-wrap;"> </span>for(size_t k=0; k<len;k++)</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>{</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>DEBUG("k=%d
wordindex:%d wordstring:%s",k,WordIDs[k],g_srilm_vocab->getWord(WordIDs[k]));</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>}</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>fResult=g_model->wordProb(WordIDs[0], &WordIDs[1]);</div>
<div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>if(fResult==LogP_Zero)fResult=0.0;</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>DEBUG("prob=%f",fResult);<span style="white-space:pre-wrap;"> </span></div>
<div style="font-size:small;"><br></div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>//g_model->sentenceProb(words, stats);</div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>delete[] WordIDs;</div>
<div style="font-size:small;"><br></div><div style="font-size:small;"><span style="white-space:pre-wrap;"> </span>return fResult;</div><div style="font-size:small;"><br></div><div style="font-size:small;">}</div><div style="font-size:13px;">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;"><br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
//in the main function</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;"><br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
<br></div><div style="background-color:transparent;"><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>//--------------------------------</span></div>
<div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>// Load LM</span></div><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>//--------------------------------</span></div>
<div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>g_srilm_vocab= new Vocab;</span></div><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>g_model=new Ngram(*g_srilm_vocab,nOrder);</span></div>
<div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>File file(strLangModelFile.c_str(), "r");</span></div><div style="background-color:transparent;">
<span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>if(!file)</span></div><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>{</span></div>
<div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>ERROR("Could not open file %s",strLangModelFile.c_str());</span></div><div style="background-color:transparent;">
<span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>}</span></div><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>g_model->read(file, 0);<span style="white-space:pre-wrap;"> </span></span></div>
<div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>for(size_t i=0;i<nOrder;i++)</span></div><div style="background-color:transparent;"><span style="font-size:12.727272033691406px;"><span style="white-space:pre-wrap;"> </span>TRACE("Num of ngram in model order %d:%d",i+1,g_model->numNgrams(i+1));</span></div>
<div style="font-style:normal;font-size:12.727272033691406px;"><br></div></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">Best regards,</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;">
Yasser</div></font></span></div><div style="font-size:10pt;"><br></div> <div style="font-size:10pt;">
<div style="font-size:12pt;"> <div dir="ltr"> <font face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Andreas Stolcke <<a rel="nofollow" ymailto="mailto:stolcke@icsi.berkeley.edu" target="_blank" href="mailto:stolcke@icsi.berkeley.edu">stolcke@icsi.berkeley.edu</a>><br>
<b><span style="font-weight:bold;">To:</span></b> Yi Yang <<a rel="nofollow" ymailto="mailto:yangyiycc@gmail.com" target="_blank" href="mailto:yangyiycc@gmail.com">yangyiycc@gmail.com</a>> <br><b><span style="font-weight:bold;">Cc:</span></b> <a rel="nofollow" ymailto="mailto:SRILM-User@speech.sri.com" target="_blank" href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a> <br>
<b><span style="font-weight:bold;">Sent:</span></b> Friday, March 8, 2013 10:37 PM<br> <b><span style="font-weight:bold;">Subject:</span></b> Re: [SRILM User List] c++ sample for building language model<br> </font> </div>
<br>
<div>
<div><div><div class="yiv1699939155h5">
<div>On 3/8/2013 8:06 AM, Yi Yang wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Mohsen,
<div><br>
</div>
<div>Hope the following codes can be helpful:</div>
</div>
</blockquote>
you forgot to create the Vocab object. In your case you could
create it globally to your code works otherwise:<br>
<br>
Vocab vocab;<br>
<br>
Andreas<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>
<div>void SrilmTest::srilm_init(const char* fname, int order)
{</div>
<div> File file(fname, "r", 0);</div>
<div> assert(file);</div>
</div>
</div>
</blockquote>
<br>
</div></div><blockquote type="cite"><div><div class="yiv1699939155h5">
<div dir="ltr">
<div>
<div> ngram = new Ngram(vocab, order);</div>
<div> ngram->read(file, false);</div>
<div> cerr << "Done\n";</div>
<div>}</div>
<div><br>
</div>
<div>int SrilmTest::srilm_getvoc(const char* word) {</div>
<div> return vocab.getIndex((VocabString)word);</div>
<div>}</div>
<div><br>
</div>
<div>float SrilmTest::srilm_wordprob(int w, int* context) {</div>
<div> return (float)ngram->wordProb(w,
(VocabIndex*)context);</div>
<div>
}</div>
</div>
</div>
</div></div><div><div><div class="yiv1699939155h5"><br>
<br>
<div>On Thu, Mar 7, 2013 at 3:23 PM, mohsen
jadidi <span dir="ltr"><<a rel="nofollow" ymailto="mailto:mohsen.jadidi@gmail.com" target="_blank" href="mailto:mohsen.jadidi@gmail.com">mohsen.jadidi@gmail.com</a>></span>
wrote:<br>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div dir="ltr">Hey,
<div><br>
</div>
<div>I need to use srilm in my c++ code to build ML. all
the examples and slides in the internet explained it
using ngram-count command not code. I know should use
<Ngram.h> <Vocab.h>. can you point me to a
starting point?</div>
<div><br>
</div>
<div>cheers,</div>
<span><font color="#888888">
<div>
<div><br>
</div>
<div><br>
</div>
-- <br>
Mohsen<br>
</div>
</font></span></div>
<br>
_______________________________________________<br>
SRILM-User site list<br>
<a rel="nofollow" ymailto="mailto:SRILM-User@speech.sri.com" target="_blank" href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a><br>
http://www.speech.sri.com/mailman/listinfo/srilm-user<br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
Sincerely,<br>
Yi Yang<br>
</div></div>http://www.cc.gatech.edu/%7Eyyang319/<br>
</div><div class="yiv1699939155im">
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
SRILM-User site list
<a rel="nofollow" ymailto="mailto:SRILM-User@speech.sri.com" target="_blank" href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a>
<a rel="nofollow" target="_blank" href="http://www.speech.sri.com/mailman/listinfo/srilm-user">http://www.speech.sri.com/mailman/listinfo/srilm-user</a></pre>
</div></blockquote>
<br>
</div>
</div><div class="yiv1699939155im"><br>_______________________________________________<br>SRILM-User site list<br><a rel="nofollow" ymailto="mailto:SRILM-User@speech.sri.com" target="_blank" href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a><br><a rel="nofollow" target="_blank" href="http://www.speech.sri.com/mailman/listinfo/srilm-user">http://www.speech.sri.com/mailman/listinfo/srilm-user</a><br>
<br> </div></div> </div> </div></div><br>_______________________________________________<br>
SRILM-User site list<br>
<a rel="nofollow" ymailto="mailto:SRILM-User@speech.sri.com" target="_blank" href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a><br>
<a rel="nofollow" target="_blank" href="http://www.speech.sri.com/mailman/listinfo/srilm-user">http://www.speech.sri.com/mailman/listinfo/srilm-user</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Mohsen Jadidi<br>
</div>
</div><br><br> </div> </div> </div></body></html>