<div dir="ltr">Thanks for your reply. Very helpful. But don't we need to use ssIndex and seIndex to build lexicon language model and then use it for next step? <div><br></div><div><br></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Fri, Mar 8, 2013 at 9:49 PM, yasser hifny <span dir="ltr"><<a href="mailto:yhifny@yahoo.com" target="_blank">yhifny@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div style="font-size:10pt;font-family:times new roman,new york,times,serif"><div><span><font><div style="font-size:small">Hi Mohsen,</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">code sample that I use in my work is</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">//
gloable variables</div><div style="background-color:transparent"><div style="background-color:transparent"><span style="font-size:12.727272033691406px;background-color:transparent">Vocab*</span><span style="font-size:12.727272033691406px;background-color:transparent;white-space:pre-wrap"> </span><span style="font-size:12.727272033691406px;background-color:transparent">g_srilm_vocab;</span><br>
</div><div style="background-color:transparent"><span style="font-size:12.727272033691406px">Ngram*<span style="white-space:pre-wrap"> </span>g_model;</span></div></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">double GetLogWordProb(const std::string& strCurrWord,const vector<std::string>& vstrHistory)</div>
<div style="font-size:small">{</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>float<span style="white-space:pre-wrap"> </span>fResult;</div><div style="font-size:small"><br></div><div style="font-size:small">
<span style="white-space:pre-wrap"> </span>size_t len = 1 + vstrHistory.size()+1;</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>VocabIndex* WordIDs=new VocabIndex[len];</div><div style="font-size:small">
<span style="white-space:pre-wrap"> </span>WordIDs[0] = g_srilm_vocab->getIndex((char*)(strCurrWord.c_str()), g_srilm_vocab->unkIndex());</div><div style="font-size:small"><span style="white-space:pre-wrap">
</span>for(size_t i=0;i<vstrHistory.size();i++)</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>WordIDs[i+1] = vstrHistory[i]!="<s>"?</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>g_srilm_vocab->getIndex((char*)(vstrHistory[i].c_str()), g_srilm_vocab->unkIndex()):Vocab_None;</div>
<div style="font-size:small"><span style="white-space:pre-wrap"> </span>WordIDs[vstrHistory.size()+1] = Vocab_None;</div><div style="font-size:small"><br></div><div style="font-size:small"><br></div><div style="font-size:small">
<span style="white-space:pre-wrap"> </span>for(size_t k=0; k<len;k++)</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>{</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>DEBUG("k=%d
wordindex:%d wordstring:%s",k,WordIDs[k],g_srilm_vocab->getWord(WordIDs[k]));</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>}</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>fResult=g_model->wordProb(WordIDs[0], &WordIDs[1]);</div>
<div style="font-size:small"><span style="white-space:pre-wrap"> </span>if(fResult==LogP_Zero)fResult=0.0;</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>DEBUG("prob=%f",fResult);<span style="white-space:pre-wrap"> </span></div>
<div style="font-size:small"><br></div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>//g_model->sentenceProb(words, stats);</div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>delete[] WordIDs;</div>
<div style="font-size:small"><br></div><div style="font-size:small"><span style="white-space:pre-wrap"> </span>return fResult;</div><div style="font-size:small"><br></div><div style="font-size:small">}</div><div style="font-size:13px">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif"><br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
//in the main function</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif"><br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
<br></div><div style="background-color:transparent"><div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>//--------------------------------</span></div>
<div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>// Load LM</span></div><div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>//--------------------------------</span></div>
<div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>g_srilm_vocab= new Vocab;</span></div><div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>g_model=new Ngram(*g_srilm_vocab,nOrder);</span></div>
<div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>File file(strLangModelFile.c_str(), "r");</span></div><div style="background-color:transparent">
<span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>if(!file)</span></div><div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>{</span></div>
<div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>ERROR("Could not open file %s",strLangModelFile.c_str());</span></div><div style="background-color:transparent">
<span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>}</span></div><div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>g_model->read(file, 0);<span style="white-space:pre-wrap"> </span></span></div>
<div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>for(size_t i=0;i<nOrder;i++)</span></div><div style="background-color:transparent"><span style="font-size:12.727272033691406px"><span style="white-space:pre-wrap"> </span>TRACE("Num of ngram in model order %d:%d",i+1,g_model->numNgrams(i+1));</span></div>
<div style="font-style:normal;font-size:12.727272033691406px;font-family:'times new roman','new york',times,serif"><br></div></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
<br></div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">Best regards,</div><div style="font-style:normal;font-size:12.727272033691406px;background-color:transparent;font-family:'times new roman','new york',times,serif">
Yasser</div></font></span></div><div style="font-family:'times new roman','new york',times,serif;font-size:10pt"><br></div> <div style="font-family:'times new roman','new york',times,serif;font-size:10pt">
<div style="font-family:'times new roman','new york',times,serif;font-size:12pt"> <div dir="ltr"> <font face="Arial"> <hr size="1"> <b><span style="font-weight:bold">From:</span></b> Andreas Stolcke <<a href="mailto:stolcke@icsi.berkeley.edu" target="_blank">stolcke@icsi.berkeley.edu</a>><br>
<b><span style="font-weight:bold">To:</span></b> Yi Yang <<a href="mailto:yangyiycc@gmail.com" target="_blank">yangyiycc@gmail.com</a>> <br><b><span style="font-weight:bold">Cc:</span></b> <a href="mailto:SRILM-User@speech.sri.com" target="_blank">SRILM-User@speech.sri.com</a> <br>
<b><span style="font-weight:bold">Sent:</span></b> Friday, March 8, 2013 10:37 PM<br> <b><span style="font-weight:bold">Subject:</span></b> Re: [SRILM User List] c++ sample for building language model<br> </font> </div>
<br>
<div>
<div><div><div class="h5">
<div>On 3/8/2013 8:06 AM, Yi Yang wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Mohsen,
<div><br>
</div>
<div>Hope the following codes can be helpful:</div>
</div>
</blockquote>
you forgot to create the Vocab object. In your case you could
create it globally to your code works otherwise:<br>
<br>
Vocab vocab;<br>
<br>
Andreas<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>
<div>void SrilmTest::srilm_init(const char* fname, int order)
{</div>
<div> File file(fname, "r", 0);</div>
<div> assert(file);</div>
</div>
</div>
</blockquote>
<br>
</div></div><blockquote type="cite"><div><div class="h5">
<div dir="ltr">
<div>
<div> ngram = new Ngram(vocab, order);</div>
<div> ngram->read(file, false);</div>
<div> cerr << "Done\n";</div>
<div>}</div>
<div><br>
</div>
<div>int SrilmTest::srilm_getvoc(const char* word) {</div>
<div> return vocab.getIndex((VocabString)word);</div>
<div>}</div>
<div><br>
</div>
<div>float SrilmTest::srilm_wordprob(int w, int* context) {</div>
<div> return (float)ngram->wordProb(w,
(VocabIndex*)context);</div>
<div>
}</div>
</div>
</div>
</div></div><div><div><div class="h5"><br>
<br>
<div>On Thu, Mar 7, 2013 at 3:23 PM, mohsen
jadidi <span dir="ltr"><<a rel="nofollow" href="mailto:mohsen.jadidi@gmail.com" target="_blank">mohsen.jadidi@gmail.com</a>></span>
wrote:<br>
<blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hey,
<div><br>
</div>
<div>I need to use srilm in my c++ code to build ML. all
the examples and slides in the internet explained it
using ngram-count command not code. I know should use
<Ngram.h> <Vocab.h>. can you point me to a
starting point?</div>
<div><br>
</div>
<div>cheers,</div>
<span><font color="#888888">
<div>
<div><br>
</div>
<div><br>
</div>
-- <br>
Mohsen<br>
</div>
</font></span></div>
<br>
_______________________________________________<br>
SRILM-User site list<br>
<a rel="nofollow" href="mailto:SRILM-User@speech.sri.com" target="_blank">SRILM-User@speech.sri.com</a><br>
<a href="http://www.speech.sri.com/mailman/listinfo/srilm-user" target="_blank">http://www.speech.sri.com/mailman/listinfo/srilm-user</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
Sincerely,<br>
Yi Yang<br>
</div></div><a href="http://www.cc.gatech.edu/%7Eyyang319/" target="_blank">http://www.cc.gatech.edu/%7Eyyang319/</a><br>
</div><div class="im">
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
SRILM-User site list
<a rel="nofollow" href="mailto:SRILM-User@speech.sri.com" target="_blank">SRILM-User@speech.sri.com</a>
<a rel="nofollow" href="http://www.speech.sri.com/mailman/listinfo/srilm-user" target="_blank">http://www.speech.sri.com/mailman/listinfo/srilm-user</a></pre>
</div></blockquote>
<br>
</div>
</div><div class="im"><br>_______________________________________________<br>SRILM-User site list<br><a href="mailto:SRILM-User@speech.sri.com" target="_blank">SRILM-User@speech.sri.com</a><br><a href="http://www.speech.sri.com/mailman/listinfo/srilm-user" target="_blank">http://www.speech.sri.com/mailman/listinfo/srilm-user</a><br>
<br> </div></div> </div> </div></div><br>_______________________________________________<br>
SRILM-User site list<br>
<a href="mailto:SRILM-User@speech.sri.com">SRILM-User@speech.sri.com</a><br>
<a href="http://www.speech.sri.com/mailman/listinfo/srilm-user" target="_blank">http://www.speech.sri.com/mailman/listinfo/srilm-user</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Mohsen Jadidi<br>
</div>