<div dir="ltr"><font color="#333399"><font size="4">Thanks a lot for your reply ,, <br><br><br>I'm trying to build FLM with the following FLM specifications file: <br><br><span style="color:rgb(0,102,0)">## normal trigram LM</span><br style="color:rgb(0,102,0)">
<span style="color:rgb(0,102,0)">1</span><br style="color:rgb(0,102,0)"><span style="color:rgb(0,102,0)">W : 2 W(-1) W(-2) FLMCount.count FLMLM.lm 3</span><br style="color:rgb(0,102,0)"><span style="color:rgb(0,102,0)">W1,W2 W2 wbdiscount interpolate</span><br style="color:rgb(0,102,0)">
<span style="color:rgb(0,102,0)">W1 W1 wbdiscount interpolate</span><br style="color:rgb(0,102,0)"><span style="color:rgb(0,102,0)">0 0 wbdiscount</span><br><br><br>I generate my FLM model using the following command : <br>
<br style="background-color:rgb(255,255,255);color:rgb(0,102,0)"><span style="background-color:rgb(255,255,255);color:rgb(0,102,0)">fngram-count -factor-file FLMDes -debug 2 -text TrainFLM -lm FLMLM .lm -write-counts FLMcount.count -no-virtual-begin-sentence -nonull</span><br>
<br>It runs without errors .. I then measure the ppl of the generated FLM with the following command: <br><br style="color:rgb(0,102,0)"><span style="color:rgb(0,102,0)">fngram -factor-file FLMDes -debug 2 -ppl FLMTest -nonull</span><br>
<br><br>Unfortunately, when I tried to test the main step I got an error :( ... I search the mailing list archive but I didn't find similar problem <br><br>The command I used to test disambig with my FLM model was : <br>
<br style="color:rgb(0,102,0)"><span style="color:rgb(0,102,0)"> disambig -text FLMTest -map 3.map -factored -lm FLMLM.lm</span><br><br>The output of this command was: <br><br><span style="color:rgb(255,0,0)">No known factors found in Aa</span><br style="color:rgb(255,0,0)">
<span style="color:rgb(255,0,0)">No known factors found in AA</span><br style="color:rgb(255,0,0)"><span style="color:rgb(255,0,0)">No known factors found in aa</span><br style="color:rgb(255,0,0)"><span style="color:rgb(255,0,0)">No known factors found in Bb</span><br style="color:rgb(255,0,0)">
<span style="color:rgb(255,0,0)">No known factors found in bb</span><br style="color:rgb(255,0,0)"><span style="color:rgb(255,0,0)">No known factors found in BB</span><br style="color:rgb(255,0,0)"><span style="color:rgb(255,0,0)">No known factors found in CC</span><br style="color:rgb(255,0,0)">
<span style="color:rgb(255,0,0)">No known factors found in cc</span><br style="color:rgb(255,0,0)"><span style="color:rgb(255,0,0)">No known factors found in Cc</span><br style="color:rgb(255,0,0)"><span style="color:rgb(255,0,0)">FLMLM.lm: line 2: Error: couldn't form int for number of factored LMs in when reading FLM spec file</span><br>
<br><br>I don't know what dose it mean by "No known factors found in ......"<br><br>And I wonder about the error message "</font></font><font color="#333399"><font size="4">couldn't form int for number of factored LMs in when reading FLM spec file</font></font><font color="#333399"><font size="4">" .... As you can see above in my FLM specifications file, I determined the number of FLM specifications ! <br>
<br><br><br>Some notes may help you to solve my problem : <br><br>-- I've built my model to test disambig with FLM before using it in my project so it was build with training data of only 28 sentences, 138 words<br>
<br>-- The mapping file (named 3.map) used to test disambig was : <br><span style="color:rgb(0,102,0)">W-aa Aa 0.5 AA 0.4 aa 0.1</span><br style="color:rgb(0,102,0)"><span style="color:rgb(0,102,0)">W-bb Bb 0.6 bb 0.1 BB 0.3</span><br style="color:rgb(0,102,0)">
<span style="color:rgb(0,102,0)">W-cc CC 0.7 cc 0.1 Cc 0.2</span><br><br>-- The FLMTest contains only one sentence: <br><span style="color:rgb(0,102,0)"><s> W-aa W-bb W-cc </s></span><br><br><br><br>Am I doing something wrong ? <br>
<br><br>Your help and support is really greatly appreciated .. I've a graduation project that needs a disambiguator for highly inflected language I'm worried that I could not use your disambig program with FLM model :( <br>
<br><br><br><br><br><br clear="all"></font></font><div dir="ltr">Best Regards,,<br>Reham</div><br>
<br><br><div class="gmail_quote">On 5 March 2012 21:09, Andreas Stolcke <span dir="ltr"><<a href="mailto:stolcke@icsi.berkeley.edu">stolcke@icsi.berkeley.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div class="im">
On 3/5/2012 7:17 AM, Reham Al-Majed wrote:
<blockquote type="cite">
<div dir="ltr"><font color="#333399"><font size="4"><br clear="all">
</font></font>
<div class="gmail_quote"><br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<font color="#333399"><font size="4">Hello ,, <br>
<br>
I've built class based n-gram by : <br>
<br>
1- define my classes<br>
2- use replace-words-with-classes<br>
3- use ngram-count to estimate the LM<br>
<br>
I want to use this class based n gram model with
disambig tool ,, The options (-factored and -count-lm)
interpret the LMs as factored and count based LMs ...
What about class-based ? How to tell disambig to
interpret the LM as a class-based ? <br>
<br>
I'm trying to use my class-based as an original n-gram
model, however the output for sample test seems
strange ... words in the test sample are always
disambiguated using the last word in the mapping file
!<br>
<br>
Actually I want the words be disambiguated using the
LM probabilities only without considering the
probabilities in the mapping file.. I use the options
-lmw 1 and -mapw 0 but the output still the same ... <br>
<br>
<br>
In short my questions are : <br>
<br>
1- Is it possible to use class-based n gram with
disabmig tool ? Or should I build my own
disambiguator using the output of ngram tool ?<br>
</font></font></div>
</blockquote>
</div>
</div>
</blockquote>
<br></div>
Unfortunately disambig currently does not support the use of
class-based ngram LMs (what is implemented by ngram -classes).<br>
Two workarounds are <br>
1) if feasible, expand the class-ngram LM into a word-ngram LM
(using ngram -expand-classes).<br>
2) rewrite the class-ngram as a factored LM. This will require some
investment into understanding the much more general FLM mechanism.<div class="im"><br>
<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr"><font color="#333399"><font size="4"><br>
2- How to make disambig tool use the probabilities of
LM ONLY ? <br>
</font></font></div>
</blockquote>
</div>
</div>
</blockquote>
<br></div>
disambig -mapw 0 will do that.<span class="HOEnZb"><font color="#888888"><br>
<br>
Andreas <br>
<br>
<br>
</font></span></div>
</blockquote></div><br></div>