<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Andreas,<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Sorry to bother you with this old issue.</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>The two-step lattice-tool process worked perfectly. First the rescoring and second the conversion to CN.</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>But, unfortunately I have seen a few unks while rescoring the lattice (not as many as writing the mesh).</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>The command I use to rescore is:</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> lattice-tool -lm ../../lm/interpolated-lm.en -in-lattice wordlattice0.slf -read-htk -out-lattice out.</span>slf<span class="Apple-tab-span" style="white-space:pre"> -write-htk -keep-unk -print-sent-tags -htk-logbase 2.71828</span></div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>And I find lines like these: (Whithin these lines the <unk> tag should be queit)</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre">J=26 S=19 E=24 W=qu a=0 l=-13.8261
J=27 S=19 E=25 W=que a=0 l=-11.4986
J=28 S=19 E=26 W=<unk> a=0 l=-2.76367
J=29 S=19 E=27 W=quest a=0 l=-10.831
J=30 S=19 E=28 W=quiet a=0 l=-10.57
J=31 S=19 E=29 W=quit a=0 l=-10.4455
J=32 S=20 E=21 W=row a=0 l=-10.1076
J=33 S=21 E=24 W=qu a=0 l=-14.9448
J=34 S=21 E=25 W=que a=0 l=-12.6173
J=35 S=21 E=26 W=<unk> a=0 l=-3.88236
J=36 S=21 E=27 W=quest a=0 l=-11.9497
J=37 S=21 E=28 W=quiet a=0 l=-11.6887
J=38 S=21 E=29 W=quit a=0 l=-11.0153
J=39 S=22 E=19 W=arrow a=0 l=-12.6258</span></div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>I have to say that I use the rescoring to give probabilities to the archs from misspelling corrections. So I do not have any acoustic scores. (I set all them equal).</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Regards,</div><div><br></div><div>Lluís<br><div>
<font class="Apple-style-span" color="#4181c0" face="Arial" size="2"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px; -webkit-border-vertical-spacing: 2px; "><b><span><img height="133" width="589" id="a56df6cb-69c2-4dfb-8591-d8f96cd1c800" apple-width="yes" apple-height="yes" src="cid:387CDDED-8ECE-409A-9D4A-D1C4B69B666C"></span>
</b></span></font></div><font class="Apple-style-span" color="#4181c0" face="Arial" size="2"><b>
</b></font><br><div><div>El 23/05/2012, a les 2:05, Andreas Stolcke va escriure:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
On 5/22/2012 10:56 AM, Lluís Formiga i Fanals wrote:
<blockquote cite="mid:9D363DB8-11DF-4534-AEB5-058E96E3A74C@upc.edu" type="cite">Hi,
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>I
was trying to execute the following command:</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre">
lattice-tool -in-lattice-list lattice_lists.txt -read-htk -lm
/veu4/usuaris24/lluisf/EMS/misspelling2012/lm/interpolated-lm.en
-write-mesh-dir out -keep-unk</span></div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>but
I find that unks ("<unk>") are still on the written CN
(-write-mesh).</div>
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>Does
-keep-unk option work only for lattices output? Am I doing
something wrong?</div>
</blockquote>
No, the code is working as intended.<br>
<br>
The option is described as <br>
-keep-unk<br>
Treat out-of-vocabulary words as <unk> but
preserve their labels in lattice output.<br>
<br>
What you are outputting is confusion networks, not lattices. In the
CN building process, lattice nodes that are mapped to <unk>
are treated as equivalent, and the word information is lost in the
process.<br>
<br>
I would suggest that you simple do your lattice rescoring with
-keep-unk, output the rescored lattices, and then run lattice-tool a
second time without -keep-unk and without the -vocab option, so all
word labels are preserved (all words are implicitly added to the
vocabulary).<br>
<br>
Andreas <br>
<br>
<br>
<blockquote cite="mid:9D363DB8-11DF-4534-AEB5-058E96E3A74C@upc.edu" type="cite">
<div><br>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>Thanks,</div>
<div><br>
</div>
<div>Lluís</div>
<div>
<div>
<font class="Apple-style-span" color="#4181c0" face="Arial" size="2"><span class="Apple-style-span" style="-webkit-border-horizontal-spacing: 2px;
-webkit-border-vertical-spacing: 2px; "><b><span><span><Adjunt de Mail.jpeg></span></span>
</b></span></font></div>
<div><br>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</blockquote></div><br></div></body></html>