------=_Part_99087_4163798.1168990735389
Content-Type: multipart/alternative;
boundary="----=_Part_99088_19975429.1168990735389"
------=_Part_99088_19975429.1168990735389
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi,
I was seeing weird behavior in lattice-tool, mixing in an external LM to a
lattice for nbest decoding.
Tracking things down, I found that if I zeroed out the external LM scores as
they were added into the lattice during expansion, the resulting hyp scores
were always zero, that is the scores from the lattice were discarded. I
observed this for both HTK and PFSG lattices.
Attached is a patch (to version 1.5.1) which I believe fixes the problem.
What I found is that, as old transitions were replaced during expansion
(Lattice::expandAddTransition() in LatticeExpand.cc), the old weights were
discarded. This caused the problem because theinitial transitions loaded
from the lattice files were replaced during expansion.
Cheers,
tm
P.S. I also made some changes to functionality, let me know if anyone is
interested in them: (1) allowing scaling of the external LM as it's used to
reweight the lattice and (2) outputing (weighted) acoustic and LM scores to
the nbest list as they were actually evaluated during decoding; currently
only the original scores from the lattice are output for HTK lattices and
zeros are output for PFSG lattices, because they don't fill the internal HTK
structures used for score output.
------=_Part_99088_19975429.1168990735389
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi,<br><br>I was seeing weird behavior in lattice-tool, mixing in an external LM to a lattice for nbest decoding.<br><br>Tracking things down, I found that if I zeroed out the external LM scores as they were added into the lattice during expansion, the resulting hyp scores were always zero, that is the scores from the lattice were discarded. I observed this for both HTK and PFSG lattices.
<br><br>Attached is a patch (to version 1.5.1) which I believe fixes the problem. What I found is that, as old transitions were replaced during expansion (Lattice::expandAddTransition() in LatticeExpand.cc), the old weights were discarded. This caused the problem because theinitial transitions loaded from the lattice files were replaced during expansion.
<br><br>Cheers,<br><br>tm<br><br><br>P.S. I also made some changes to functionality, let me know if anyone is interested in them: (1) allowing scaling of the external LM as it's used to reweight the lattice and (2) outputing (weighted) acoustic and LM scores to the nbest list as they were actually evaluated during decoding; currently only the original scores from the lattice are output for HTK lattices and zeros are output for PFSG lattices, because they don't fill the internal HTK structures used for score output.
<br><br>
------=_Part_99088_19975429.1168990735389--
------=_Part_99087_4163798.1168990735389
Content-Type: application/octet-stream; name=LatticeExpand.patch
Content-Transfer-Encoding: base64
X-Attachment-Id: f_ex0wbgtf
Content-Disposition: attachment; filename="LatticeExpand.patch"
SW5kZXg6IGxhdHRpY2Uvc3JjL0xhdHRpY2VFeHBhbmQuY2MKPT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gbGF0dGlj
ZS9zcmMvTGF0dGljZUV4cGFuZC5jYwkoLi4uLzEuNS4xL2xhdHRpY2Uvc3JjL0xhdHRpY2VFeHBh
bmQuY2MpCShyZXZpc2lvbiAzNDUpCisrKyBsYXR0aWNlL3NyYy9MYXR0aWNlRXhwYW5kLmNjCSgu
Li4vdHJ1bmsvbGF0dGljZS9zcmMvTGF0dGljZUV4cGFuZC5jYykJKHdvcmtpbmcgY29weSkKQEAg
LTE0NDgsNyArMTQ0OCw4IEBACiAJfQogICAgIH0KIAotICAgIExhdHRpY2VUcmFuc2l0aW9uIG5l
d1RyYW5zKHRyYW5zUHJvYiwgb2xkVHJhbnMtPmZsYWdzKTsKKyAgICAvLyBCVUcgRklYOiBPbGQg
dHJhbnNpdGlvbiB3YXMgYmVpbmcgcmVwbGFjZWQgYnkgbmV3LCBidXQgbmV3IGRpc2NhcmRlZCBv
bGQgd2VpZ2h0IQorICAgIExhdHRpY2VUcmFuc2l0aW9uIG5ld1RyYW5zKHRyYW5zUHJvYiArIG9s
ZFRyYW5zLT53ZWlnaHQsIG9sZFRyYW5zLT5mbGFncyk7CiAgICAgaW5zZXJ0VHJhbnMobmV3SW5k
ZXgsICpuZXdJbmRleDIsIG5ld1RyYW5zLCAwKTsKIAogICAgIC8vIHJlc3RvcmUgZnVsbCBjb250
ZXh0Cg==
------=_Part_99087_4163798.1168990735389--
Click here to go to the SRILM home page.