<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7036.0">
<TITLE>Optimizing Weights in Log-Linear Interpolation </TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P><FONT FACE="Times New Roman">Hello,</FONT>
<BR><FONT FACE="Times New Roman">I was wondering how I can train the weights in log-linear interpolation of several language models (as in Klakow's paper). </FONT></P>
<P><FONT FACE="Times New Roman">I have successfully used "compute-best-mix" script to use linear interpolation weights but do not see how to modify the process to optimize log-linear interpolation weights so that the perplexity is minimized on a cross-validation set.</FONT></P>
<P><FONT FACE="Times New Roman">Thank you,</FONT>
<BR><FONT FACE="Times New Roman">Sibel Yaman</FONT>
<BR><B><FONT FACE="Times New Roman">From:</FONT></B><FONT FACE="Times New Roman"> Andreas Stolcke <stolcke at ADDRESS HIDDEN><BR>
</FONT><B><FONT FACE="Times New Roman">Date:</FONT></B><FONT FACE="Times New Roman"> Mon, 18 Jul 2005 07:28:40 PDT</FONT>
<BR><FONT FACE="Times New Roman">In message <32809.213.58.88.69.1081673875.squirrel at ADDRESS HIDDEN>you wrote:<BR>
><BR>
> Hi!<BR>
><BR>
> Does anyone know a program or toolkit allowing to do log-linear<BR>
> interpolation of different language models? since SRILM only permit to do<BR>
> linear interpolation.<BR>
> Thanks for your help,<BR>
><BR>
> Ciro Martins<BR>
<BR>
Ciro,<BR>
<BR>
sorry for the late response ;-)<BR>
<BR>
There is now, in the current development version of SRILM, an<BR>
implementation of log-linear interpolation. The class name is<BR>
LoglinearMix, and the ngram -loglinear-mix option triggers its use.<BR>
Note that log-linear interpolation is much slower to evaluate than<BR>
linear interpolation, due to the need to normalize the combined LM.<BR>
This is done somewhat efficiently in SRILM by caching the normalizers<BR>
for previously seen contexts.<BR>
<BR>
You might also want to try using log-linear combination of LM scores<BR>
without normalization. This can be done in the nbest or lattice<BR>
rescoring framework implemented by the toolkit, simply by computing<BR>
scores from multiple LMs.<BR>
<BR>
The latest version of the toolkit can by downloaded in the usual way<BR>
by choosing the "1.4.5 (Beta)" version in the web form.<BR>
<BR>
--Andreas<BR>
</FONT>
<BR><FONT FACE="Times New Roman">Click </FONT><A HREF="file:///projects/srilm/"><U><FONT COLOR="#0000FF" FACE="Times New Roman">here</FONT></U></A><FONT FACE="Times New Roman"> to go to the SRILM home page.</FONT>
</P>
</BODY>
</HTML>