Search SRILM-USER Archives

Re: SRILM 1.4.5 released

In message <006601c5af7d$e7ff2cc0$7a40ba85@pcn122>you wrote:
> Dear Dr.Stolcke,
>
> Thanks for the reply. Recently I have made an experiment on loglinear
> integration of multiple LMs. I found if the input LMs are in Arpa format,
> the output loglinear LM can be turned into Arpa format. I think you could
> give me some comments on the work because it seems you are thinking in the
> same way as I. I have made a program to implement the idea. Could you take
> some time to have a look at the page?
> http://www.slc.atr.jp/~rzhang/lma.html

Ruiqiang,

Thank you for this pointer to your web page.
I haven't had time to understand your code in detail, but I believe
you can achieve a similar effect using SRILM, maybe not as efficiently.

You can use the following steps to construct a merged N-gram LM
incorporating log-linear mixture probabilities.

Suppose your input LMs are called LM1, LM2, LM3, etc.

1. Use the standard linear mixture LM to create a new LM that has the
   union of all the ngrams:

ngram -lm LM1 -mix-lm LM2 ... -write-lm MIXLM

2. Recompute the probabilities in the merged LM using the log-linear model:

ngram -lm LM1 -mix-lm LM2 ... -rescore-ngram MIXLM -write-lm LOGMIXLM

(I omitted the lambda weight options in the last command.)
Note this will be rather slow, since the probability normalization has
to be carried out for each ngram context in the LM.

--Andreas

> Thanks,
> -Ruiqiang
>
>
> ----- Original Message -----
> From: "Andreas Stolcke" <stolcke at ADDRESS HIDDEN>
> To: "Ruiqiang Zhang" <ruiqiang.zhang at ADDRESS HIDDEN>
> Sent: Tuesday, August 30, 2005 3:05 PM
> Subject: Re: SRILM 1.4.5 released
>
>
> >
> > In message <016301c5ad28$30ce7120$7a40ba85@pcn122>you wrote:
> > > Dear Dr.Stolcke,
> > >
> > > Thanks for informing me the latest version. I tested the new features,
> > > however, i got no output when I was using the "loglinear-mix". The usage
> and
> > > errors are shown below. Please give me a reply in your free time.
> > >
> > > [rzhang@pxn147 test]$ ngram -lm lm1 -mix-lm lm2 -lambda
> > > 0.5 -loglinear-mix -write-lm lm.o
> > > lm1: line 1034: warning: non-zero probability for <unk> in
> closed-vocabulary
> > > LM
> > > lm2: line 1191: warning: non-zero probability for <unk> in
> closed-vocabulary
> > > LM
> > > write() method not implemented
> >
> > -loglinear-mix does not support merging of LMs (static interpolation).
> > you can only evaluate the mixture LM using -ppl and other options that
> compute
> > LM probabilies (dynamic interpolation).
> >
> > the first two messages are just warnings.  Use the -unk option to avoid
> > them.
> >
> > --Andreas
> >
> >
>
>

Click here to go to the SRILM home page.