In message <20060919160016.52311.qmail at ADDRESS HIDDEN>you wrote:
> --0-789738089-1158681616=:50607
> Content-Type: text/plain; charset=iso-8859-1
> Content-Transfer-Encoding: 8bit
>
> I am developing language models of different order (2 to 5) with Good-Turing
> discounting and Katz backoff for Smoothing. I all cases, I have got the foll
> owing warning message:
> discount coeff 1 is out of range : 6.2135e-17
>
> I could not get the reason for the warning message. I develop language models
> 5 days ago using the same data and smoothing techniques, but this warning m
> essage was no there.
Something must have changed. What was it? Has the software been updated?
>
> Could you please tell me the reason behind? Does it affect the quality of my
> language models?
The warning is issued because discount coefficients (the factors by which
the maximum likelihood estimates are reduced) should be between 0 and 1.
The value you are getting is effectively zero. It indicates an
anomaly (non-smoothness) in the count-of-count of your data.
--Andreas
Click here to go to the SRILM home page.