How to generate bigram/trigram/unigram probability?

**findjoe** · 10-03-2010, 06:57 AM

Hi, all:

I want to generate word unigram/bigram/trigram probability. Is there an example to show how to do it? like "I am newbie......" in a file. The generated list may be:

bigram:
1. I am 0.23
2. I want 0.20
3. I like 0.19
...

I read a very short piece by Manning, but it does not show to compute.

Please help.

Thank you in advance.

**Norm** · 10-03-2010, 02:15 PM

How does this problem relate to java programming?

**findjoe** · 10-03-2010, 03:25 PM

It is closely related to Java for search engine building, abstract extraction, data estimates and prediction, speech recognition...

**Fubarable** · 10-03-2010, 03:51 PM

I'm with Norm as this seems to me to be an algorithm question, not a Java question per se.

**findjoe** · 10-04-2010, 08:01 AM

Dear Norm and Fubarable:

Thanks. I know little about Lucene. But, Is there sth in Lucene that generates bigram/trigram for indexing and search. Does it relate to Java programming?

Thanks.