Printable View
Hello!
Can someone point me to a book or a web site that has a lot of tutorial-type examples in the area of text (natural language) processing? Related to this is an exploding field of applications named "text analytics" and some of the major programs here are JAVA-based. Thanks in advance.
You know more about this than I do, so why don't you Google it? There is a reason why Google has it's own religion now.
Thanks. I was hoping to find a new colleague who is already "into" this class of JAVA-based applications and knows a good place where beginners would find annotated code.
Well before you jump into any special area of programming, its normally a good idea to understand the base language first.
Of course, and I **did** do that before posting my question in this Forum. I guess I am in the wrong place. Cheers!
If you want to parse a natural language, grammar based, have a look at Daniel Sleator's 'link grammar'. Basically it's just a contect free grammar parser with a huge dictionary of words including their inflections. Sorry, no link here but Google is your friend.Quote:
Originally Posted by theordore
kind regards,
Jos
jtmt - Java Text Mining Toolkit
It is a long trail of articles on textmining starting in 2008 and will take a lot of work and downloading libraries, but I got most of it working.
Thanks Jos, I will check your link. It will probably be useful if I can get at the source code. Yes, Google is helpful -- I found one site that specializes in string processing with JAVA. Cheers!
RE: jtmt - Java Text Mining Toolkit
Jodokus, I think that with you I have hit the jackpot; because my primary interest was in finding a colleague who is "into" this type of programming, so that we can communicate. I have been ramping up my JAVA learning (I already write in several other languages) in order to get into the two most well-known open source NLP packages, both of which are JAVA-based: LUCENE and GATE. I am doing this for my org., which will not approve the spending to buy the commercial software being sold by people like IBM and SAS. Their programs, esp. what IBM+SPSS is selling, seem very very powerful but even a one-desk license costs an arm and a leg. So I'm trying to see if we could get done what we need to do by modifying LUCENE (esp.). I'm interested to know whether your toolkit is available in some way, and will send you a follow-up private message.
@Theodore,
I surely don't hope that I gave the impression that it's MY toolkit. It is from a Californian programmer, Sujit Pal, who did a very good job learning textmining and developing code in the process that he published on his weblog.
When I said "I got it working" I ment reading it, downloading it and searching for the needed libraries. I'm willing to help you, but I'm not "The Sujit Pal" himself. I liked the trail because he developes it over the months and in my opinion it is impressive code, but I can't judge it compared with professional software.
I'll read your message tomorrow more thorough to understand your intentions better.
Thank you for your clarification, Jodokus. I look forward to hearing from you again once you have had a chance to reflect on the particular application that I have in mind.
I visited Sujit Pal's site after sending you my previous message, and looked at the number of the files. It is an impressive collection; but for someone coming into the field preoccupied with the practical application I have in mind, I think I would need a lot of introductory documentation in order to collect and properly link the modules he has that would be useful for my work. Unfortunately, he says that he is paid very little attention to documentation beyond a lot of clarifying comments inserted into his code, and trying to make progress by working with these comments would represent too great a time investment for me.
At the same time, I feel that I represent a class of professionals and small-business people that constitute an important opportunity to people like Sujit, if he's inclined to be entrepreneurial. Practical applications of natural language processing are " sprouting all over the place", and not only for the big companies in biotechnology and large-scale marketing to consumers that are supporting the related software work being done by giants like IBM and SAS.
People like me have a decent shot at gaining benefits from such applications, since we are already reasonably tech-savvy; but they have to be packaged and delivered to us with appropriate documentation ( about organizing input data for example) and "buttons for us to press", so that we can quickly use a program to go through a vast block of electronic text, and come back out with information that can help us in our work.
The time demands of our work are such that there is no chance for us to dig is deeply into the software design dimension as Sujit has, so if he wants to be successfully entrepreneurial in dealing with us he needs to meet us part-way between where he is as a software developer and where we are as people needing applications of natural language processing.
Cheers!
A warning first: it is considered very bad behaviour to take a discussion offline (away from the forum), especially one-sided. The normal reaction is to publish everything right away in the open, what I intend to do next time
(and not guaranteeing that I won't when it becomes relevant in my opinion). I definitily don't plan to discuss anything offside the forum.
This sounds rather fishy and rude to me. There is no reason to lecture "people like Sujit" how to approach "people like you". They are a rare breed of people to be grateful to, publishing their effort in the open for everyone to learn and use.Quote:
People like me have a decent shot at gaining benefits from such applications, since we are already reasonably tech-savvy; but they have to be packaged and delivered to us with appropriate documentation ( about organizing input data for example) and "buttons for us to press", so that we can quickly use a program to go through a vast block of electronic text, and come back out with information that can help us in our work.
The time demands of our work are such that there is no chance for us to dig is deeply into the software design dimension as Sujit has, so if he wants to be successfully entrepreneurial in dealing with us he needs to meet us part-way between where he is as a software developer and where we are as people needing applications of natural language processing.
But if you think he would be willing to be "entrepreneurial with "us"" just send him a mail, you might be successful.
RE: A warning first: it is considered very bad behaviour to take a discussion offline (away from the forum) ...
First, I originated the discussion. Second, the Forum software clearly provides for a private message. Third, are we to believe that people either publish all their thoughts or say nothing -- there is no room for a private discussion? You've got to be joking.
No, I'm not joking. It is irrelevant that you started the discussion. A thread is also there to be followed by other users who try to find solutions and to learn. It is inpolite to let them wonder where people are talking about because of them keeping information to themselves.
It is irrelevant that a tool exists as an argument to use it anyway you want. I could use it for example to ask certain people to help me find that recent thread where somebody was ridiculed to the ground (and his code published) because he send lots of sensitive code as a private message. But then my private message is not informative for others.
(????? Please help me find it or I send you a private message ?????)
You can also see it from the perspective of the receiver (me in this case). On a forum I'm free to answer when it suits me, when it doesn't other people take over. Now I'm lured into a kind of personal commitment I didn't ask for. When I get to know somebody he (or she!) and me can decide to start something beautiful and leave the forum wondering what we are doing.
One compliment: you could opt for "duck and cover", but you chose the confrontation.