Results 1 to 2 of 2
Thread: Lucene indexing help
- 07-25-2011, 08:16 AM #1
Member
- Join Date
- Oct 2010
- Posts
- 3
- Rep Power
- 0
Lucene indexing help
Hi
I am trying to refresh a lucene index in incremental mode that is updating documents that have changed and keeping other unchanged documents as it is. For updating changed documents I am deleting those documents using IndexWriter.deleteDocuments(Query) and then adding updated Lucene document using IndexWriter.addDocument(). The Query object used in the IndexWriter.deleteDocuments is containing approx 12-15 terms. In the process of refreshing the index I also sometimes need to do a FULL refresh by deleting all the documents using IndexWriter.deleteDocuments and then adding the new documents. The problem is when i called IndexWriter.flush() after say approx 100000 docs deletion it takes long time to execute and goes into OutOfMemoryError. If I disable flushing the indexing goes fast upto say 2000000 docs deletion and then it thows OutOfMemoryError. I have tried to set the IndexWriter.setRAMBufferSizeMB to 500 to avoid OutOfMemoryError but no luck. The index size is 1.8 GB. Please let me know if you have faced such prob and resolution for same
Thanks in advance
JP
- 07-25-2011, 05:34 PM #2
Member
- Join Date
- Jan 2011
- Posts
- 3
- Rep Power
- 0
Similar Threads
-
Indexing HTML with Lucene, null pointer exception
By stephanruhl in forum LuceneReplies: 1Last Post: 03-14-2011, 02:21 AM -
Digester parsing and Lucene indexing: empty index...
By aneuryzma in forum LuceneReplies: 0Last Post: 02-24-2011, 03:37 PM -
Indexing XML using lucene
By peliukasss in forum LuceneReplies: 0Last Post: 03-28-2010, 10:20 PM -
Lucene indexing ans searching code needed
By vgarg80 in forum JavaServer Pages (JSP) and JSTLReplies: 0Last Post: 06-07-2008, 11:31 AM -
Lucene Re-Indexing
By connect2srinath in forum LuceneReplies: 1Last Post: 05-11-2008, 05:35 PM


LinkBack URL
About LinkBacks
Reply With Quote
Bookmarks