Results 1 to 2 of 2
  1. #1
    JP10 is offline Member
    Join Date
    Oct 2010
    Posts
    3
    Rep Power
    0

    Default Lucene indexing help

    Hi
    I am trying to refresh a lucene index in incremental mode that is updating documents that have changed and keeping other unchanged documents as it is. For updating changed documents I am deleting those documents using IndexWriter.deleteDocuments(Query) and then adding updated Lucene document using IndexWriter.addDocument(). The Query object used in the IndexWriter.deleteDocuments is containing approx 12-15 terms. In the process of refreshing the index I also sometimes need to do a FULL refresh by deleting all the documents using IndexWriter.deleteDocuments and then adding the new documents. The problem is when i called IndexWriter.flush() after say approx 100000 docs deletion it takes long time to execute and goes into OutOfMemoryError. If I disable flushing the indexing goes fast upto say 2000000 docs deletion and then it thows OutOfMemoryError. I have tried to set the IndexWriter.setRAMBufferSizeMB to 500 to avoid OutOfMemoryError but no luck. The index size is 1.8 GB. Please let me know if you have faced such prob and resolution for same

    Thanks in advance
    JP

  2. #2
    gadek.a is offline Member
    Join Date
    Jan 2011
    Posts
    3
    Rep Power
    0

    Default

    try to reduce setMaxBufferedDeleteTerms doing it more often u'll need less resources

Similar Threads

  1. Indexing HTML with Lucene, null pointer exception
    By stephanruhl in forum Lucene
    Replies: 1
    Last Post: 03-14-2011, 03:21 AM
  2. Replies: 0
    Last Post: 02-24-2011, 04:37 PM
  3. Indexing XML using lucene
    By peliukasss in forum Lucene
    Replies: 0
    Last Post: 03-28-2010, 11:20 PM
  4. Lucene indexing ans searching code needed
    By vgarg80 in forum JavaServer Pages (JSP) and JSTL
    Replies: 0
    Last Post: 06-07-2008, 12:31 PM
  5. Lucene Re-Indexing
    By connect2srinath in forum Lucene
    Replies: 1
    Last Post: 05-11-2008, 06:35 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •