Results 1 to 7 of 7
  1. #1
    segolas is offline Member
    Join Date
    Jan 2010
    Posts
    7
    Rep Power
    0

    Default Heap Space Problem

    Hi,

    I'm writing an application which parse every email in a folder (emails are in text format). the problem is I've got lots of email, like 200-300 per folder, and many of them got big attachments, like 5mb or even 30mb.

    While the software iterate through the messages, they are loaded (with javamail) with this piece of code:

    InputStream source = new FileInputStream(msgFile);

    I think there were created too much InputStream objects before the garbage collector remove them from the memory.

    Is there any way to force the garbage collector? I'm in a for loop and I'm parsing one email per iteration. So I just need one message in memory.

    I found many solutions for increasing the heap size, but there really isn't any cleaner solution?

    Regards,
    Segolas

  2. #2
    masijade is offline Senior Member
    Join Date
    Jun 2008
    Posts
    2,571
    Rep Power
    8

    Default

    Then make sure you are closing the inputstreams and not saving any references to the "parsed" emails. It also depends on what you are doing with the "data" from the "parsed" emails. Make sure you are not "hanging" on to that in memory.

  3. #3
    segolas is offline Member
    Join Date
    Jan 2010
    Posts
    7
    Rep Power
    0

    Default

    the InputStream was close.

    I'm parsing the emails and grouping them basing on some header field. Than I build a hashtable like this:

    Message-id --> Vector (paths to messages)

    Buth this doesn't create any problem.

    Here is the software logic:

    MailManager.java is an object for parsing and manipulating the message. Of course it have a private File field holding the email.

    here is the for:

    for (every email in the folder){
    mm = new MailManager();
    // use the mm for populate the hashtable
    }

    the memory problem is when creating the new MailManager.

    I think there is only a way. Removing the file object from the MailManager and using a String with the path instead. So I can create the File every time I need it. Of course this will decrease the performance... but my app it's just a batch process, so isn't a big deal.

  4. #4
    masijade is offline Senior Member
    Join Date
    Jun 2008
    Posts
    2,571
    Rep Power
    8

    Default

    If he only information your "saving" from the mail is the message id and the filename, then that is not a problem, but I have no idea what you are doing with the "MailManager" class of yours. If that is the only thing you are calling in your loop, then it would definately help to see that code, don't you think?

    In any case, your best bet is to use a profiler and find out what is really using up the space, rther than "guessing" at the cause.

  5. #5
    segolas is offline Member
    Join Date
    Jan 2010
    Posts
    7
    Rep Power
    0

    Default

    Here is the code:

    Java Code:
    public class MailManager {
    
        private File msgFile; // here I hold the message file
        private MimeMessage message; // here is the MimeMessage
        private Logger logger;
        private Properties prop;
    
    
        /*
         * Constructor
         */
        public MailManager(LoadDefault def, File msgFile) {
            this.msgFile = msgFile;
            //LoadDefault def = new LoadDefault();
            // create the logger
            logger = def.getLogger(MailManager.class.getName());
            // load properties file
            prop = def.getProperties();
            
            if (prop.getProperty("verbose").equals("true")) {
                logger.setLevel(Level.ALL);
            }
            logger.fine("Proprietą lette con successo");
            // dummy... just for reading the message from a file
            Properties props = System.getProperties();
            props.put("mail.host", "smtp.dummydomain.com");
            props.put("mail.transport.protocol", "smtp");
            Session mailSession = Session.getDefaultInstance(props, null);
            try {
    
                InputStream source = new FileInputStream(msgFile); // here is where the exception is raised
                message = new MimeMessage(mailSession, source);
                source.close();
            } catch (FileNotFoundException ex) {
                Logger.getLogger(MailManager.class.getName()).log(Level.SEVERE, null, ex);
            } catch (MessagingException ex) {
                Logger.getLogger(MailManager.class.getName()).log(Level.SEVERE, null, ex);
            }
            catch(IOException ex){
                Logger.getLogger(MailManager.class.getName()).log(Level.SEVERE, null, ex);
            }
    
            logger.info("MailManager inizializzato con successo");
    
        }
    
    // lots of  methods

  6. #6
    masijade is offline Senior Member
    Join Date
    Jun 2008
    Posts
    2,571
    Rep Power
    8

    Default

    Other than the fact that the close should be done in a finally I don't see much glaringly wrong with that (although I have no idea what this loaddefault thing is) nor what the "lots of methods" does. So, as I have already said, use a profiler and stop guessing.

  7. #7
    segolas is offline Member
    Join Date
    Jan 2010
    Posts
    7
    Rep Power
    0

Similar Threads

  1. Java heap space,
    By babyboban in forum Advanced Java
    Replies: 18
    Last Post: 01-14-2010, 12:22 PM
  2. Problem with Java Heap Space using Socket
    By mvillara in forum Networking
    Replies: 3
    Last Post: 08-21-2009, 04:45 AM
  3. Java heap space OutofMemoryError
    By everlast88az in forum Advanced Java
    Replies: 3
    Last Post: 05-18-2009, 08:12 AM
  4. Java Heap Space
    By sandeeprao.techno in forum Advanced Java
    Replies: 19
    Last Post: 10-30-2008, 11:27 AM
  5. Java heap space?
    By javanewbie in forum New To Java
    Replies: 1
    Last Post: 06-24-2008, 06:55 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •