Results 1 to 2 of 2
  1. #1
    jojo is offline Member
    Join Date
    Dec 2009
    Posts
    36
    Rep Power
    0

    Default sql: store data in list or constantly ask the database?

    Hello,


    i got to a point where im intergrating databases in my java application. Im using postgresql, all is fine, i know how it works and what i have todo..

    but now im asking myself this:

    scenario:

    I will have lots of worker threads (100+) which all have to do simple tasks, but those simple tasks require data, and you guessed it, the data comes from the database.. now it will need the data every 1-2mins, acutally it needs only the next row which is randomly chosen.

    But as these list are pretty large i guess.. contains about 100-200 items, and as they are stored in the RAM memory (this is right or isn't???) i think my 3Gig ram will be eaten in no time.. so i was thinking about something like contacting the database and ask everytime a single row... but this will require memory too, but i guess it will use less then storing all my data in lists, and i the gc will release the memory after the request was succesful(?)

    What i want to mention also, is that multiple worker threads use the same list to work...

    Now my question is what will be the best..??

    btw now that i'm writing this topic, im thinknig about caching resultsets, like caching 20 items out of my 100 items in database.. and if its a request is made and the cachelist already contains it no need, else i will remove the tail of the list and add the new request.. what you guys think???


    I know this is pretty advanced, at least i think xD, so if someone has experience with this, post!

  2. #2
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    11,846
    Rep Power
    19

    Default

    "100-200 items".
    Unless each item is big (lots and lots of columns), then that isn't large, so holding the entire list may not be a bad idea.
    If, however, this is likely to expand over time (into the thousands and thousands) then I'd consider requesting from the db each time.

    Also, every 1-2 minutes is a really slow data rate to require, so the database side might be a good choice anyway.

Similar Threads

  1. how to store images in access database
    By sridarshan in forum JDBC
    Replies: 11
    Last Post: 06-26-2010, 02:26 AM
  2. What would be the best way to store a large Database with J2me
    By father jack hackett in forum CLDC and MIDP
    Replies: 0
    Last Post: 04-08-2010, 06:08 PM
  3. how to store the data in data base
    By eclipse3.4ide in forum New To Java
    Replies: 5
    Last Post: 02-03-2009, 04:25 AM
  4. How to store Multiple calendars in PIM Database
    By thirupathik in forum CDC and Personal Profile
    Replies: 0
    Last Post: 07-13-2008, 01:45 PM
  5. How to store extracted xml values in a database?
    By palanikumark in forum Advanced Java
    Replies: 6
    Last Post: 05-30-2008, 12:14 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •