Results 1 to 6 of 6
  1. #1
    gabel8 is offline Member
    Join Date
    Jan 2014
    Posts
    27
    Rep Power
    0

    Default Need recommendation

    I have a java swing application that is very data intensive. To be general about the application, I have a jtable with multiple fields that a user enters and can enter multiple lines. When they are done entering the line information, the application has to check whether or not this data has been entered before for a given user. The line information is stored in multiple tables so currently I do a query with table joins to retrieve all possible lines the user has entered and return the result set. This result set can get quite large with thousands of records. Then for each new line the user entered into the jtable, I have to check against every line from the result set unless a match is found and then the loop breaks. But this gets to be memory intensive and also takes a long time to execute. I was wondering if there is a better way to do this? Possibly using temp tables, etc? Any idea would be helpful at this point as this is currently crippling the application.

    Thanks.

  2. #2
    jim829 is offline Senior Member
    Join Date
    Jan 2013
    Location
    Northern Virginia, United States
    Posts
    6,226
    Rep Power
    14

    Default Re: Need recommendation

    Not certain I entirely understand. Is the user entering information about themselves? If so, wouldn't that be a good time to simply populate the table with what was already added previously. Are you comparing fields to each other for duplicate data or just checking if the field is filled or not?

    Regards,
    Jim
    The JavaTM Tutorials | SSCCE | Java Naming Conventions
    Poor planning on your part does not constitute an emergency on my part

  3. #3
    Xen
    Xen is offline Member
    Join Date
    Jan 2015
    Posts
    86
    Rep Power
    0

    Default Re: Need recommendation

    So basically (without me knowing anything about jTables) you have a history of previously entered lines in the database and you need to check each new line against the database.

    You are saying you are not using database mechanics to do the checking. This means you are operating on a returned result set and are doing these searches manually, so to speak, through thousands of lines, per user.

    It seems to mean that, while searching this set, you have no further interaction with the database server.

    However, it would seem to me that SQL servers are accustomed to and optimized for matching search queries like that. Ideally you would, or could, simply perform an SQL query that does the matching. This would require database communication but would ideally be very fast.

    On the other hand, you are not even mentioning a remote server.

    Typically if you wanted to speed up list searching, you would index the thing and this is of course the subject of database software and techniques. If it was sorted, you could use a binary tree or in any case a search tree or simply do a form of binary search on an array. Arrays (Java Platform SE 7 ).

    A binary search is simply the fastest search algorithm, or the simplest in any case (?) (my knowlegde is ....fading) for a sorted tree or list, and there are others that probably perform better. A binary search starts in the middle of the list and then jumps midway up or down depending on the comparison with the middle element. Then, in the half-list that results, it does this again, until it narrows down where the match should be and it finds this in absolutely sub-linear time (something like 2logn).

    If you can't sort your (history) list, you can still create a sorted index to the list, if you wanted.

    1. Put all the numbers of all the lines in a List or array
    2. Sort the numbers based on comparisons you do of the real list
    3. Whenever two lines numbers reference two real lines that would require to be sorted, swap the two numbers in the number list
    4. You end up with a 'garbled' or 'mixed' up numbers list as if the real list has been sorted (but it isn't).
    5. Every line/number in the sorted / mixed up list now indicates the proper position that number would have if really sorted.
    6. If you have a search string (or what you use) go to the middle of the number list, and compare the string to the string in the real list as referenced by the number you find. If it is "less than", then jump up (down) in the number list -- go to a lower index. Then repeat that step until when you compare the number you find (and its accompanying real list reference text) to your search string, you have found the exact match.
    7. In this way you will have reduced the linear search time you previously had to a 2-log-n search time without using any external tools, and you only need maybe one or two classes for it.

  4. #4
    Xen
    Xen is offline Member
    Join Date
    Jan 2015
    Posts
    86
    Rep Power
    0

    Default Re: Need recommendation

    I was referencing better search alternatives but I was really thinking of sort algorithms. Bubble Sort is the easiest and most easy to understand/write sort algorithm, but there are better ones. However, I don't think you can beat binary search on a list like this.

    I could even write this for you, could be fun. But I guess you can do it yourself, or I could provide a reference implementation using as much ready-made classes as I can (array and search and sort operations that already exist).

  5. #5
    JosAH's Avatar
    JosAH is offline Moderator
    Join Date
    Sep 2008
    Location
    Voorschoten, the Netherlands
    Posts
    14,422
    Blog Entries
    7
    Rep Power
    28

    Default Re: Need recommendation

    Quote Originally Posted by Xen View Post
    I was referencing better search alternatives but I was really thinking of sort algorithms. Bubble Sort is the easiest and most easy to understand/write sort algorithm, but there are better ones. However, I don't think you can beat binary search on a list like this.
    Yes you can: read Donald Knuth's volume III, "Searching and Sorting"; oh, and btw, never mention bubble sort in anything serious (more or less).

    kind regards,

    Jos
    Build a wall around Donald Trump; I'll pay for it.

  6. #6
    Xen
    Xen is offline Member
    Join Date
    Jan 2015
    Posts
    86
    Rep Power
    0

    Default Re: Need recommendation

    Quote Originally Posted by JosAH View Post
    Yes you can: read Donald Knuth's volume III, "Searching and Sorting"; oh, and btw, never mention bubble sort in anything serious (more or less).
    You'd have to find something that you can keep updated easily as you add more strings. Probably not hard. That book is expensive! It would not be a first choice for me to read/order, but it does seem very interesting (although I have read before on sort algorithms) (but never this much!!!).

Similar Threads

  1. Java Recommendation for task
    By neodjandre in forum New To Java
    Replies: 5
    Last Post: 05-21-2014, 10:18 AM
  2. wrapper recommendation
    By newbie123 in forum New To Java
    Replies: 0
    Last Post: 12-16-2011, 08:04 PM
  3. Replies: 5
    Last Post: 02-03-2011, 06:20 AM
  4. File Uploader Recommendation?
    By rsquaredpgh in forum Java Applets
    Replies: 1
    Last Post: 10-13-2010, 03:46 AM
  5. Recommendation about databases
    By tommy in forum JDBC
    Replies: 2
    Last Post: 07-28-2007, 05:04 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •