Multi-threading with shared varibles

[gump74]

Greetings,
I been searching the net and so far I have not seen something that can help. I am not new to Java but I do have limited exprience with threads. The programs I have written with threads were animations in swing. I would like to upgrade my slow program to have have multi-thread capabilities so it will run faster, the issue I am having is volatile varibles cannot be accessed by the static methods, and I do not know what to do. The program that I wrote is very complex and I don't want to waste anyones time trying to figure out what I have done so far without threads so I will make up a scenario that will be less complex. The basics of the program is to sort and store data as groups of ranges. I will write some psudo-code to represent my scenario. The program will read a file pull the names of people and save the names in listdata. It will then parse through listdata and group the names by last name and then first name.

class names
arraylist <String[]> listdata = new arraylist <String[3]>
arraylist <classA> NGroups = new arraylist <classA>

main ()
readInput() // this will read the contents of the file and save the names(last, first) into listdata

//parse listdata
loop //through the listdata collection
if NGroups contains last name
find index of last name
NGroups.get(x).process( last, first) // line 15
else
NGroups.add(lat, first) // line 17

// report the findings. it will show the names last, first, first. Smith, Bob, Bill, Mark ...
loop //through all the NGroups
NGroups.report()


That is a more sipmple example of the program. In my case I have 4.2 million elements in listdata and running the program takes about 8.5 hours. I would like to process lines 15 and 17 each to be proccessed as threads so the program can move on to the next name to be processed. The issue I have run into so far is that I do not know how to pass the names to the thread and more importantly if I make NGroups volatile then I cannot use it at the report section. If I could get each name to be processed and stored as a thread I belive the program would run much faster than 8 hours. I am not really concerned if one thread finishes before some other thread. Does any one have any ideas?

[jim829]

Check out concurrency in The Java™ Tutorial and also java.util.concurrent in the Java Platform SE 7

Regards,
Jim

[Steve11235]

A couple of suggestions. First, static methods cannot access instance variables directly. You have to go through the instance. Second, volatile variables can add a lot of overhead if you constantly access them. You may be better off using a synchronized block at certain points if one thread is constantly accessing variables, while another thread only accesses them occasionally. Third, use the concurrent collections. They eliminate much of the need for synchronization. Fourth, try to maintain separation of concerns between threads. In other words, don't have two threads working on the same data. Last, be sure that any updates to Swing components go through the EventQueue.doLater().

[DarrylBurke]

Um, EventQueue (or SwingUtilities) #invokeLater(...)

db

[gump74]

Greetings all,
Sorry for the long delay. It does seem that I should be using concurent collections to do the work however, I am getting an error. I will reference what was working without concurentcy.

ArrayList<classB> classbd = new ArrayList<classB>();
classbd.add( new classB( CC, AC, OC, EX, dir ) );

I now have;

NavigableSet <classB>classbd = new ConcurrentSkipListSet<classB>();
classbd.add( CC, AC, OC, EX, dir );

classb is

public classB( String CC, String AC, String OC, String EX, String dir ){
add( CC, AC, OC, EX, dir);
}

the error I am getting is;

classa.java:48: cannot find symbol
symbol : method add(java.lang.String,java.lang.String,java.lang.St ring,java.lang.String,java.lang.String)
location: interface java.util.NavigableSet<classB>
classbd.add( CC, AC, OC, EX, dir );

I have also tried;

classbd.add( new classB( CC, AC, OC, EX, dir ) );

and got this error, even though it is not doing a compare;

Exception in thread "main" java.lang.ClassCastException: classB cannot be cast to java.lang.Comparable
at java.util.concurrent.ConcurrentSkipListMap.compara ble(Unknown Source)
at java.util.concurrent.ConcurrentSkipListMap.doPut(U nknown Source)
at java.util.concurrent.ConcurrentSkipListMap.putIfAb sent(Unknown Source)
at java.util.concurrent.ConcurrentSkipListSet.add(Unk nown Source)
at classA.add(classa.java:47)
at LdapSummary.main(LdapSummary.java:34)

It doesn't seem like this is a major issue but the solution seems to be escaping me.
Any Ideas.

[Tolls]

Please use [code] tags [/code] when posting code.
ConcurrentSkipListSet is a sorted set, so requires its elements implement Comparable.

If you just want a synchronised set then use Collections.synchronizedSet().

[gump74]

I will use the code tags next time, sorry for the inconvenience.
The syncronised set sounds good I will give it a try. If everything works ok should I post the working code so people can use it as an example?

[gump74]

Ok, I don't think I've had this many issues in one program ever. I have found some examples that are close to what I am looking for, a syncronized arraylist but not a syncronized arraylist of type classB. I have tried several different ways of trying to modify the contructor to make it type classB but I end up with issues.

The example that is the closest to what I was looking for is

ArrayList classbd = (ArrayList) Collections.synchronizedList(new ArrayList());

but this is not of type classB and my enhanced for loop fails.

ArrayList<classB> classbd = (ArrayList) Collections.synchronizedList(new ArrayList<classB>());

does not compile

ArrayList<classB> classbd = (ArrayList<classB>) Collections.synchronizedList(new ArrayList<classB>());

does compile but when I try to execute it I receive the following error;
>java -Xmx1024m LdapSummary
Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: java.lang.ClassCastException: java.util.Collections$SynchronizedRandomAccessList cannot be cast to java.util.ArrayList
at classA.<init>(classa.java:16)
at LdapSummary.<clinit>(LdapSummary.java:14)
Could not find the main class: LdapSummary. Program will exit.

The original code of this program was not syncronized or treadded but it was

ArrayList<classB> classbd = new ArrayList<classB>();

It seems like I should be able to cast the syncronized arraylist to calssB. Can someone tell me why this is happening?

[Tolls]

List<classB>, not ArrayList<classB> as the type of the 'classbd' variable.
You are asking for a List that's synchronised.

List<classB> classbd = Collections.synchronizedList(new ArrayList<classB>());

Also, no need to cast anything as it's a generic method, so it understands the type of ArrayList should also be the type of the returned List.

[gump74]

Thank you tolls, the program is now running. It is not running as fast as I had hoped but by my math the processing time should be between 5 and 7 hours, where it was running in 9.5 to 10 hours. I guess I need more cores.

[jim829]

I am curious as to what type of processing you are doing that would take that long? Not that there aren't processes which take a lot longer (like factoring very large numbers). Just a matter of curiosity.

Regards,
Jim

[gump74]

Hello Jim,
The program is used to summarize information from a database server. The database server is used to store data about phone numbers and information associated with phone numbers. There is almost 5 million numbers stored in the server. What I need is to summarize the numbers into ranges. Numbers are broken down into country code, area code, office code and then extensions. The people I work for want to know how many office code are in use in a given area code, or how many extensions are in use in a given office code, for example. Also I need this so I can maintain the records in the database so I know what is no longer needed or what I need to add, the only way I know if something is needed now is if something fails to work. I hope I have explained it well enough for an understanding.

[jim829]

Thanks for the explanation. I was just wondering if a HashMap or HashMap of HashMaps might not be useful in this situation. But since I don't know how you are processing the data or all that you need to do, it was just thought. No need for more info. Good luck on your task.

Regards,
Jim