Export query result, procces is to slow

[Daniel]

I have to create an export of a query result, the number of rows always are more than 2 Millons. My proccess starts through a JSP file.
The application is running into a Apache Tomcat 5.5
I have tried to write a txt file but the procces is to slow
I don't know if you can tell me some tips ?

Daniel

[Albert]

2 millon rows is alot to export no matter what you do. One suggestion would be to split up the query into multiple threads. Basically you create a thread to do the query and some number of threads to write out separate files. This isn't easy but you'd then be limited by the I/O of the system and not anything that java is doing.

Albert

[Marcus]

I guess that an important question would be what do you do with data after it is exported? If you are reading it with a different Java process then you might get some performance gains with serialization.

However, that would require Java to read the file. If this data is to be imported to something else then can you skip the export to file altogether and write it directly to the destination?

Lastly, do you know where your bottleneck really is? Is it Java or the database or the file I/O or what? Before going too far in optimizing the code you might want make sure that you are optimizing the correct thing.

Marcus