I am new to multi threading in Java, I am trying to fix a multi threading issue. My code is designed to extract store data from 2 different databases, writes it into separate files and merge them based on a criteria to a single file.This process is repeated for multiple clubs using multi threading by DataCollector Class that implements Runnable interface.

I have WorkFlowManager class that instantiates the start() method of DataCollector class with a specific club number. run method of DataCollector class has 6 method calls :
1. generateFileNameClub - used to generate file for database 1 as per standard naming convension dynamically.
2. extractDataClub - extract data from Database 1 and write data to file generated using generateFileNameClub method
3. generateFileNameHO - used to generate file for database 2 as per standard naming convension dynamically.

4. extractDataHO - extract data from Database 2 and write data to file generated using generateFileNameHO method
5. generateFileNameExtract - used to generate file as per standard naming convension dynamically to write the merged data.
6. MergeData - compare the data from file 1 and file 2 and merge based on basis of a criteria


These methods are defined in another Class CommonPIProcess which extends a gateway that has generic services to connect to data base and fetch results. Gateway is built using service locator design pattern.

The issue is that generated files for different threads are not synchronizing properly and writing into files of each other. I have used synchronized block for calling these methods for atomic execution. and used synchronized methods to execute extract process.

I am compiling my code on JRE 1.5.while the code somehow runs fine on JRE 1.5 , it completely breaks on JRE 1.4.2 where it has to be deployed.


I am testing only for 4 threads at a time. Please help me understand what am I doing wrong here.



code snippet of run method of DataCollector Class:
************************************************** *****
public void run() {

Properties p_sProp = null;
/* This block is added to calculate VAT inclusive pricing from Club Database for each Club# * */
synchronized (this) {
p_sProp = l_oCNPPricingProcess.generateFileNameExtract(l_sCl ubNbr);
Properties l_sPropHO = l_oCNPPricingProcess.generateFileNameHO(l_sClubNbr );
l_oCNPPricingProcess.extractDataHO(l_sClubNbr, l_sPropHO);
Properties l_sPropClub = l_oCNPPricingProcess .generateFileNameClub(l_sClubNbr);
l_oCNPPricingProcess.extractDataClub(l_sClubNbr,
l_sPropClub);
l_oCNPPricingProcess.MergeData(l_sPropHO, l_sPropClub,
p_sProp);
}

**************************************************

Definition of extractDataHO method

public synchronized void extractDataHO (String l_sClubNbr ,Properties l_pProps)

Service l_oExtractService = null;
File l_File = null;
String l_sExtractOutput = null;

String l_Pricing = super.getConfig().getProperty(
CommonPI_Property_Constants.PRICING_TYPE_CLUB);
clubBasedPricing = Boolean.parseBoolean(l_Pricing);
if ((l_sProcessName
.equalsIgnoreCase(CommonPI_Property_Constants.PROC ESS_CNPPRICING))
& (clubBasedPricing)) {
l_File = new File(l_pProps
.getProperty(Property_Constants.EXTRACT_OUTPUT_HO) );
l_sExtractOutput = l_pProps
.getProperty(Property_Constants.EXTRACT_OUTPUT_HO) ;

} else {
l_File = new File(l_pProps
.getProperty(Property_Constants.EXTRACT_OUTPUT));
l_sExtractOutput = l_pProps
.getProperty(Property_Constants.EXTRACT_OUTPUT);

}
if (l_File == null || l_File.exists())
return;

//Added for setting the sql query [Dipti]
String l_sSqlQuery = GatewayProcess.applySQLFilter(super
.getConfig().getProperty(
Property_Constants.EXTRACT_SQL_DATA), super
.getConfig().getProperty(
Property_Constants.PATTERN_FILTER_CATERGORY),
categories);
super.getConfig().setProperty(Property_Constants.E XTRACT_SQL,
l_sSqlQuery);
super.getConfig().setProperty(Property_Constants.E XTRACT_OUTPUT,
l_sExtractOutput);

super.getConfig().setProperty(
Property_Constants.EXTRACT_DRIVER,
super.getConfig().getProperty(
Property_Constants.DATA_DRIVER));
super.getConfig().setProperty(Property_Constants.E XTRACT_URL,
super.getConfig().getProperty(Property_Constants.D ATA_URL));
super.getConfig().setProperty(
Property_Constants.EXTRACT_USERNAME,
super.getConfig().getProperty(
Property_Constants.DATA_USERNAME));
String l_sPassword = super.getConfig().getProperty(
Property_Constants.DATA_PASSWORD);

if (l_sPassword != null) {
super.getConfig().setProperty(
Property_Constants.EXTRACT_PASSWORD, l_sPassword);
} else {
super.getConfig().getProperties().remove(
Property_Constants.EXTRACT_PASSWORD);
super.getConfig().setProperty(
Property_Constants.EXTRACT_APPLICATION_ID,
super.getConfig().getProperty(
Property_Constants.DATA_APPLICATION_ID));
super.getConfig().setProperty(
Property_Constants.EXTRACT_DATABASE_ID,
super.getConfig().getProperty(
Property_Constants.DATA_DATABASE_ID));
super.getConfig().setProperty(
Property_Constants.EXTRACT_HOST_ID,
super.getConfig().getProperty(
Property_Constants.DATA_HOST_ID));
super.getConfig().setProperty(
Property_Constants.EXTRACT_SECURITY_FILE_PATH,
super.getConfig().getProperty(
Property_Constants.DATA_SECCURITY_FILE_PATH));
}
String l_sInformixServer = super.getConfig().getProperty(
Property_Constants.DATA_INFORMIX_SERVER);

if (l_sInformixServer != null) {
super.getConfig().setProperty(
Property_Constants.EXTRACT_INFORMIX_SERVER,
l_sInformixServer);
} else {
super.getConfig().getProperties().remove(
Property_Constants.EXTRACT_INFORMIX_SERVER);
}

super.getConfig().setProperty(
Property_Constants.EXTRACT_DAO,
super.getConfig().getProperty(
Property_Constants.EXTRACT_DAO_DATA));

l_oExtractService = ServiceFactory.getInstance().getExtractService(
l_pProps); // Instantiate the Service Factory Class

String[] l_sArrQueryArgs = { l_sClubNbr };

ServiceIO serviceIO = new ServiceIO();
serviceIO.setQueryArgs(l_sArrQueryArgs);
l_oExtractService.setServiceIO(serviceIO);

l_oExtractService.run(); // Calling the run method on the Service
// instance
validateMin(l_oExtractService);

} catch (SQLException sqlEx) {
super.getLogger().onMethodException(this.getClass( ).getName(),
Message_Constants.EXTRACT,
Message_Constants.DATA_EXTRACTION_FAILED, sqlEx); // Sending
super.getAlert().raise(Message_Constants.DATA_EXTR ACT_FAIL_CHECK);
throw sqlEx;
} catch (Exception excep) {
excep.printStackTrace();
super.getLogger().onMethodException(this.getClass( ).getName(),
Message_Constants.EXTRACT,
Message_Constants.DATA_EXTRACTION_FAILED, excep); // Sending
super.getAlert().raise(Message_Constants.DATA_EXTR ACT_FAIL_CHECK);
throw excep;
}
}
**************************************************
similar method definition for synchronised extractDataClub only we set same properties with different values.

The program is running fine but giving unexpected results.