data loss in the secure file transfer using java
Hi,
My java Appllication uses the sftp(secure file transfer protocol) for file transferring. when the load is high like 150 GB , data loss occuring for some files especially last chunk of data is missing.
Core file transfer code
while (true)
{
int read = bis.read(buffer); //Reading bytes into the byte array (buffer) from network stream (bis).
if (read==-1) {
break; //Break the loop, if there are no more bytes available in the InputStream.
}
bos.write(buffer,0,read); //Writing the same number of bytes what we read into buffer byte array into OutputStream
complete.update(buffer,0,read); //Giving the same number of bytes what we read above to the checksum calculation.
bytesSoFar += read; //Incrementing the bytes that we have read from the source.
//if the bytes transferred are equal to the source file size
If (bytesSoFar == fileInfo.getFileSize().intValue())
{
// Calculate the file checksum.
// Update DbAuditLog with checksum and no. of bytes read
}
else
{
// Update DbAuditLog with no. of bytes read
}
}
bos.flush();
Please help , why the data loss is happening when the big size files transfer
Note:
1.150 gb not a single file multiple files.
2.SFTP no java specific API, So connection is happening through java .io.sockect connection
Due to data loss, checksum also not correct.
This issue not occurring continuously.. some times whole transfer failing or success.
Is NFS mount plays any role when this occurrence happening .. OS is HP-UX ia64 itanium version.
Now we are observing with glance tool which is provided by hp to analyze system activity
Application is running along with jconsole.
All the transaction are running under multithreaded way.. that means,most of the txns run multi threaded way , few txns run under singel thread. it depends on the threads available in the thread pool.
Try with a working Java SFTP tool
You should make a test with an applet that supports SFTP such as JFileUpload (there is a free full featured trial). If it works then it means that you have a problem in your code.
