Appending to a File Used for Serialization

[Lil_Aziz1]

I was reading on Serialization and I started playing with it. Then I wondered if it was possible to serialize the same object again in the same file but different times. Here is an example:

This class will be our perpetual class.

import java.io.Serializable;
import java.util.Date;


public class PersistentTime implements Serializable {
	
	private Date time1;
	
	public PersistentTime() {
		time1 = new Date();
	}
	public Date getTime() {
		return time1;
	}

}

Run the idiom below twice so four PersistentTime objects are be conserved. (Only two are saved I think) Wait two seconds between the first and second run because there is a Thread.sleep() in it.

import java.io.FileOutputStream;
import java.io.ObjectOutputStream;


public class FlattenTime {
	public static void main(String[] args) throws Exception {
		
		String filename = "time.ser";
		PersistentTime time = new PersistentTime();
		
		Thread.sleep(1000);
		
		PersistentTime time1 = new PersistentTime();
		FileOutputStream fos = null;
		ObjectOutputStream out = null;
		
		fos = new FileOutputStream(filename, true);
		out = new ObjectOutputStream(fos);
		out.writeObject(time);
		out.writeObject(time1);
		out.close();

	}

}

The code below reads the 4 objects (only reading 2).

import java.io.FileInputStream;
import java.io.ObjectInputStream;
import java.util.Date;


public class InflateTime {
	public static void main(String[] args) throws Exception {
		
		String filename = "time.ser";
		PersistentTime time1 = null, time2 = null, time3 = null, time4 = null;
		FileInputStream fis;
		ObjectInputStream in;
		
		fis = new FileInputStream(filename);
		in = new ObjectInputStream(fis);
		time1 = (PersistentTime) in.readObject();
		time2 = (PersistentTime) in.readObject();
		time3 = (PersistentTime) in.readObject();
		time4 = (PersistentTime) in.readObject();

		
		System.out.println("Current Time: " + new Date());
		System.out.println("Flattened Time #1: " + time1.getTime());
		System.out.println("Flattened Time #2: " + time2.getTime());
		System.out.println("Flattened Time #3: " + time3.getTime());
		System.out.println("Flattened Time #4: " + time4.getTime());
	}

}

Output:

Current Time: Sun Jan 31 14:56:11 CST 2010
Flattened Time #1: Sun Jan 31 14:55:58 CST 2010
Flattened Time #2: Sun Jan 31 14:56:00 CST 2010
Exception in thread "main" java.lang.NullPointerException
	at InflateTime.main(InflateTime.java:29)

^As you can see, it's only reading two times, instead of four. :(

I have another question. Would it be better to wrap the FileInputStream and FileOutputStream into BufferedInputStream and BufferedOutputStream before sending it to the ObjectInputStream/ObjectOutputStream?

Like this:

		fos = new FileOutputStream(filename, true);
		out = new ObjectOutputStream(new BufferedOutputStream(fos));

If yes, why?

Any help is appreciated. Thanks in advance.

[masijade]

As per the API docs for ObjectOutputStreams constructor that takes an OutputStream as an argument (and the "default" constructor is protected, and so, only for use by your own ObjectOutputStream class extensions)

Creates an ObjectOutputStream that writes to the specified OutputStream. This constructor writes the serialization stream header to the underlying stream; callers may wish to flush the stream immediately to ensure that constructors for receiving ObjectInputStreams will not block when reading the header.

So, there are five objects in the file, not four. The third one being the "new" header.

[Lil_Aziz1]

Umm... So what should I do? Should I call flush right after time2? Like this?:

		time1 = (PersistentTime) in.readObject();
		time2 = (PersistentTime) in.readObject();
                in.flush();
		time3 = (PersistentTime) in.readObject();
		time4 = (PersistentTime) in.readObject();

Doesn't flush empty the buffer? If yes, what exactly does that mean?

[travishein]

Well, since the ObjectOutputStream is writing some kind of header chatter into the file, what if we re-initialized the objectinput stream without closing the underlying file input stream in between. this is kind of like a read equivalent of append write..

public class InflateTime {
	public static void main(String[] args) throws Exception {
		
		String filename = "time.ser";
		PersistentTime time1 = null, time2 = null, time3 = null, time4 = null;
		FileInputStream fis;
		ObjectInputStream in;
		
		fis = new FileInputStream(filename);
		in = new ObjectInputStream(fis);
		time1 = (PersistentTime) in.readObject();
		time2 = (PersistentTime) in.readObject();

		in = new ObjectInputStream(fis);
		time3 = (PersistentTime) in.readObject();
		time4 = (PersistentTime) in.readObject();

	
		System.out.println("Current Time: " + new Date());
		System.out.println("Flattened Time #1: " + time1.getTime());
		System.out.println("Flattened Time #2: " + time2.getTime());
		System.out.println("Flattened Time #3: " + time3.getTime());
		System.out.println("Flattened Time #4: " + time4.getTime());
	}
}

Dunno, does that work?

[Lil_Aziz1]

:O it works! I don't get how though. :( Like how does re-initializing make it work? What's a header chatter?

Also, I'd be very grateful if these two questions can also be answered:

1. Does flush empty the buffer? If yes, what exactly does that mean? If no, I still want to know what flush is.

2. Would it be better to wrap the FileInputStream and FileOutputStream into BufferedInputStream and BufferedOutputStream before sending it to the ObjectInputStream/ObjectOutputStream?

Like this:

		fos = new FileOutputStream(filename, true);
		out = new ObjectOutputStream(new BufferedOutputStream(fos));

If yes, why?

[travishein]

When object output stream is constructed, it writes out some small administrative data, probably like the java vm version, not really sure there. This sort of preamble happens before you give it an object to serialize out. So i just reasoned the ObjectInputStream must know to also try to read this what ever kind of header they put there.


1. yes, flush() method of an output stream is available for the kinds of output streams that support buffering to mean to have the buffer flushed out, such as write the stuff in the memory buffer to the file or what ever is backing the output stream. I'm not entirely sure, but i think the flush() method is internally called by the close() method in some output streams, that is to have the effect when you want to close it, if there was anything still in the memory buffer, to have them written out before closing the file.

2. this i am not sure. It will really depend on the size of the objects. and how many of them. I know if I make a program that reads something from some data source and I want to write out say a .csv file, where there are many many lines that I would write and each line is comparatively small, in this case it makes sense to use a buffered output stream to wrap around the underlying file output stream, because i notice when i do not use a buffer, the disk is used for every single line write, and this can have an impact on how the program performs. So using a buffered output stream around the file output stream you have there should also work. to help reduce the amount of disk write operations if you were going to write many small sized objects over a long period of time. But im sure for the cases where you only will be writing a few objects, or if you do the open the output stream, write one big bunch of objects, then close the outut stream, then using a buffer would not really add any performance gain here. It won't hurt things either, it just would not be noticable.

[Lil_Aziz1]

Ooh alright. To get everything cleared up, a buffer is like transient memory. When used in an input stream (specifically a file), everything from the file is written in the buffer (making the buffer full). When the buffer is full, methods like readLine() can be used. One can use methods like readLine() until the buffer is empty or when you flush() or close() (which empty the buffer).

One final question (Sorry :S). Is my definition of a buffer correct?

btw, Thanks a ton! I would rep you but it won't let me.

[travishein]

Yes, I would say that how you are thinking of a buffer as it is used for input and output streams is correct. Where for input streams, the buffer tries to read ahead a bit from the current spot where your program has read so far, so as to try to have some of the next things available in this transient memory internal to the buffered input stream. and a buffered output stream tries to use its buffer of internal transient memory to pool up some of the write operations, until the internal threshold or capacity is reached or the what ever sensible mechanism it uses, and then writes the buffer contents out all at once, such as to reduce the number of small content sized discrete write operations, which if the output stream happens to be writing to a file system, can improve performance.

and the flush() method tells the buffered output stream to go ahead and write the buffer contents now, instead of waiting for the next time it would automatically write it out.

Note though, that there is no flush() for buffered input stream, this is only relevant for the buffered output stream.

[Lil_Aziz1]

Ah so having a flush() in the close() method would make perfect sense. Thank you verrrry much!

I'm not going to mark this thread as SOLVED because I might have more questions on streams. Once again, thank you!