Hello, I have an assignment where we are to open a file, "tokenize" it, then print each token into an output file. This is easy with the string tokenizer class, but we cannot use it. We have to come up with a way around it. This is what I have so far, I want it to read characters, keep adding them to an empty string, when it reads a whitespace, get out of loop, save that string into an array of strings, and continue again....

Java Code:
		String filename = "input.txt";

		String[] tokens = new String[20];
		
	      try{
	    // Open the file that is the first 
	    // command line parameter
	    	  StringBuffer buffer = new StringBuffer();
	    FileInputStream fstream = new FileInputStream(filename);
	    // Get the object of DataInputStream
	    DataInputStream in = new DataInputStream(fstream);
	        BufferedReader br = new BufferedReader(new InputStreamReader(in));

	    int ch;
	    int i=0;
		while ((ch = in.read()) > -1) 
		{
			if(Character.isLetter(ch) || Character.isDigit(ch))
			{
				buffer.append((char)ch);
			}
			tokens[i] = buffer.toString();
			i++;
		}
	    
               //print array
		for (i=0; i < tokens.length; i++)
			System.out.println (tokens[i]);
	    
	    //Close the input stream
	    in.close();
	    }catch (Exception e){//Catch exception if any
	      System.err.println("Error: " + e.getMessage());
There might be an easier way of doing this? Reading chars from a .txt file, seeing if they are a letter or digit?