Results 1 to 14 of 14
  1. #1
    matzahboy is offline Member
    Join Date
    Nov 2008
    Posts
    67
    Rep Power
    0

    Default A more efficient way to read from a text file

    I have a large text file (about 270,000 lines with 1 word per line, 2.9 MB). I need to read it into a String array when my program starts. The array size needs to be the exact number of words (i.e. no empty spaces). However, with my current code, it takes several seconds for the array to load. Is there a more efficient way to code this?

    Java Code:
    private void loadDict(String dictPath) {
        	//read the dictionary
        	try {
        		Scanner s = new Scanner(new BufferedReader(new FileReader(dictPath)));
        	
        		ArrayList<String> d = new ArrayList<String>(300000);
        		while(s.hasNext()) {
        			d.add(s.nextLine());
        		}
        		d.trimToSize();
        		dict = new String[d.size()];
        		for(int i=0; i<d.size(); i++)
        			dict[i]=d.get(i);
        	}
        	catch (FileNotFoundException e) {
        		System.out.println("Error loading the dictionary");
        		System.exit(1);
        	}
        }

  2. #2
    mrmatt1111's Avatar
    mrmatt1111 is offline Senior Member
    Join Date
    Aug 2009
    Location
    San Jose, CA, USA
    Posts
    320
    Rep Power
    5

    Default

    You might try:

    Instead of this:
    Java Code:
    dict = new String[d.size()];
    for(int i=0; i<d.size(); i++)
       dict[i]=d.get(i);
    try this:

    Java Code:
    dict = d.toArray(new String[d.size()]);

    You might also try putting more then one word (like 10) per line in the file rather then just one word per line. It might be faster.
    My Hobby Project: LegacyClone

  3. #3
    matzahboy is offline Member
    Join Date
    Nov 2008
    Posts
    67
    Rep Power
    0

    Default

    Quote Originally Posted by mrmatt1111 View Post
    try this:

    Java Code:
    dict = d.toArray(new String[d.size()]);
    Thanks a lot for that (that solves part of the problem). I didn't know that there was an optional parameter for the toArray method.

  4. #4
    matzahboy is offline Member
    Join Date
    Nov 2008
    Posts
    67
    Rep Power
    0

    Default

    I have a new question related to this. Is there a way to have this import method run while I'm getting user input from the keyboard (so that by the time the user is done typing, this import method will be done or close to done)?

  5. #5
    camickr is offline Senior Member
    Join Date
    Jul 2009
    Posts
    1,234
    Rep Power
    7

    Default

    Use a separate Thread to read the data from the file.

  6. #6
    matzahboy is offline Member
    Join Date
    Nov 2008
    Posts
    67
    Rep Power
    0

    Default

    Quote Originally Posted by camickr View Post
    Use a separate Thread to read the data from the file.
    How do I do that?

  7. #7
    r035198x is offline Senior Member
    Join Date
    Aug 2009
    Posts
    2,388
    Rep Power
    7

    Default

    You read about threads and make an attempt.

  8. #8
    matzahboy is offline Member
    Join Date
    Nov 2008
    Posts
    67
    Rep Power
    0

    Default

    Quote Originally Posted by r035198x View Post
    You read about threads and make an attempt.
    Okay. I'll go do that.

  9. #9
    Willi's Avatar
    Willi is offline Member
    Join Date
    Oct 2009
    Posts
    40
    Rep Power
    0

    Default

    Hi! Can someone explain why the use of array is more efficient than a String? I think this is a common question to any newbie like me! Thanks ^_^
    Write a program to achieve anything you want in your life!

  10. #10
    Willi's Avatar
    Willi is offline Member
    Join Date
    Oct 2009
    Posts
    40
    Rep Power
    0

    Default

    Does this have anything related to String Constant Pool?
    Write a program to achieve anything you want in your life!

  11. #11
    r035198x is offline Senior Member
    Join Date
    Aug 2009
    Posts
    2,388
    Rep Power
    7

    Default

    Quote Originally Posted by Willi View Post
    Hi! Can someone explain why the use of array is more efficient than a String? I think this is a common question to any newbie like me! Thanks ^_^
    Arrays and strings are used for different things. You don't substitute one for the other in the hope of improving efficiency.
    PS Why post your question in someone else's thread?

  12. #12
    Willi's Avatar
    Willi is offline Member
    Join Date
    Oct 2009
    Posts
    40
    Rep Power
    0

    Default

    Quote Originally Posted by r035198x View Post
    Arrays and strings are used for different things. You don't substitute one for the other in the hope of improving efficiency.
    PS Why post your question in someone else's thread?
    The reason I ask this question is I have been told about this several times. When I posted my solution in some other forum, some seniors told me that in some situation, when we want to deal with single characters in a string, then we should use an array of characters rather than a String. :confused:
    Write a program to achieve anything you want in your life!

  13. #13
    r035198x is offline Senior Member
    Join Date
    Aug 2009
    Posts
    2,388
    Rep Power
    7

    Default

    Yeah well if you want to deal with single characters then you do need a char array.
    Using a String would be just using the wrong tool for the job.

  14. #14
    Willi's Avatar
    Willi is offline Member
    Join Date
    Oct 2009
    Posts
    40
    Rep Power
    0

    Default

    problem solved. Thank you!
    Write a program to achieve anything you want in your life!

Similar Threads

  1. Read and edit text file
    By VinTiger in forum New To Java
    Replies: 5
    Last Post: 05-14-2009, 01:18 AM
  2. Read from a text file?
    By aaronfsimons in forum New To Java
    Replies: 3
    Last Post: 05-01-2009, 04:42 AM
  3. read from text file
    By rayda in forum New To Java
    Replies: 5
    Last Post: 04-10-2009, 03:51 AM
  4. Read and modify text file
    By heartysnowy in forum New To Java
    Replies: 27
    Last Post: 11-19-2008, 10:03 AM
  5. How to read a text file from a Java Archive File
    By Java Tip in forum Java Tip
    Replies: 0
    Last Post: 02-08-2008, 09:13 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •