Results 1 to 11 of 11
  1. #1
    Krons is offline Member
    Join Date
    Apr 2009
    Posts
    2
    Rep Power
    0

    Default Improve write speed

    Hi!

    I have the following code to write in a file some info. The problem is this method is too slow when it's try to write a 8mb file. ¿What can I do to improve the speed?. It takes more than 1 min now...Thanks!


    Java Code:
    
    
    
        public Boolean guardarJugadores(TreeMap hashJugadores){
    
            Boolean res = false;
    
            try {
    
                RandomAccessFile fichero = new RandomAccessFile(hashOpciones.get("jugadores").toString()+".temp", "rw");
                Iterator it = hashJugadores.values().iterator();
                Iterator itRegistros = hashJugadores.values().iterator();
    
                fichero.writeDouble(Constantes.principioArchivo);
                fichero.writeDouble(0);
                fichero.writeInt(Integer.reverseBytes(Constantes.cantJugadores));
    
                while(it.hasNext()){
    
                    Jugador jug = (Jugador) it.next();
                    Cabecera cab = jug.getCabecera();
    
                    fichero.writeInt(Integer.reverseBytes(cab.getId()));
                    fichero.writeByte(0);
                    fichero.writeInt(Integer.reverseBytes(cab.getInicio()));
                    fichero.writeInt(Integer.reverseBytes(cab.getLargo()));
    
                    jug = null;
                    cab = null;
                    
                }
    
                while(itRegistros.hasNext()){
    
                    Jugador jugador = (Jugador) itRegistros.next();
    
                    fichero.writeShort(Short.reverseBytes(jugador.getSumaLargos()));
                    fichero.writeShort(Short.reverseBytes(Constantes.inicioRegistro));
                    fichero.writeBoolean(jugador.getTipoJugador());
                    cFuncion.writeUnsignedShort(jugador.getIdJugador(), fichero);
                    fichero.write(jugador.getDorsal());
    
                    fichero.writeShort(Short.reverseBytes(jugador.getLargoNombreCorto()));
                    cFuncion.grabarCadena(jugador.getNombreCorto(), jugador.getLargoNombreCorto(), fichero);
                    
                    fichero.writeShort(Short.reverseBytes(jugador.getLargoNombre()));
                    cFuncion.grabarCadena(jugador.getNombre(), jugador.getLargoNombre(), fichero);
    
                    fichero.write(jugador.getDorsalGenerico());
                    fichero.writeBoolean(jugador.getUtilizaDorsalGenerico());
    
                    fichero.write(jugador.getRol1());
                    fichero.write(jugador.getRol2());
                    fichero.write(jugador.getRol3());
                    fichero.write(jugador.getRol4());
                    fichero.write(jugador.getRol5());
                    fichero.write(jugador.getRol6());
    
                    fichero.write(jugador.getNacionalidad());
                    fichero.write(jugador.getColorPiel());
                    fichero.write(jugador.getColorPelo());
                    fichero.write(jugador.getPosicion());
                    fichero.write(jugador.getPelo());
                    fichero.write(jugador.getBelloFacial());
                    fichero.writeBoolean(jugador.getNacionalizado());
    
                    fichero.write(jugador.getDiaNacimiento());
                    fichero.write(jugador.getMesNacimiento());
                    fichero.writeShort(Short.reverseBytes(jugador.getAnyoNacimiento()));
    
                    fichero.write(jugador.getAltura());
                    fichero.write(jugador.getPeso());
    
                    if(!jugador.getTipoJugador()){
                        fichero.write(jugador.getPaisNatal());
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoCiudadNacimiento()));
                        cFuncion.grabarCadena(jugador.getCiudadNacimiento(), jugador.getLargoCiudadNacimiento(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoEquipoProcedencia()));
                        cFuncion.grabarCadena(jugador.getEquipoProcedencia(), jugador.getLargoEquipoProcedencia(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoVecesInternacional()));
                        cFuncion.grabarCadena(jugador.getVecesInternacional(), jugador.getLargoVecesInternacional(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoCadena1()));
                        cFuncion.grabarCadena(jugador.getCadena1(), jugador.getLargoCadena1(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoCualidadesTecnicas()));
                        cFuncion.grabarCadena(jugador.getCualidadesTecnicas(), jugador.getLargoCualidadesTecnicas(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoPalmares()));
                        cFuncion.grabarCadena(jugador.getPalmares(), jugador.getLargoPalmares(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoInternacional()));
                        cFuncion.grabarCadena(jugador.getInternacional(), jugador.getLargoInternacional(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoAnecdotario()));
                        cFuncion.grabarCadena(jugador.getAnecdotario(), jugador.getLargoAnecdotario(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoUltimaTemporada()));
                        cFuncion.grabarCadena(jugador.getUltimaTemporada(), jugador.getLargoUltimaTemporada(), fichero);
    
                        fichero.writeShort(Short.reverseBytes(jugador.getLargoTrayectoria()));
                        cFuncion.grabarCadena(jugador.getTrayectoria(), jugador.getLargoTrayectoria(), fichero);
                    }
    
                    fichero.write(jugador.getVelocidad());
                    fichero.write(jugador.getResistencia());
                    fichero.write(jugador.getAgresividad());
                    fichero.write(jugador.getCalidad());
    
                    fichero.write(jugador.getRemate());
                    fichero.write(jugador.getRegate());
                    fichero.write(jugador.getPase());
                    fichero.write(jugador.getTiro());
                    fichero.write(jugador.getEntradas());
                    fichero.write(jugador.getPortero());
    
                    fichero.write(jugador.getPie());
                    fichero.write(jugador.getPenalti());
                    fichero.write(jugador.getCornerIzquierdo());
                    fichero.write(jugador.getCornerDerecho());
                    fichero.write(jugador.getFaltaIzquierda());
                    fichero.write(jugador.getFaltaDerecha());
    
                }
    
                fichero.close();
                File fich1 = new File(hashOpciones.get("jugadores").toString());
                File fich2 = new File(hashOpciones.get("jugadores").toString()+".temp");
    
                fich1.delete();
                fich2.renameTo(fich1);
    
    
            } catch (IOException ex) {
                Logger.getLogger(cJugador.class.getName()).log(Level.SEVERE, null, ex);
            }
    
            return res;
        }
    
        public void grabarCadena(String cadena, int largo, RandomAccessFile fichero){
    
            int[] grabar = convCadenaPCFAscii(cadena);
    
            for(int i = 0;i<=largo-1;i++){
                try {
                    fichero.write(grabar[i]);
                } catch (IOException ex) {
                    Logger.getLogger(cFunciones.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        }

  2. #2
    xcallmejudasx's Avatar
    xcallmejudasx is offline Senior Member
    Join Date
    Oct 2008
    Location
    Houston, TX & Flint, MI
    Posts
    609
    Rep Power
    7

    Default

    This is a little past my scope but the only thing I can think of would be adding an extra thread or two to speed up the writing.

    I'm assuming you've looked into the different types of trees(AVL, Red Black, etc) and used the fastest one for your needs?
    Liberty has never come from the government.
    Liberty has always come from the subjects of government.
    The history of liberty is the history of resistance.
    The history of liberty is a history of the limitation of governmental power, not the increase of it.

  3. #3
    Krons is offline Member
    Join Date
    Apr 2009
    Posts
    2
    Rep Power
    0

    Default

    yes, i need to use this kind of tree...

    I don't work before with threads, so i'm gonna need to learn it xD.

    Using DataOutputStrem will improve the speed?

  4. #4
    xcallmejudasx's Avatar
    xcallmejudasx is offline Senior Member
    Join Date
    Oct 2008
    Location
    Houston, TX & Flint, MI
    Posts
    609
    Rep Power
    7

    Default

    I honestly have no clue. I've never worked with threads or have a familiarity with the DataOutputStream but it seems to make sense that a stream designed for writing data would work faster then a bunch of different write methods. Perhaps one of the Gurus could hop in and give you more useful advice.
    Liberty has never come from the government.
    Liberty has always come from the subjects of government.
    The history of liberty is the history of resistance.
    The history of liberty is a history of the limitation of governmental power, not the increase of it.

  5. #5
    markw8500's Avatar
    markw8500 is offline Senior Member
    Join Date
    Jul 2008
    Location
    Pennsylvania, USA
    Posts
    136
    Rep Power
    0

    Default

    I just started to work with Threads and they might not work out for you... The reason being that from your code it looks like all the data that you are writing to a file needs to be in order and if you mix Threads in with that, then each Thread will be writing to the file different information at different times... So you will probably be overwriting you data and mixing it all up...
    Who Cares... As Long As It Works...

  6. #6
    OrangeDog's Avatar
    OrangeDog is offline Senior Member
    Join Date
    Jan 2009
    Location
    Cambridge, UK
    Posts
    838
    Rep Power
    6

    Default

    A few things:

    Don't use a RandomAccessFile, opening one for output only will be quicker.
    Use generics for your collections - so a Hash<Jugador> - to avoid casts.
    Using an enhanced for loop - for (Jugador j : Jugadores) - will give most efficient iterator for the system.
    Restructure the Jugador class so there is, for example, a single write(java.io.OutputStream) method that writes directly from the fields. This will avoid all the repeated method calls for every field.

    These should speed up the program to varying extents. If it's still not fast enough try running it through a profiler and seeing where the bottlenecks are. Typically, using the simplest objects (a raw OutputStream) will provide faster methods, but at an ease-of-use cost.
    Don't forget to mark threads as [SOLVED] and give reps to helpful posts.
    How To Ask Questions The Smart Way

  7. #7
    neilcoffey is offline Senior Member
    Join Date
    Nov 2008
    Posts
    286
    Rep Power
    7

    Default

    The main thing that determines the speed of I/O is the number of operating system calls you need to make per byte written. The classes BufferedInputStream and BufferedOutputStream (and other cousins such as BufferedReader) aim to minimise the number of I/O calls by storing your data into a buffer, and only actually writing it block at a time. At around 8K/call, you reach a point where there's not much benefit in buffering any more.

    The problem with RandomAccessFile is that it doesn't do any buffering. Every time you make a read/write call on RAF, you are making an operating system call. So for example, your grabarCadena() method is making a call for every character written! This is clearly bad.

    The solution is to change your code to write via a buffered output stream.

    For more information on buffering, see my article on How big should my input stream buffer be? (I believe the figures are essentially similar for output).

    The absolute fastest way to write the data is probably to use a mapped ByteBuffer with the NIO package, but this introduces some other problems, so probably not recommended.

  8. #8
    OrangeDog's Avatar
    OrangeDog is offline Senior Member
    Join Date
    Jan 2009
    Location
    Cambridge, UK
    Posts
    838
    Rep Power
    6

    Default

    A rough order for how much different things slow the program down, slowest first:

    Native method calls
    Disk operations
    Other operating system calls
    Java method calls
    Memory operations
    Java primitive operations (thread-related stuff is slowest)

    Obviously, doing quick stuff repeatedly is going to be slower than doing slow stuff once.
    Don't forget to mark threads as [SOLVED] and give reps to helpful posts.
    How To Ask Questions The Smart Way

  9. #9
    neilcoffey is offline Senior Member
    Join Date
    Nov 2008
    Posts
    286
    Rep Power
    7

    Default

    OK, bear in mind with the last three:
    - the JIT compiler tries to take steps to make sure that method calls per se have close to zero cost when it matters (e.g. if you sit in a loop calling a short-ish method, then the method gets inlined)
    - not sure exactly what you mean by "Java primitive operations", but if you mean simply operations on a local variable, then that's probably true, except that the boundary between when you think Java is operating on a "primitive" and on something else isn't necessarily where you think it is. So you might logically expect to be writing to a register/local variable instead of to "main memory", but that ends up not being the case, and vice versa.

  10. #10
    OrangeDog's Avatar
    OrangeDog is offline Senior Member
    Join Date
    Jan 2009
    Location
    Cambridge, UK
    Posts
    838
    Rep Power
    6

    Default

    -If you sit in a loop making lots of calls including recursion there's not much JIT can do for you. In general, assume maximal uselessness from a compiler and that every method call is going to involve a new stack frame (I know for a fact that Sun's javac doesn't optimise tail recursion)
    -Primitive operations are those that have equivalent ops in Java Bytecode. While results differ depending on platform and implementation, you can generally assume that they will be pretty quick, though those that don't compile simply to base machine code may take longer (like object and thread management stuff)

    Anyway, this is all rather advanced. Just cut out all those method calls and use a buffered stream should get all the speed-up you need.
    Don't forget to mark threads as [SOLVED] and give reps to helpful posts.
    How To Ask Questions The Smart Way

  11. #11
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    Quote Originally Posted by OrangeDog View Post
    Don't use a RandomAccessFile, opening one for output only will be quicker.
    Yep...that'll help. Even more importantly, use a BufferedOutputStream. You want to avoid the transaction costs of the write method.

    Substituting the following for the RandomAccessFile will probably make the other optimization choices moot:

    Java Code:
    BufferedOutputStream fichero = new BufferedOutputStream(new FileOutputStream(hashOpciones.get("jugadores").toString()+".temp"));
    (there is no writeShort method for an OutputStream, but it looks like maybe you're starting with bytes anyway?)

Similar Threads

  1. Test internet speed
    By kalpo in forum Networking
    Replies: 0
    Last Post: 03-31-2009, 01:20 AM
  2. how to improve my security?
    By anthrax in forum JavaServer Pages (JSP) and JSTL
    Replies: 1
    Last Post: 03-13-2009, 10:08 AM
  3. How to speed sql Statements?
    By bezudar in forum Advanced Java
    Replies: 3
    Last Post: 11-20-2008, 10:53 AM
  4. how to improve the performance of JWS?
    By dinesh kaushik in forum Java Applets
    Replies: 0
    Last Post: 11-21-2007, 09:46 AM
  5. compare speed
    By bbq in forum JDBC
    Replies: 1
    Last Post: 06-28-2007, 06:34 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •