Results 1 to 4 of 4
  1. #1
    jerry_nf is offline Member
    Join Date
    Mar 2011
    Posts
    2
    Rep Power
    0

    Default to log the very fast data source

    HI, all,

    I am writing a client logging program to log the incoming UDP data frames.

    It is very fast, 1000 frames/second coming over.


    I am using the following mechanism to read the UDP data:

    But I find it seems this mechanism cannot catch up the speed of the data source sometime, there are always data frames missing. And, the more data frames I logged, the more possible frames getting lost. I can see the gap of the sequence numbers between 2 frames from the output error messages.

    When I use the same software to log the 1 frame/second data source, it is fine.

    Any suggesting to improve this ?

    Thanks,
    Jerry

    //$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $$$$$$$$
    public class Data_Listener extends UDP_Listener {

    private static long myseq = 0L;

    public ProcessData_Listener(UDP_Client client)throws IOException {

    super(client);
    }

    @Override
    public void run() {
    // TODO Auto-generated method stub
    while( true){


    Read_Data();

    }//end of while

    }


    private void Read_Data() {


    data = new byte[DATA_LEN];
    for (int i =0; i<data.length; i++)
    data[i] = 0;

    try {
    // read data from message
    data = udpClient.readMsg(data, DATA_LEN);

    // read sequence number from data frame
    long seq = readSeq(data);

    // check any data frames missing
    if (seq != myseq + 1 && myseq != 0) {
    System.out.println("ERROR: -- data received, seq: " + seq);
    System.out.println("ERROR: -- previous seq: " + myseq);
    }

    myseq = seq;



    // logging process
    .....


    } catch (IOException e) {

    e.printStackTrace();
    }

    }

    }

  2. #2
    mrmatt1111's Avatar
    mrmatt1111 is offline Senior Member
    Join Date
    Aug 2009
    Location
    San Jose, CA, USA
    Posts
    320
    Rep Power
    6

    Default

    "Lacking reliability, UDP applications must generally be willing to accept some loss, errors or duplication."

    User Datagram Protocol - Wikipedia, the free encyclopedia
    My Hobby Project: LegacyClone

  3. #3
    jerry_nf is offline Member
    Join Date
    Mar 2011
    Posts
    2
    Rep Power
    0

    Default

    yes, you are right. UDP is connectionless protocol.

    But, in this case, I used WireShark to catch the incoming data frames. No one missing.

    So, I think the problem is in my java client.

  4. #4
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    Anything you insert in your processing loop that slows it down, including logging, is going to cause you to fall behind. There's a sort of uncertainty principle at play here as a result.

    If you want to minimize lost packets, take everything out of the read loop except the read itself:

    - pre-allocate a cache of byte arrays big enough to read the packets and use those rather than creating them new

    - once you've read the byte array, put it into a queue to be processed in one or more other threads to readSeq and log, rather than doing those steps in the same thread

Similar Threads

  1. Fast Data Transfer 0.9.1
    By Java Tip in forum Java Software
    Replies: 0
    Last Post: 04-24-2008, 07:41 PM
  2. Fast Data Transfer 0.8.5
    By JavaBean in forum Java Software
    Replies: 0
    Last Post: 11-12-2007, 07:11 PM
  3. Fast Data Transfer 0.6.7
    By Jamie in forum Java Software
    Replies: 0
    Last Post: 06-14-2007, 03:55 PM
  4. Fast Data Transfer 0.6.4
    By levent in forum Java Software
    Replies: 0
    Last Post: 05-21-2007, 11:11 AM
  5. Fast Data Transfer 0.6.4
    By levent in forum Java Software
    Replies: 0
    Last Post: 05-20-2007, 09:55 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •