Best practices in handling big data streams
I need to develop a mechanism to receive the content of a big file through the input stream of a Servlet, maybe transform the data, and then sned this to an ending stream (like a file, another URL, ...).
I know how to read from the servlet stream. But suppose you have to handle thousands of megabytes of data. What would be the best method to handle this data? I can't load all the data, because memory will run out. I think I could store data into a temporary file, and then handle it later, but would it be a good practice? Remember that i'm working with servlets, so there is a Tomcat present, and in my local workspace I do what I want in my filesystem. Not the same about customer's workspace (I'm very limited about filesystem privileges).