Java pipe streams and concurrency

The Java pipe stream is the old terminology from the Java 1. Pipes in Java allows two threads running in the same JVM can communicate, one is the source and another is the destination. In MyCollab product, the end users can export the data to the downloaded file with the formats are PDF, Excel or CSV. The challenge is the process of exporting may be long because we do not limit the number of records to export, and the system may execute some complex business rules which consume time to finish. We do not wait for the process to export file to the server and then the user will download the whole file to their site. The pipes solution solve this case as what we expected.

How pipes to improve the respond time of exporting files

We created two pipes on the two different threads. One for output stream writes the bytes data from exporting process, and another pipe writes the fileĀ  to the download stream.

By theory, the result looks good but our initial version can not be stable enough for the production, sometimes it works well and sometimes it throws the EOL exception. The reason is the not thread-safe in read-write of pipes in two different threads. The output stream may not write data when the input stream request to read data, and the thread of input stream is stopped. The class CountDownLatch of java concurrent package can help us to resolve this issue.

CountDownLatch allows one or more threads to wait until a set of operations being performed in other threads completes, we create an instance CountDownLatch and set the value is 1. Only when the input stream is closed, the CountDownLatch will set the value to 0. Otherwise, the Thread that runs the output stream will never stop to wait for the output from the exporting process.

Above implementation helps our export process can run like a charm, the download file to the client server runs parallel with the exporting data from the server. The memory is not high while we do not keep all data to memory but a partial is exporting then it is moved to the downloading process, and client can see the downloading progress right after they export the data.

Leave a Reply

Your email address will not be published. Required fields are marked *