Jackie on Thursday, May 28, 2009

Streaming in Mule

4

Streaming enables efficient processing of large data objects such as files, documents, and records by streaming the data through Mule rather than reading the whole thing into memory. Streaming provides the following advantages:

  • Allows services to consume very large messages in an efficient way
  • Message payloads are not read into memory
  • Simple routing rules based on message metadata are still possible
  • You can combine streaming and non-streaming endpoints

Streaming Transports

The following Mule transports support streaming:

Streaming Transformers and Filters

Many Mule transformers and filters can read input streams, process the contents, and send them on. However, most of these do not process the stream in real time; instead, they read the stream, load it into memory, process it, and then send it on. Therefore, transformers and filters can become a bottleneck in your application if you regularly stream large files.

The following transformers and filters do support true streaming and process the data as streams without loading them into memory first:

No related posts.

Interested in 3 days of knowledge sharing, hands-on labs, industry focused sessions, and plenty of networking? Register for the premier integration event, CONNECT London »

4 Responses to “Streaming in Mule”

Sebastien May 30th, 2009, 2:32 am

- XSLT Transformer

Chained xsl transformers are not streamed at all in Mule.

Alessio May 30th, 2009, 1:59 pm

Hi Jackie
What if the stream is dispatched to a tapper ? I mean for example, a CXF inbound endpoint and a wire tap on JMS. For my experiences if the stream is consumed it won’t be more available..
In case of multicast outbound ?

Ross Mason June 1st, 2009, 12:07 am

Hi Alessio,
Good question :) . In the scenario you describe the message will be read into memory and a copy will be sent to JMS and one to the Web Service. Obviously, this defeats the purpose of streaming, but for SOAP messages it is less likely that the SOAP envelope will be huge. Note that JMS does not support streaming in that you can’t stream a message directly to the JMS server.

Rule of thumb, if you are processing very large messages in Mule, you need to use point to point routing so that the stream flow can be preserved. You can still to conditional routing but make sure that the current message is routed to a single location.

Introducing Mule Enterprise Security November 6th, 2012, 11:12 pm

[...] these existing security features while leveraging the benefits associated with Mule flows such as support for streaming and the Mule Expression Language. With that in mind, let’s take a tour of the new features that [...]

Leave a Comment