fernando.federico on Monday, January 30, 2012

Adding Flows Dynamically

27

Mule has a new really cool module ready to be used in Muleforge called Dynamic Flows. The idea behind this new module might sound strange at first but it is actually quite simple: it allows you to add new flows to your existing mule application without restarting the server. It provides three simple operations for flow management:

1) Add

Creates a new mule context with the name and configuration provided. It allows adding to your application all the flows defined in the config XMLs passed to it. For example, the addFlow:

Receives a context name and a list of mule configs XMLs (the XMLs, not the files) through a http post, then the transformer creates a map grouping all the XMLs under the same key and finally the module processes those XMLs and creates a new context in your mule server. After the flow finishes, we have new flows ready to be used.

2) Run

It executes a dynamic flow, for example, consider the following flow:

If we write: http://localhost:10443/run?contextName=testContext&flowName=testVmFlow on our web browser, it executes the dynamic flow called “testVmFlow” in the dynamic context “testContext” added by the module.

There are two ways you can call a flow: using a VM transport or without it. If you choose the first option, remember to declare the vm inbound in your flows.

3) Remove

Removes a entire dynamic context from your mule application.

A demo application

This module is quite useful when you need to execute logic that depends on dynamic information, for example, let’s consider the following requirement:

Our users have several campaigns in Salesforce, all of them have a custom field called ‘info’. The information stored in this custom field depends on the campaign. For example, a user A has two campaigns: ‘Phone marketing’ and ‘Email sales’, for the first one,  the field ‘info’  must contain all the user’s available twilio phone numbers  separated by comas, but for the ‘Email sales’ campaign it must contain the email template stored in constant contact. 

We need to create a mule application that sets the ‘info’ field for our users.

As we can see, this problem is kind of tricky, because we don’t know which process we need to execute before hand, it depends on the salesforce campaign so what do we code in mule?, do we write an if condition in our mule context?, what happen if the user creates a third campaign?. Doing this application with the static mule configuration is hard, however we can simplify it a lot by using Dynamic Flows module. We just open MuleStudio, create the following flows:

And we are done, every time our users create a new campaign, they can tell mule how to get the data that has to stored into the ‘info’ field,  without stopping and starting the mule server.

Links and further reading

For further information, see Dynamic Flows documentation.

To see the code check the Dynamic Flows source code, you can find a demo app to see its usage.

No related posts.

Interested in 3 days of knowledge sharing, hands-on labs, industry focused sessions, and plenty of networking? Register for the premier integration event, CONNECT London »

27 Responses to “Adding Flows Dynamically”

Mohamed March 28th, 2012, 1:13 am

Thanks, that’s very similar to something I was trying to do. But I didn’t think of it as plugin. How did you get that to work in MuleStudio?

fernando.federico March 28th, 2012, 5:19 am

Thanks for your interest, this is a project though for advance users of mule, this is not integrated to studio, I’ll comment this to the Studio product management team.

However ;-) , there is always a work around….

In your studio project create a “plugins” folder under main/app, and drop the module zip file in there..

You can get the zip file from the public repository…
https://repository.mulesoft.org/nexus/content/repositories/releases/org/mule/module/dynamicFlows/dynamic-flows/1.0/

This will make studio include the module when it starts mule, but you will need to modify the XML by hand (you won’t have a UI).

that should be all, you might need to add the jar file in you build path if you don’t want to see any errors in studio while developing….

Mohamed March 29th, 2012, 1:25 am

Thanks Fernando, I was just curios how did you manage to add a plugin to MuleStudio since it comes packaged with embedded Mule runtime. Thanks for answering my question.

I think this plugin has a great potential, I would think it should be added to Mule standard distribution. I have a use case in which this plugin might address, still investigating but I managed to get it to work after fixing some Maven issues on my end.

How much have you test this? is there any known issues with it?

Regards,
Mohamed

fernando.federico March 29th, 2012, 4:51 am

So far there is no known issue with it, I’ve run some jmeter tests and it seems to behave correctly. It would be really nice to see it running on a production environment!, let me know if you need any help to get your code running, I’ll be following the blog and the forums to give you support if needed.

Thanks!

Mohamed March 29th, 2012, 8:19 am

Thanks Fernando, this may be a great help for us. I will keep you posted.

Mohamed April 13th, 2012, 8:40 am

Hi Fernando,

I have a question.

Looking at the code, how does remove actually works?

I can see it removes the context reference from the contexts map but remove processor also fires onNotification? in that case how MuleContextNotification.CONTEXT_STOPPED is mapped to remove action?

Regards,
Mohamed

fernando.federico April 13th, 2012, 8:45 am

Hello, mohamed, It stops the context, dispose it and then remove it. (check the last version of the code in github, there was an unpushed file…)

The notification process is done when you kill mule from console. If you kill mule all your dynamic flows will be removed….

Mohamed April 13th, 2012, 9:46 am

Thanks a lot Fernando, I see that you updated the remove method to stop the context as well.

Mohamed April 17th, 2012, 8:58 am

Hi Fernando,

Another issue just showed up last week, and I am not sure if it’s a mulesoft issue or maybe you need to publish the schemas again because they were removed from musesoft.org

http://www.mulesoft.org/schema/mule/dynamicflows
http://www.mulesoft.org/schema/mule/dynamicflows/1.0/mule-dynamicflows.xsd

I understand that they should be scanned in the Jar but doesn’t seem to be the case, I also modified the schema to use explicit META-INF/mule-dynamicflows.xsd.

There seems to be either a problem with the updated version of devkit annotations 4.2 or something else has to do with removing the schemas.

Do you think you need to publish the schemas again?

Regards,
Mohamed

fernando.federico April 17th, 2012, 9:51 am

Hello mohamed, could you attach the error you are getting? the schema should be read from the jar and it is there. I’ll verify if there is a bug related, but I don’t think so

Thanks

Fernando

Mohamed April 17th, 2012, 10:00 am

Yes, the schema is there. I just suspect that it’s Mule devkit related bug which is unable to properly scan the .xsd in the JAR. I can assert that I started to get this problem after the online schema was removed.

ERROR 2012-04-17 15:57:39,937 [WrapperListener_start_runner] org.mule.module.launcher.application.DefaultMuleApplication: null
org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element ‘dynamicflows:configs’. One of ‘{“http://www.mulesoft.org/schema/mule/core”:annotations}’ is expected.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)

fernando.federico April 17th, 2012, 10:44 am

please check the following things

the jar is in your classpath
if you are using maven. do you have the pom.xml as shown in the module documentation?
is the jar included in the Zip file?

One thing I find strange of the error you show is that it says dynamicflows:configs while it should be dynamicflows:config (without the last ‘s’)

Fernando

Mohamed April 18th, 2012, 1:22 am

Thanks Fernando,

There are two:

One at the top of the XML

The other is for

It seemed like a JAR issue, I cleaned my Maven repository and seems to be working now.

Regards,
Mohamed

fernando.federico April 18th, 2012, 5:01 am

Great!, I happy to hear it is working now!

RM June 15th, 2012, 2:06 pm

I am currently using Delete + Add to modify flow on the fly. So m question here is, what will happen to the message currently under processing when I delete a flow. Is there any flow state management? I mean like BPMs is there any flow state persistence mechanism in place

fernando.federico June 18th, 2012, 6:53 am

No, there isn’t. Your message will not be processed completely… It could be an improvement though

RM June 18th, 2012, 1:35 pm

Thanks fernando for a quick reply. I have one more question, Is there a way to call the flows in main context from the dynamic flows? In other way can the dynamic flows call the flows from a differt context or mail context?

RM June 18th, 2012, 2:08 pm

Ignore my previous question. It is pretty straight forward

fernando.federico June 18th, 2012, 5:31 pm

Yep, you can use it as if they are in your same context. This is the great benefit of father child relationship (you can use everything declared in the parent)

RM June 26th, 2012, 1:30 pm

Is there a way to share the custom expression evaluators. Expression manager from the mule context do not contain the custom expression evaluators registered on the parent mule context.

fernando.federico June 27th, 2012, 6:25 am

Each context has its own ExpressionManager, so it is not possible by default. But there is always a hack, in your child configuration create a MuleContextAware component that gets the ExpressionManager and registers the expression evaluator that you have in your parent (that is also in the context). It is kind of advance hack, fun to do…

mmwhita September 27th, 2012, 6:02 am

Trying to install this and keep getting following error:
ERROR 2012-09-27 08:57:09,076 [main] org.mule.module.launcher.application.DefaultMuleApplication: null
org.xml.sax.SAXParseException; systemId: http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd; lineNumber: 475; columnNumber: 114; src-resolve: Cannot resolve the name ‘mule:nameValueType’ to a(n) ‘type definition’ component.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)

I’m not using maven, I do have the jar file (which i downloaded from nexus repository) on build path, I put the zip file in the main/app/plugins folder and am using Mule Server 3.3.

Any ideas on this would be greatly appreciated!

David Dossot October 1st, 2012, 2:44 pm

Where do you get this error? In Studio or when starting Mule Standalone?

JM January 20th, 2013, 1:52 am

Hello,
I’ve tested the dynamic flow and it’s working fine in Mule or MuleStudio. Thanks for the great work!
I have a question : Where should I put the plugins folder in an application deployed in an application server (Tomcat) ?

fernando.federico January 23rd, 2013, 11:24 am

you don’t need to add the plugin folder, just include this in your pom


<dependency>
    <groupid>org.mule.module.dynamicFlows</groupid>
    <artifactid>dynamic-flows</artifactid>
    <version>1.0</version>
</dependency>

you need the jar, not the plugin

JM January 25th, 2013, 1:53 pm

This is working fine. Thx Fernando!

fernando.federico January 26th, 2013, 5:38 pm

Glad to hear! Thanks for trying this out!

Leave a Comment