Introduction

MuleSoft’s iON is a cloud-based integration Platform as a Service (iPaaS) that isn’t  just for “cloud-to-premise” scenarios, but for “cloud-to-cloud” integration as well. With the current API explosion, a vast array of services is emerging which can be brought together in unimagined ways, and iON is there to enable this—like lightening between the clouds!

In this post, I’d like to demonstrate how to integrate two such cloud services. The first is Google’s well-known Gmail and the other is PivotalTracker, Pivotal Labs’ cloud solution for agile software development and project management. Developers, analysts and testers can work together by posting, editing and moving “stories” from the initial phase of gathering requirements to final delivery. Our friends at Pivotal Labs have exposed the core functionality to us as a REST API. Our solution will exploit the Cloud Connector DevKit and Mule Studio to deploy an iApp, an integration Application, to iON. Creating and commenting on stories posted to will be as easy as sending an email!

Use Case

Emails arriving in a developer’s mailbox are considered to be requests for features in the development of new software.

  1. If the email is new, they should be placed on Pivotal Tracker as new “feature” type stories with the subject of the email as the name of the story and the body of the email as description of the story. For future correlation, the story should be labeled with the Message-ID header of the email.
  2. If the email is a reply to the thread created by #1, do the following:
  • If there is a story on Pivotal Tracker labeled with the first value of the References header on this email, then append the body of the email as a comment to that story.
  • Otherwise, create a new story as in #1.

Overview of Solution

A Mule Flow which comprises:

  • Pop3 (or IMAP) connector for consuming emails (comes with Studio)
  • XSLT Transformer to create XML requests for PivotalTracker (comes with Studio)
  • PivotalTracker cloud connector (we’ll create that!)

Pivotal Tracker Cloud Connector

According to DevKit instructions

For the initial question, just accept the default value.

Change the directory to pivotal-tracker-connector and run mvn eclipse:eclipse to convert it to an Eclipse project. Once in Eclipse, import it as an existing project into your workspace and then proceed to convert it to a Maven project (right-click Config -> Convert to Maven Project).

Take a look at the class generated at com.mulesoft.se.PivotalTrackerConnector

@Configurable

We need one such annotated method for every property which we need to configure from a flow using the config element.

@Connect

We won’t exploit this, so just return true in the isConnected method:

@Processor

We need one for every operation we wish to expose in the flow. @param: It is important to list all parameters passed to processor methods in the javadocs, for proper generation of docs. Otherwise the Maven build will fail.

Now, let’s spice up our connector so that it caters to the use case highlighted above! We need 4 methods:

  1. create-story, which should expect the story payload.
  2. label-story, which we need to correlate stories with subsequent emails in the thread.
  3. find-story-by-filter for searching
  4. comment-story, to add subsequent emails in the thread as comments to the original story.

Before we start coding, let’s add the following dependencies to our pom, as we will be using jersey as our REST-ful webservice client and logback for logging:

  • org.slf4j:slf4j-api:1.6.4
  • org.slf4j:jcl-over-slf4j:1.6.4
  • ch.qos.logback:logback-classic:1.0.0
  • com.sun.jersey:jersey-client:1.12

Once that’s done, we need to update doc/PivotalTracker-connector.xml.sample to facilitate doc generation:

Building the Connector for Studio

Now, while the dev-team is busy enhancing Studio to cater to our use case, we are going to do a little under-the-bonnet work and deploy our connector to Studio so that it is recognizable in our flow.

Execute: mvn install -Ddevkit.studio.package.skip=false

This will generate /pivotal-tracker-connector/target/pivotal-tracker-connector-1.0-studio.zip

This zip file contains both the binary jar for use as a dependency and icon information so that Studio can include the connector in its pallete. Unzip this file to $studioHome/dropins and restart Studio. You should see the icon for PivotalConnector in the pallete under cloud connectors.

Creating the iApp

So what is an iApp?  Its an integration App that creates a composition of other applications and services to create additional value. In this case, it is a flow that integrates two or more public APIs and is deployed to Mule iON. Our app will cater fully to the use case described above by consuming emails coming into our Gmail account, transforming them into XML payloads for Pivotal Tracker and then routing them to Pivotal Tracker through our cloud connector.

Before the flow element, we need to configure our cloud connector:

And here’s the flow

Email arrives on the Inbound Pop3 Endpoint. We immediately save its From header to Session scope for use in the other flows. The Choice Routing Message Processor determines if Message-ID is present in the header to determine whether the email is new or a reply to the thread.

If the email is new, processing is delegated to the newEmail Sub-flow. Immediately we wrap the body of the email with XML markup in order to make it suitable for the XSLT Transformer. The latter prepares the XML request for Pivotal Tracker before dispatching the message to the in-memory queue, VM Endpoint.

New Emails

A separate flow, createStory, consumes the transformed message on vm://createStory and immediately posts the message to Pivotal Tracker using our new cloud connector. We save the Story Id returned by Pivotal Tracker for use in other flows.

Immediately we prepare a new request to Pivotal Tracker which will label our new story with the Message-ID header of the email. We will use this later on as a corelation mechanism.

Again in a separate flow (we do this to facilitate  finer-grained exception handling, which I talk about below), we consume the previously dispatched message at vm://labelStory, invoke the Pivotal Tracker cloud connector and log the successful invocation.

 

Reply Emails

When our main flow determines that the incoming email is a reply on the thread, then it invokes the replyEmail sub-flow, which saves the first value of the References  header to Session and subsequently dispatches the message to vm://commentStory.

The commentStory flow picks this up by listening to vm://commentStory. It invokes the search operation on our PivotalTracker cloud connector.

If a matching story is found on Pivotal Tracker we proceed through our Choice Routing Message Processor to add the body of the email as a comment to the story again using the cloud connector after having suitably transformed the message using the XSLT Transformer.

Exception Handling

We chose to break up our solution into separate flows in order to catch potential exceptions in the invocation of our Pivotal Tracker cloud connector. (We can’t assume the availability of the Pivotal Tracker API). Our strategy is simple. Keep retrying every five minutes up to a maximum of twelve retries. A subsequent failure is caught with our default exeption strategy. We then create a delay of 1 hour using the Quartz Endpoint before dispatching the message as it was before the exception to our Active MQ error queue. As we have a composite Message Source for each of these flows, we can consume these failed messages using the same flow which participates in the happy path.

Deployment to the Cloud!

This couldn’t be easier! Right-click on the Project in the Package Explorer and select Mule iON -> Deploy to iON.

 

 

 

 

Enter your username, password, domain name and it’s deployed!

 

 

 

 

That’s it!

You now have a new cloud connector already exploited in an iApp running in the cloud. You can see the full Mule configuration here and the source code of the cloud connector here.

No related posts.


7 Responses to “iPaaS: Cloud-to-Cloud Integration of Gmail and Pivotal Tracker”

Ryan April 25th, 2012, 3:17 pm

I tried dropping the zip file into the dropins directory and it didn’t work. Ideas?

Nial Darbey April 25th, 2012, 3:45 pm

Hi Ryan,
did you unzip the file in the dropins directory. Studio won’t unzip it for you. You have to do it manually. (Studio will do all of this for us soon!)
Nial

Niraj Trip October 12th, 2012, 1:56 pm

When I tried to download the configurations using the URL mapped to last line of this article – and not working. It brings the page – NOT Found.

How can I download the configurations using below URLs?
http://blogs.mulesoft.org/pivotal-tracker-connector/
http://blogs.mulesoft.org/pivotal-tracker-iapp-flow/

Nial Darbey October 12th, 2012, 3:40 pm

Hi Niraj,
yes, I´ve confirmed these links are not working. I´ve contacted admin so it should be available next week. Apologies for the inconvenience.
Nial

Nial Darbey October 13th, 2012, 3:41 am

Hi Niraj,
I’ve changed those links to go directly to their gists.
Thanks,
Nial

Yahoo Mail July 31st, 2013, 3:21 am

I like the valuable info you provide to your articles. I’ll bookmark your blog and test again here regularly. I’m relatively sure I
will learn many new stuff right right here!
Best of luck for the following!

Tova Binam November 16th, 2013, 10:31 pm

I do like the way you have framed this issue plus it really does give me personally some fodder for thought. On the other hand, coming from what precisely I have seen, I simply trust when other feedback pack on that individuals continue to be on point and don’t embark upon a soap box of the news of the day. Yet, thank you for this outstanding point and while I can not necessarily concur with it in totality, I value your standpoint.

Leave a Comment