Ross Mason on Thursday, February 10, 2011

Groundhog Day: Cloud Silos

6

There is no denying it, the cloud is having a massive impact on all our lives. In the enterprise many of the applications that we to host in our own data centers now live in the cloud.  We are a fairly typical company in terms of our IT, we don’t host any business software in house. We use Google Apps, Salesforce, Intaact, Marketo, and more. We do it because it is convenient, quickly and usually painless.  This instant app-gratification that SaaS and cloud provides is inviting us to make the same mistakes again; we’re putting our data in silos.

Full circle

The reason I can be an integration nerd is thanks to the 20 years of enterprises introducing applications that didn’t talk to each other followed by an onslaught of ill-conceived point to point integrations followed by a more strategic and pragmatic approach to integration, such as ESB and bottoms-up SOA.  Now with our data increasingly peppered over many different SaaS providers we are falling into the same traps that ate enterprise IT budgets over the last two decades.  It is estimated that enterprise spend three times more on integrating their applications compare to the cost of acquiring the applications. We need to break this cycle in the cloud.

Same Problem, new dynamics

While the cloud silos problem feels familiar to anyone who has worked on an integration project, the tenets of the problem has changed.

  • We’re almost exclusively talking over HTTP using SOAP or REST, this makes the connectivity problem easer to manage but also restricts us in how we can architecture a solution (i.e. stateless, no widely accepted way of doing transactions, etc)
  • There are many more applications to integrate with now.  The SaaS/ movement has created new classes of applications from infrastructure such as AWS to office productivity such as Google Apps, to vertical solutions such as Marketo for marketing automation. Expect this growth of new SaaS offerings to accelerate over the next couple of years.
  • These applications are more fine grained than before, my CRM vendor is different from my ERP vendor, who is different from my financials, who is different from my marketing automation vendor. Before I could buy into an application stack from SAP or PeopleSoft and get everything from one vendor.  With everything moving to the cloud, there will be more dispersion of applications whether we like it or not.
  • Acquisition of a new SaaS application is dangerously easy compared to the traditional way of procuring an application.  Previously, applications were selected with a view that that application would need to be managed and deployed in house, now a decision can be made with a credit card swipe.  This promotes the use of more a more diverse set of application in small and large companies exacerbating the need for integration
  • These new SaaS and cloud applications present a wrath of new challenges around security, identity management, repudiation, visibility, control, due to the fact that each application is created by a different vendor with varying levels of focus on these issues.

Breaking the cycle

There are already a number of integration solution that often focus on getting data to and from silos like Salesforce.  These offerings are usually delivered as a black box, point-to-point integration (i.e. CastIron). Point-to-point integration seems fine at first but we’ve learned in the enterprise that doing point-to-point doesn’t scale well, due to the burden of ongoing maintenance of multiple integration points and adaptation as your business needs change. Also, the black box nature of these solutions means that that you have limited capabilities to manage, debug, control and monitor this spaghetti integration.

Its not just about SaaS and cloud services either,  the SOA-ification of web applications means that the way new applications are being built is through composition of services, augmenting existing functionality within our own apps and leveraging the treasure trove of infrastructure and data services available on the web.  This shift means that point-to-point integration just doesn’t cut it, instead we need to orchestrate data across these different services and need to be able to quickly plug in new functionality without disrupting existing integrations. We need integration Platform as a Service.

Integration Platform as a Service

iPaaS is a cloud-based integration platform that offers capabilities and services for a broad range of integrations scenarios from simple point-to-point integrations to full SOA at cloud-scale. An iPaaS is event-driven, processing operational data (i.e. new customer was just added) firing events at realtime or at least near-realtime. Orchestration of services and applications is critical with the ability to integrate with on-premise, cloud and custom applications using well defined integration patterns and connectors. Management, control and visibility are crucial as is the ability to manage cloud and on premise infrastructure together.  Of course it is almost impossible to mention cloud without talking about security.   needs security beyond traditional platforms to provide fine-grained access control, integration with on premise user directories and support for new security protocols such as OAuth.

From my point of view another important difference is that iPaaS is targeted at IT and developers.  This is a platform after all and while the platform should simplify how integrations are built, it should be possible to drop into code when needed or start with code depending on your preference.

I believe there is massive scope for value-added services on top of iPaaS but the points above define the key characteristics.

What about aPaaS?

Much of the interest around PaaS at the moment has been around application PaaS or aPaaS; platforms for building applications.  Prominent players here include Heroku (recently acquired by Salesforce), Google App Engine and Joyent. These platforms offer ways to deploy complete applications in various languages.  The iPaaS is focused integration orchestrating data between different applications and optional exposing these integrations as services.  The iPaaS can be used to intergrate with or provide integration services to aPaaS.

Introducing Mule iON

Mule iON iPaaS

Mule iON is the first iPaaS offering to address these needs of this cloud integration landscape.  It is a fully cloud-based solution, enabling customers to take advantage of the economics and elasticity of the cloud for their integration infrastructure.   While existing integration services have focused on tactical point-to-point integration, Mule iON is built on the leading Mule technology at its core, enabling scalable and flexible integration scenarios, from point-to-point to full-scale SOA. With Mule iON, developers can be up and running in hours, while ensuring that they have the scalability, flexibility and manageability that they have come to expect from Mule.

With Mule iON, users can leverage a library of out-of-the-box cloud connectors to integrate with popular SaaS offerings, cloud services, and social media platforms. At the same time, Mule iON provides a secure gateway to enterprise, allowing application teams to integrate and orchestrate their enterprise applications along with their cloud-based services.

We just announced the private beta for Mule iON, if you’d like to try it out you can sign up here, note that this is a limited beta so tell us what you’d like to do with Mule iON to get an early access account.

No related posts.


6 Responses to “Groundhog Day: Cloud Silos”

David Dossot February 10th, 2011, 9:43 am

I imagine so but does iON also provide access to payment providers via the newly launched Mule Payment Services? (http://blogs.mulesoft.org/mule-payment-services/)?

Ross Mason February 10th, 2011, 11:48 pm

Yes Mule Payment Services can be used with iON but we don’t make them available by default (yet), but you can still add them directly to your app

From the Mule’s Mouth » Cloud APIs and the new spaghetti factory April 27th, 2011, 4:29 am

[...] what have created a monster integration challenge that I talked about in a previous post about cloud silos. However, to get an idea of how this challenge will manifest itself I thought it would be [...]

From the Mule’s Mouth » Integration is the biggest challenge in the cloud May 25th, 2011, 5:02 am

[...] across departments in a single company, which means more integration points. The is the Cloud Silos effect that I have blogged about before. The number of organizations adopting SaaS and Cloud [...]

MuleSoft takes full-service integration to the cloud with iON iPaaS ESB platform | ZDNet June 3rd, 2011, 11:15 am

[...] integration (EAI) as a function is moving out of the enterprise and into the cloud. So called integration platform as a service (iPaaS) has popped up on the edge of the enterprise. But true cloud integration as a [...]

MuleSoft takes full-service integration to the cloud with iON iPaaS ESB platform | BriefingsDirect August 4th, 2011, 8:16 am

[...] integration (EAI) as a function is moving out of the enterprise and into the cloud. So called integration platform as a service (iPaaS) has popped up on the edge of the enterprise. But true cloud integration as a [...]

Leave a Comment