Top Down Procurement

Imagine you have a unique business offering that needs IT delivery and support. There are two approaches you can take to do this:

Bottom Up:

  1. Determine your business requirements.
  2. Choose an application which you think best fits those requirements.
  3. Spend a lot of money customizing the application to better fit your requirements and integrating it to other systems.
  4. Deploy the application.
  5. Hope you support the business adequately within time and budget.

Top Down:

  1. Map out the business processes to support your offering.
  2. Determine the services required to support the business processes.
  3. For services that already exist – do nothing.
  4. For new services – build or buy applications to support those services.
  5. Implement the processes.
  6. Hope….(ditto).

Often “bottom up” is the default approach, even though “top down” can be more cost-effective and provide a closer fit to your business requirements. However “top down” comes with some risk, because most off-the-shelf applications are built to support “bottom up” acquisition. You might end up having to do integration anyway.

As the world moves toward service portfolio management rather than application portfolio management, vendors move towards offering services rather than applications and the “top down” approach will become more widespread and more viable. That will be a good thing, I think.

The Real Economics of SOA

The title of the InfoQ article, “Economics of Service Orientation” caught my interest, but unfortunately I was left disappointed. I felt that the article missed the mark somewhat and also missed some of the important costs of Service Orientation in its eagerness to concentrate on the benefits.

In my view, the real economics of SOA is about how many times you pay for the same function.

In the world of packaged applications you have very coarse control over the number and bundling of the functions that you have available. If you buy a packaged application, then you typically buy a bundle of functions or capabilities – some of these hopefully will be functions that you need, yet others will be functions that you don’t need.

An example of this is if you buy a billing application which has inbuilt functionality for handling customer contact details. You may not need customer contacts in your billing application because it already exists in your CRM (for example). However, as delivered, the billing application requires its own proprietary customer contact details module because it is incapable of working with a 3rd party’s CRM data. So there are a bunch of costs associated with purchasing a packaged application:

  1. The cost of the functionality that you require.
  2. The cost of the functionality that you don’t require (the vendor charges licence and support fees for the whole package, not just the bits you really want to use).
  3. The infrastructure cost of hosting all this functionality – required and not required.
  4. The cost of integrating the functionality you require into your business processes.
  5. The cost of integrating and potentially mitigating the functionality that you don’t require.

If we consider cost 5) in the context of my example. Even though you have a perfectly good CRM, the billing system requires contact details to be managed within its own application database, so you have to integrate your CRM with the billing application. There may be side-effects of this integration which need to be mitigated. You may need to invent whole new business processes to manage duplication between two or more systems and the inevitable synchronization problems that arise.

In an SOA utopia, there is more fine grained control over the functions that you acquire from a vendor. In particular you can buy only the functions that you need and then have them plug into other services in your information infrastructure. This saves a number of costs:

  1. You only pay for the function that you require
  2. You don’t pay for the functions that you don’t require
  3. You only pay to host the functionality you have bought – not a bunch of tag-alongs. And if your function is remotely hosted, the costs of hosting the new functionality may be very low.
  4. You pay only the cost of integrating the required functionality into your business processes
  5. You don’t have to pay to integrate and mitigate functionality you don’t require

So the economic value of SOA is in the finer granularity with which you can manage the cost of your IT functionality. The job of the enterprise architect moves from managing a portfolio of applications – complete with duplicates and overlaps – to one of managing services or functions, with hopefully minimal duplicates and overlaps.

But this picture wouldn’t be complete without considering the additional overheads required for a well functioning SOA. Architectural standards and governance are required to ensure that you buy the right services at the right time and that the services can be integrated properly into your business processes. You should then add in costs associated with:

  • There is a cost for running an enterprise architecture function – managing standards and governance.
  • The cost of acquiring a specific function may be higher because of the additional planning and research required to get the “right” function for your business processes and to fit into your enterprise services framework.
  • The integration cost of exposing functions as services is generally going to be higher than just building a point integration. This is because of the greater degree of planning involved with providing a service in a reliable and re-usable manner.

But hopefully the additional cost of architectural governance and standards should be offset by the savings in managing services as opposed to packaged applications.

As a final note, it’s interesting to note that the economic model is similar for the vendors as it is for their customers. This is why many packaged application vendors are hopping on the bandwagon of SOA. These vendors have acquired applications with overlapping functionality and require a more fine-grained approach for managing and selling their functionality.

Web-Oriented SOA

A little while ago I expressed my dislike for the term WOA on the grounds that it refers to a style of SOA and my feeling that new TLAs just lead to more confusion. Its good to see the ZapThink guys are in full agreement and state their case in their trademark inimitable style.

While the WOA concept does indeed provide deeper insights into how to best implement a Service and create an infrastructural approach for scaling Services, we simply don’t see a need to identify this as a truly separate architectural approach…

ZapThink believes that the term Web-Oriented SOA represents greater clarity than WOA, since it disambiguates the desire to position WOA as an alternative to SOA as well as more accurately positions the concept at a lower level of abstraction than the SOA concept. Going forward, hence, we will prefer the term Web-Oriented SOA over WOA, since it provides greater clarity. And clarity is exactly what companies today need to make SOA a reality.

My emphasis added – hear hear!

But I still have a problem with this terminology. It doesn’t exactly roll off the tongue and the use of “oriented” twice in the same term is really ugly. Moreover, we’re still mixing two levels of abstraction into the same term.

“Web Oriented” is a reference to the style of implementation of the Services which comprise the SOA. So wouldn’t it be better to use the terminology “Web Oriented Services”?

To take it a step further, I think we can do better than “Web Oriented” in categorizing the Service implementation. The two main Service implementation patterns that we have in the current debate are implementations based on Web Services standards (ws-*) and those based on REST. A key differentiator between these implementation styles is that Web Services are based on an “interface description” of the service, whereas REST is based on a “resource” as the key entity that we operate on.

Hence I propose the terminology “Interface-Based Services” to refer to the ws-* Service implementations and “Resource-Based Services” to refer to Services implemented in a RESTful manner.

So at the top of the ontology we have SOA – our architecture comprised of Services as first-order citizens. Those Services may in turn be Interface-Based or Resource-Based in their implementation. Note that an SOA could quite easily comprise a combination of Resource-Based and Interface-Based Services.

I think this nomenclature is clearer…but it won’t catch on because these are not TLAs!

Waiting for the great leap forward

One of the original and fundamental tenets of the SOAP standard was that the SOAP message is independent of the underlying transport. Ostensibly you could use SOAP over HTTP, JMS, email, FTP etc. but the reality is that a standard binding has only ever existed for SOAP over HTTP. To paraphrase Henry Ford – “you can have any SOAP transport you like, as long as its HTTP”.

While HTTP is undoubtedly a good choice for SOAP – given its ubiquity – there is at least one other transport which demands attention. This is the JMS transport which is widely used inside the firewall of many organizations. Of all the companies that I work with, their SOA infrastructure heavily relies on JMS transports inside the firewall, with HTTP transports outside the firewall or to selected service end-points such as web pages. Of course my experience has significant selection effects, but nevertheless JMS is an important transport in many SOAs. Testament to this is that every major web-services product vendor (save Microsoft) supports SOAP over JMS (and even Microsoft now has SOAP over MSMQ as an important part of WCF).

The fly in the ointment is that there has never been a standardized binding for SOAP over JMS and as a result there is little interoperability between SOAP/JMS solutions provided by different platform vendors. If you happen to have any combination of different web-service platforms in your organization, then they cannot easily communicate with each other using SOAP over JMS without performing some unnatural acts.

Some of issues that need to be considered with a SOAP binding to JMS are:

  • How do you represent the message content – text or binary? Most vendors have chosen a text message representation, but that has problems with multi-byte encodings, so other vendors have gone with a byte message representation.
  • What headers do you define and what should their names be? How do you use the standard JMS headers? different vendors have different naming conventions and semantics.
  • In the WSDL description, how do you represent the connection details to the JMS provider?
  • How do your service endpoints manage the different message exchange patterns that are available with message-oriented transports?

Each of the vendors went their own way on many of these issues and as far as interoperability was concerned they basically ceded the field to HTTP. They made life difficult for large organizations with heterogeneous platforms and in my opinion didn’t do themselves any favours on the way. (Actually SOAP-encoding interoperability was so broken for a while that noone noticed the JMS issues…so maybe it wasn’t so bad).

Subsequently it was great to see some of the vendors get together a couple of years ago to agree on a standard SOAP binding for JMS that addresses most of the important considerations. The result was a Member Submission to W3C in September last year. My understanding is that this submission was previously circulated through most of the vendor community so hopefully it has general agreement on the technical details.

This has now taken its first steps to standardization with the initiation of a SOAP-JMS Binding Working Group who aim to publish a recommendation by April next year. Hopefully vendor support of the binding will be hot on its trail.

Note that the standard binding won’t address the fact that different JMS implementations do not interoperate. For example, a TIBCO JMS client will not be able to talk to a Websphere JMS provider because JMS is an API standard, not a wire-protocol standard. What the SOAP/JMS binding standard does mean is that once you have settled on a standard JMS provider for your services, you could define your service description in standard WSDL and your service provider (say Websphere or TIBCO or WSO2) and your service consumer (say TIBCO or WebLogic or Axis) would be able to communicate directly using SOAP over JMS “out of the box”.

Its been eight years (almost to the day) since SOAP 1.1 came out with the HTTP binding. Wouldn’t it be great if a standard JMS binding could be achieved within the decade! It’s been a very long wait. The JMS binding should have happened a lot sooner and I can’t say the “wait has been worth it” but it does fill an important hole in the Web-Services standards.

So what do we do in the meantime? You can eschew JMS altogether and stick with HTTP, but that requires another lot of hard work. You can stick with one and only one service platform, but that is difficult in large heterogeneous organizations – which is where SOA is supposed to provide maximum benefit. Or you can continue to do what many SOA implementers have done and deal with SOAP directly at the JMS layer – effectively using SOAP as plain-old-XML over JMS. I wrote more about this approach recently.

Another thing you can do in the meantime is ask your vendor when will they support the new SOAP/JMS binding?

WS-* vs REST (ack)

There’s a lot of interesting bloodletting debate going on about the different technical approaches to services development – RESTful versus ws-* (ws-splat). The great thing about this debate is that it helps expose fundamental requirements, preconceptions and trade-offs.

I’m currently on the sidelines…my own position is that while acknowledging that ws-* is overly complex and has led us down a lot of blind alleys in the standardization wars process, the fundamental aspects are useful and productive if used properly. REST approaches are conceptually elegant and simple in their use of and adherence to the “resource oriented” view of the world-wide-web, but I haven’t used them “in anger” yet and need to reserve judgement.

There is currently a very interesting discussion on the value of WSDL happening at TSS. One of the best contributions to the discussion from William Childers says:

WSDL is supposed to give us away of tying together the service, its address, its operations, the required message formats and the supported delivery channels. That’s very useful and is something that can be processed and used at either design time or run-time (e.g. WSIF).

If WSDL (and Schema) did not exist we’d have to invent them. You can argue about how good they are as solutions (not so great) but I don’t see how you can argue that solutions aren’t needed which I believe is one of Joseph O’s points.

On the other side…Stefan Tilkov at InfoQ gives a good defence of REST against the doubters.

Well reasoned discussion about these approaches will help to clarify each and also help to educate the wider user community. Currently the REST approach certainly dominates public web services (such as Google, Amazon, Facebook etc) and that ws-* dominates enterprise web services. I feel there is significant common ground between the two and we will see a middle way that bridges some of the differences and leads to both better ws-* implementations and better REST implementations.