Entries Tagged 'soa' ↓

I’ve been busy with customer projects for the last few weeks and hence the lack of postings. In this post I want to touch on some conversations about top-down versus bottom-up approaches to SOA.

This is a pertinent question because there is still a communications gap between the “business” and “IT” in many (especially large) organizations and as a result SOA initiatives are often driven from the IT side of the fence with little support or even understanding from the business.

There is a fundamental “chicken and egg” problem with SOA. It won’t gain support from the business unless a business case can be made…but conversely SOA payoffs are elusive without business support. Faced with this situation, many IT architects who want to develop an SOA approach are forced to try to conjure the egg out of thin air and hope that a chicken will hatch. The danger with this approach is that rather than gain an SOA you may end up with just a bunch of web services.

Two key facilities that an SOA initiative needs are leverage and vantage. Vantage to see where you need to go, and leverage to push the organization in that direction.

SOA is all about providing services to support the processes of the business – typically across organizational silos. There is a need to understand all the processes and their requirements – now and for the future. This visibility needs a vantage point “above the fray”. A point from where you can see the processes in their entirety. Without this vantage point you risk building the wrong services, or building services for the wrong processes or not anticipating future process changes. Moreover this vantage point must be experienced from the point of view
of the business – not just IT. So the question for an SOA initiative is “who in your organization has this vantage?” You may be lucky and have complete visibility from the IT cockpit, but that is rare. In general no one person has this vantage and it generally resides across different roles and people. Hence the need for some governance process to capture that vantage into a SOA plan.

So let’s suppose you have this vantage and want to execute on an SOA initiative. SOA is primarily about sharing – shared services built on shared resources using shared standards. Without sharing you wont get the SOA pay-off. But sharing requires communication and cooperation. Leverage is required to reconcile conflicting priorities in terms of requirements, schedule, funding, and resources. There may be some organizations where all these conflicts can be reconciled within the IT department, but I haven’t seen any yet. Typically resolution is required at an organizational level above the participants – that is the prime purpose of organizational hirearchy. Hence you need the appropriate leverage to get everyone moving down the SOA path.

While the type of issues your SOA initiative will face are fairly general, the specific issues and their resolution will be germane to your organization. This means that the place you will achieve the right level of vantage and leverage will be different for your organization. You might be lucky enough to work in an IT department that has the appropriate leverage and vantage, but that is rare. More likely you will have to go higher up to get these facilities for a successful SOA. This is the essence of the “top-down” approach to SOA.

I guess the “middle-out” approach might be characterised by “figure out how much leverage and vantage you have access to an scope your initiative from there.”

SOA Talk ask if there is a better term than “SOA Governance”. The “Big Brother” connotations of SOA Governance have bothered me for a while but I don’t think there is a need for new terminology.

At design time, what is needed is a shared understanding of the architecture principles, frameworks, standards and best practices used within your organisation for providing and consuming Services.

How do you ensure that this shared understanding is
a) communicated effectively,
b) understood and
c) implemented by all.

This goes way beyond technology solutions and encompasses the overall development culture. Tools can help – but they should be an enabling framework rather than an enforcing roadblock.

Update: Todd Biske has recently also blogged about the relationship between governance, tools and culture (here and here).

Amberpoint just released a survey on SOA Adoption and the headlines proclaim “SOA is mature, diverse and successful.” Hmmm…lets take a look at those results.

Of the respondents roughly 60% are either experimenting or in development. I read this as saying that SOA adoption is still far from the mainstream. The press releases ignore these and concentrate only on the projects in production. And the biggest impediment to SOA adoption was cited as “lack of SOA Expertise”. This was followed by organizational reluctance to change.

However, given the early stages of these projects, organizational impediments and lack of SOA expertise…everyone seems to be wildly successful. Only 1.5% of respondents reported a lack of success with their SOA adoption. This is amazing! SOA must truly be the secret sauce to success. Or maybe people aren’t owning up to the inevitable challenges and false starts that occur with any new technology adoption.

I think this is more an artifact of the self-selecting nature of the survey. Only a few percent of the people approached responded to the survey and it is likely that only successful organizations would be enthusiastic about responding (see the methodology section at the end of the report).

I also wonder how others in the surveyed organizations would characterise success of SOA adoption. Different people on the IT side and the business side would value different success criteria and would also rate differently on those success criteria.

In my last post on this topic I talked about the concept of an ESB. Here I talk about why you would want one.

There are plenty of whitepapers, analyst reports and vendor statements about the features and functions of the various ESB products. In my experience, the key advantages of using an ESB are less about features and functions and more about how you use it.

Standardization

One of the primary advantages of an ESB is that it gives you a standardized platform for integration. When everyone is using the same tools you can develop enterprise-wide frameworks, patterns and best practices for building re-usable services. Without a unifying platform, you get a divergence of integration methods which leads to inconsistency and higher cost of management and change. So an ESB platform helps with design-time governance. Note that this is not the same as standardization in the sense of using web-services standards. The important thing is that you use the ESB to support your own enterprise standards. These may be based on external standards – but that may be of secondary importance.

Loose Coupling

The bus architecture of an ESB encourages you toward a loosely coupled architecture.

• Physically decoupled by making use of message passing mechanisms (e.g. JMS) versus direct network connections (e.g. HTTP).
• Semantically decoupled by use of message transformation so that services are exposed in a system-neutral format which reduces application lock-in and reduces the cost of change.

Scalability and Reliability

Physical loose-coupling provides scalability advantages such as high-availability, fault-tolerance and load balancing. The messaging layer in the ESB directs messages between service endpoints to the appropriate instance of the endpoint. For example, in the event of a service provider failure, messages will be redirected to a backup provider – thus supporting high availability. In the case of load balancing, messages are distributed between redundant providers (or consumers) to handle high volumes of message traffic. You could say that physical loose-coupling supports change at the “micro” level where short term changes in the system topology can be compensated for via real-time message redirection.

Routing and mediation

Message routing supports scalability and fault tolerance. An ESB can also be used to support business-level routing and mediation. For example content-based routing allows services to be invoked based on the content of a service request. A business example would be routing of a customer enquiry to the branch where that customer account is located. A technical example would be the routing of a service request based on the version of the service being invoked.

Complex message exchange patterns

Traditional HTTP-based services support only one-to-one request-reply MEPs. An ESB supports more complex MEPs such as asynchronous one-way messaging and to multiple subscribers using topic-based messaging. Asynchronous publish and subscribe mechanisms support new ways of intermediating service consumers and subscribers – such as auditing, service monitoring – which are extremely useful for runtime management and governance of your services. Beyond mere governance, higher level business functions such as complex event processing (CEP) and business activity monitoring (BAM) are supported by this ability to “listen in” to service traffic on the ESB.

The benefits of an ESB that I’ve described above stem largely from the architecture of an ESB and in particular from the use of a message bus as the primary underlying transport. But it is important to understand that these benefits don’t automatically come “out of the box”. Your solution architecture (and your architects) must recognise and utilise the architectural principles underlying the ESB.

This week I had to travel for a few days. When rushing out of the house that morning I forgot my mobile! On top of that my flight got canceled and I ended up hanging around the airport for a few hours - sans mobile.

Its times like these you really feel how dependent we are on these gadgets. The main problem is that people have an expectation that they can get you instantly on the phone. This is extending now into the email arena where the expectation is that you are instantly addressable via email on your blackberry.

Fortunately, my laptop, wi-fi and web-based services came to the rescue (somewhat). My phone carrier allows me to set call diversions from their web site. So I could divert my number to a nearby landline. For outbound calls, I was able to buy a prepaid calling card online. An access number and PIN was emailed to me and voila any nearby landline becomes a gateway to the world. I had effectively provisioned a telephone account over the web!

I suppose the really smart thing would have been to use Skype, but my laptop lacks a microphone…