A recent comment on my post about the value of enterprise architecture suggested that enterprise architects should be directly involved in Scrum projects. In that post I suggested that the primary value of EA is “continuity” and “self-correction” across all the IT initiatives. If these qualities represent the value of EA, then the mechanism for realising that value must involve communication.

Communication is the key to ensuring that IT project deliverables provide cohesion across organizational, geographical and temporal boundaries. I have previously written that communication and governance are effectively the same thing. The problem is that “governance” needs to overcome the image of heavy-handed, top-down, hierarchical “rules” handed down by “out of touch” enterprise architects – like commandments on tablets of stone. EA must be seen as an enabler, not a roadblock.

But communication and governance must also be relevant to the organizational culture. So if the culture embraces agile development methods such as Scrum, then on the face of it the governance structure ought to conform with agile practices which favour direct verbal communication over documentation. If it is true that an embedded business representative can communicate requirements more effectively than a written requirements document, then perhaps it is also true that an embedded enterprise architect can “govern” more effectively than a written EA document.

However, on reflection, I don’t think the analogy of EA as a “customer” of the project is quite as strong as I first thought. There are a couple of differences between “business requirements” and “EA requirements”.

First, business requirements for a project are communicated primarily to that project. Few if any other IT projects would be interested in these particular business requirements which we are trying to encode into an IT solution. However, EA requirements must be communicated consistently across multiple projects and consistency demands that the EA requirements – the “governance principals” – need to be encoded in some written form, independent of an individual architect or even team of architects.

Second, business requirements change very frequently within the timespan of an IT project. Indeed the development process itself can change business requirements. This is why agile methods value verbal communication methods over written – so as to allow for frequent change in business requirements. EA requirements will not change as rapidly. Even though the Enterprise Architecture will evolve over time, it should not change over the course of a single project. Indeed if your Enterprise Architecture undergoes sudden rapid shifts (e.g. adopting an overnight SOA mandate) then you have a recipe for IT disaster. Hence the agile “insight” that written business requirements cannot keep up with reality, is not quite as relevant when it comes to EA requirements.

So my response to Paul’s comment is still: Yes! I agree that an EA representation on a Scrum project would be a good idea – to communicate, elucidate and interpret EA requirements to the project and importantly to take new lessons learned back from the project into the governance structures. However, this is no substitute for EA “documentation” – artefact-based governance – which is necessary to ensure consistency and continuity.

The title of the InfoQ article, “Economics of Service Orientation” caught my interest, but unfortunately I was left disappointed. I felt that the article missed the mark somewhat and also missed some of the important costs of Service Orientation in its eagerness to concentrate on the benefits.

In my view, the real economics of SOA is about how many times you pay for the same function.

In the world of packaged applications you have very coarse control over the number and bundling of the functions that you have available. If you buy a packaged application, then you typically buy a bundle of functions or capabilities – some of these hopefully will be functions that you need, yet others will be functions that you don’t need.

An example of this is if you buy a billing application which has inbuilt functionality for handling customer contact details. You may not need customer contacts in your billing application because it already exists in your CRM (for example). However, as delivered, the billing application requires its own proprietary customer contact details module because it is incapable of working with a 3rd party’s CRM data. So there are a bunch of costs associated with purchasing a packaged application:

1. The cost of the functionality that you require.
2. The cost of the functionality that you don’t require (the vendor charges licence and support fees for the whole package, not just the bits you really want to use).
3. The infrastructure cost of hosting all this functionality – required and not required.
4. The cost of integrating the functionality you require into your business processes.
5. The cost of integrating and potentially mitigating the functionality that you don’t require.

If we consider cost 5) in the context of my example. Even though you have a perfectly good CRM, the billing system requires contact details to be managed within its own application database, so you have to integrate your CRM with the billing application. There may be side-effects of this integration which need to be mitigated. You may need to invent whole new business processes to manage duplication between two or more systems and the inevitable synchronization problems that arise.

In an SOA utopia, there is more fine grained control over the functions that you acquire from a vendor. In particular you can buy only the functions that you need and then have them plug into other services in your information infrastructure. This saves a number of costs:

1. You only pay for the function that you require
2. You don’t pay for the functions that you don’t require
3. You only pay to host the functionality you have bought – not a bunch of tag-alongs. And if your function is remotely hosted, the costs of hosting the new functionality may be very low.
4. You pay only the cost of integrating the required functionality into your business processes
5. You don’t have to pay to integrate and mitigate functionality you don’t require

So the economic value of SOA is in the finer granularity with which you can manage the cost of your IT functionality. The job of the enterprise architect moves from managing a portfolio of applications – complete with duplicates and overlaps – to one of managing services or functions, with hopefully minimal duplicates and overlaps.

But this picture wouldn’t be complete without considering the additional overheads required for a well functioning SOA. Architectural standards and governance are required to ensure that you buy the right services at the right time and that the services can be integrated properly into your business processes. You should then add in costs associated with:

• There is a cost for running an enterprise architecture function – managing standards and governance.
• The cost of acquiring a specific function may be higher because of the additional planning and research required to get the “right” function for your business processes and to fit into your enterprise services framework.
• The integration cost of exposing functions as services is generally going to be higher than just building a point integration. This is because of the greater degree of planning involved with providing a service in a reliable and re-usable manner.

But hopefully the additional cost of architectural governance and standards should be offset by the savings in managing services as opposed to packaged applications.

As a final note, it’s interesting to note that the economic model is similar for the vendors as it is for their customers. This is why many packaged application vendors are hopping on the bandwagon of SOA. These vendors have acquired applications with overlapping functionality and require a more fine-grained approach for managing and selling their functionality.

I have long held the belief that governance is more about effective communication than anything else. To quoth my own horn:

…what is needed is a shared understanding of the architecture principles, frameworks, standards and best practices used within your organisation for providing and consuming Services.

How do you ensure that this shared understanding is
a) communicated effectively,
b) understood and
c) implemented by all.

A corollary of this view is that any governance tools ought to support this communication as their primary function. I’ve often thought that a wiki would make a pretty good governance tool. Good to see that Martin Fowler agrees. He describes an interesting “Human centric registry” based on a wiki with just enough automation to remove the drudgery. Martin describes the principles of the registry as:

• People develop and use services, so orient it around people
• Don’t expect people to enter stuff to keep it up to date, people are busy enough as it is.
• Make it easy for people to read and contribute.

(sorry UDDI, thank you for playing).

I’ve been busy with customer projects for the last few weeks and hence the lack of postings. In this post I want to touch on some conversations about top-down versus bottom-up approaches to SOA.

This is a pertinent question because there is still a communications gap between the “business” and “IT” in many (especially large) organizations and as a result SOA initiatives are often driven from the IT side of the fence with little support or even understanding from the business.

There is a fundamental “chicken and egg” problem with SOA. It won’t gain support from the business unless a business case can be made…but conversely SOA payoffs are elusive without business support. Faced with this situation, many IT architects who want to develop an SOA approach are forced to try to conjure the egg out of thin air and hope that a chicken will hatch. The danger with this approach is that rather than gain an SOA you may end up with just a bunch of web services.

Two key facilities that an SOA initiative needs are leverage and vantage. Vantage to see where you need to go, and leverage to push the organization in that direction.

SOA is all about providing services to support the processes of the business – typically across organizational silos. There is a need to understand all the processes and their requirements – now and for the future. This visibility needs a vantage point “above the fray”. A point from where you can see the processes in their entirety. Without this vantage point you risk building the wrong services, or building services for the wrong processes or not anticipating future process changes. Moreover this vantage point must be experienced from the point of view
of the business – not just IT. So the question for an SOA initiative is “who in your organization has this vantage?” You may be lucky and have complete visibility from the IT cockpit, but that is rare. In general no one person has this vantage and it generally resides across different roles and people. Hence the need for some governance process to capture that vantage into a SOA plan.

So let’s suppose you have this vantage and want to execute on an SOA initiative. SOA is primarily about sharing – shared services built on shared resources using shared standards. Without sharing you wont get the SOA pay-off. But sharing requires communication and cooperation. Leverage is required to reconcile conflicting priorities in terms of requirements, schedule, funding, and resources. There may be some organizations where all these conflicts can be reconciled within the IT department, but I haven’t seen any yet. Typically resolution is required at an organizational level above the participants – that is the prime purpose of organizational hirearchy. Hence you need the appropriate leverage to get everyone moving down the SOA path.

While the type of issues your SOA initiative will face are fairly general, the specific issues and their resolution will be germane to your organization. This means that the place you will achieve the right level of vantage and leverage will be different for your organization. You might be lucky enough to work in an IT department that has the appropriate leverage and vantage, but that is rare. More likely you will have to go higher up to get these facilities for a successful SOA. This is the essence of the “top-down” approach to SOA.

I guess the “middle-out” approach might be characterised by “figure out how much leverage and vantage you have access to an scope your initiative from there.”

SOA Talk ask if there is a better term than “SOA Governance”. The “Big Brother” connotations of SOA Governance have bothered me for a while but I don’t think there is a need for new terminology.

At design time, what is needed is a shared understanding of the architecture principles, frameworks, standards and best practices used within your organisation for providing and consuming Services.

How do you ensure that this shared understanding is
a) communicated effectively,
b) understood and
c) implemented by all.

This goes way beyond technology solutions and encompasses the overall development culture. Tools can help – but they should be an enabling framework rather than an enforcing roadblock.

Update: Todd Biske has recently also blogged about the relationship between governance, tools and culture (here and here).