My last entry was about Gnip and I take this subject up again following on Dare Obasanjo’s blog entry on the same subject where Dare grapples with the problem of how to share social activity streams across the web. The fundamental problem is not new to anyone dealing with the problems of integration inside the firewall, a plethora of different apis and protocols leads to spaghetti architecture. What we seem to have now in the social web is JBORS – Just a Bunch of RESTful Services.

This is only a problem if people want these services to collaborate, but there certainly seems to be sufficient desire and opportunity to make this happen. I also think that the social web – chock full of personal and dynamic information, collaboratively maintained by an army of “ordinary people” is a good testbed for the techniques which may be applicable in other applications such as Electronic Health Records.

The Gnip approach to the challenge is to provide a type of ESB in the cloud. The basic functions of an ESB – transport and mediation – is applied to the various protocols which are exported or imported by popular social applications.

Dare worries that this approach puts Gnip in the middle of the social network as a “single point of failure”. This is the classic problem with the ESB and the source of much derision about “ESB in a box”. You’re not solving the problem of protocol/api/schema proliferation, you’re simply pushing the mappings into a central logical component which – depending on its architecture – may be a single point of failure. So what are the alternatives?

One approach would be to standardize the interfaces so as to minimise or completely eliminate the need for mediation. One might refer to this as a service-oriented architecture, but I hesitate to use such an unfashionable term. But there’s the rub. The social web is a dynamic, anarchic frontier on the bleeding edge of information technology. What effect would standardization have on that eco-system? Would it slow down or block off avenues of innovation? Would new business models be choked off, trapped in a proprietary cul de sac? Maybe! These are the dangers of premature standardization which are best applied to mature technologies and processes.

Even if you could standardize to the required level of completeness, how do you coordinate dozens of different companies to support the standards? It’s pretty much impossible. We’ve struggled with this inside the firewall for decades. For example how do you get Siebel and Metasolv and Retek and Primavera all standardizing around the same set of services? Well in this case, they all get acquired by Oracle and it becomes Oracle’s problem.

The first step in the standardization of the social web has been taken by Google with OpenSocial. If standardization of the social web ever happens I think it will be a long time before the requirement for mediation disappears. In the meantime we have Gnip, and Friendfeed and…watch this space.

From OSCON via O’Reilly Radar here’s a good case study of an architectural decision driven by the system requirements rather than the usual religious considerations that pollute the bloggosphere.

FriendFeed needed update info from Flikr but a REST-based “pull” approach is highly inefficient in this case. Instead the solution architects opted for a “push” approach using xmpp as the message transport. This is a really good presentation because it goes into the architectural choices and implications of “push” versus “pull”.

I characterize this as “pull vs push” rather than “REST vs xmpp” (or “REST vs *” or “why REST is crap”) because fundamentally it comes down to the best choice of how to synchronize changes between systems. You make this choice based on the usage characteristics of the different systems, the likely traffic volumes this will result in and the consequential resource impacts. Having made the choice between push or pull you then choose the appropriate message transport.

The web doesn’t do a lot of “push” and consequently there is not a lot of discussion about push and REST. Dare Obasanjo characterises it nicely:

Polling is a good idea for RSS/Atom for a few reasons

• there are a thousands to hundreds of thousands clients that might be interested in a resource so the server keeping track of subscriptions is prohibitively expensive
• a lot of these end points aren’t persistently connected (i.e. your desktop RSS reader isn’t always running)
• RSS/Atom publishing is as simple as plopping a file in the right directory and letting IIS or Apache work its magic

The situation between FriendFeed and Flickr is almost the exact opposite. Instead of thousands of clients interested in document, we have one subscriber interested in thousands of documents. Both end points are always on or are at least expected to be. The cost of developing a publish-subscribe model is one that both sides can afford.

Inside the firewall, the situation is often more akin to that between FriendFeed and Flikr. This is why messaging is more common inside the firewall than outside – not because of any universal superiority between REST versus messaging, but because the system requirements are different and often favour a push approach rather than pull.

While your over at Dare’s excellent Blog, be sure to also check out his discussion of push versus pull in the context of scaling Twitter and MS Exchange.  These are important considerations for designers of federated systems such as federated databases or federated messaging systems. The example of FriendFeed to Flikr could be considered as the first incremental step toward a federation.

A little while ago I expressed my dislike for the term WOA on the grounds that it refers to a style of SOA and my feeling that new TLAs just lead to more confusion. Its good to see the ZapThink guys are in full agreement and state their case in their trademark inimitable style.

While the WOA concept does indeed provide deeper insights into how to best implement a Service and create an infrastructural approach for scaling Services, we simply don’t see a need to identify this as a truly separate architectural approach…

ZapThink believes that the term Web-Oriented SOA represents greater clarity than WOA, since it disambiguates the desire to position WOA as an alternative to SOA as well as more accurately positions the concept at a lower level of abstraction than the SOA concept. Going forward, hence, we will prefer the term Web-Oriented SOA over WOA, since it provides greater clarity. And clarity is exactly what companies today need to make SOA a reality.

My emphasis added – hear hear!

But I still have a problem with this terminology. It doesn’t exactly roll off the tongue and the use of “oriented” twice in the same term is really ugly. Moreover, we’re still mixing two levels of abstraction into the same term.

“Web Oriented” is a reference to the style of implementation of the Services which comprise the SOA. So wouldn’t it be better to use the terminology “Web Oriented Services”?

To take it a step further, I think we can do better than “Web Oriented” in categorizing the Service implementation. The two main Service implementation patterns that we have in the current debate are implementations based on Web Services standards (ws-*) and those based on REST. A key differentiator between these implementation styles is that Web Services are based on an “interface description” of the service, whereas REST is based on a “resource” as the key entity that we operate on.

Hence I propose the terminology “Interface-Based Services” to refer to the ws-* Service implementations and “Resource-Based Services” to refer to Services implemented in a RESTful manner.

So at the top of the ontology we have SOA – our architecture comprised of Services as first-order citizens. Those Services may in turn be Interface-Based or Resource-Based in their implementation. Note that an SOA could quite easily comprise a combination of Resource-Based and Interface-Based Services.

I think this nomenclature is clearer…but it won’t catch on because these are not TLAs!

In “Web 2.0 success stories driving WOA and informing SOA”, Dion Hinchcliffe writes about how enterprise SOA can learn from the successes of Web2.0 and WOA^*.

“…since there’s little question that the core ideas behind SOA seem to be the right ones. Rather, it’s been how we’ve gone about designing and implementing SOAs that appears to be at the crux of the issue. As we look at the most successful examples of SOA actually working, we keep being drawn back to the Web itself.”

Along the way Dion points out a couple of differences between the Web and the enterprise which I think are pretty salient and underscore the issue that its not the technology that divides the two hemispheres of the service-oriented world.

“One big issue…is that enterprises are often very much unlike the Web. Many of the aspects that make the Web successful…just don’t exist in the enterprise with it’s wilderness of relational databases, proprietary applications, and silos of every description, despite some success in adding a traditional SOA layer on them.

The article Dion references in the last quote has this great summary graphic:

But in addition to the technical differences between the Web and the Enterprise, I think it is worth mentioning some fundamental cultural differences:

• The participants: When it comes to Web APIs that have been so successful, the conversation is basically between technologists. By-and-large it is technologists that are writing the mashups that demonstrate the success of Web2.0. Conversely with SOA in the enterprise, the conversation is (supposed to be) between technologists and business – and plenty has been written about the lack of success there.
• The success criteria: There are different success criteria for Web2.0 versus enterprise SOA. Very few Web2.0 companies currently make money out of their APIs. Success for Web2.0 is “eyeballs” and VC funding. In the enterprise, SOA success is all about hard ROI.
• The legacy: Most enterprise services must build on legacy applications, whereas Web2.0 applications are mostly greenfields and purpose built for the Web. Legacy applications rarely expose “resources” directly. If you’re lucky, the application exposes API methods which encapsulate an awful lot of business logic around a resource.
• The standards: Also part of the legacy is that WOA is “an emergent set of best practices…not a formal set of standards.” Let’s face it, WS-* are a formal set of standards designed by disparate committees often with competing agendas and ulterior motives. The structure and evolution of these standards has not been optimal for the users.

Fundamentally, I agree that Web2.0 in the enterprise and REST approaches to SOA have a lot of value and can teach us how to do SOA properly. Conversely REST needs to address enterprise concerns more fully. The key is recognizing the commonalities and the differences so that we can hit the sweet spot in between.

* BTW I’m not a big fan of the term WOA to denote “Web Oriented Architecture” or REST-ful approaches to Services. While I understand that distancing REST from WS-* makes good marketing sense, ultimately I think we are all talking about Services – so WOA is really a style of SOA. Jockeying for market position/dominance/discrimination has already got us into such a mess.

There’s a lot of interesting bloodletting debate going on about the different technical approaches to services development – RESTful versus ws-* (ws-splat). The great thing about this debate is that it helps expose fundamental requirements, preconceptions and trade-offs.

I’m currently on the sidelines…my own position is that while acknowledging that ws-* is overly complex and has led us down a lot of blind alleys in the standardization wars process, the fundamental aspects are useful and productive if used properly. REST approaches are conceptually elegant and simple in their use of and adherence to the “resource oriented” view of the world-wide-web, but I haven’t used them “in anger” yet and need to reserve judgement.

There is currently a very interesting discussion on the value of WSDL happening at TSS. One of the best contributions to the discussion from William Childers says:

WSDL is supposed to give us away of tying together the service, its address, its operations, the required message formats and the supported delivery channels. That’s very useful and is something that can be processed and used at either design time or run-time (e.g. WSIF).

If WSDL (and Schema) did not exist we’d have to invent them. You can argue about how good they are as solutions (not so great) but I don’t see how you can argue that solutions aren’t needed which I believe is one of Joseph O’s points.

On the other side…Stefan Tilkov at InfoQ gives a good defence of REST against the doubters.

Well reasoned discussion about these approaches will help to clarify each and also help to educate the wider user community. Currently the REST approach certainly dominates public web services (such as Google, Amazon, Facebook etc) and that ws-* dominates enterprise web services. I feel there is significant common ground between the two and we will see a middle way that bridges some of the differences and leads to both better ws-* implementations and better REST implementations.