The recent brouhaha about Twitter scalability has highlighted the growth of the latest spectator sport in the blogosphere – “armchair architect”. Everyone’s a Monday morning expert on which language/database/framework is/isn’t the secret to extreme scalability. The real secret is the architecture and organizational maturity. Here are some case studies to prove it:
A Conversation with Werner Vogels where Werner talks about using service-orientation to scale out massively distributed services which power the Amazon e-commerce platform. This is one of my favourites because it covers organizational as well as technical aspects of scalability. One of the unique attributes of Amazon is that service-orientation pervades everything – even their organizational structure. Developers are responsible for running their own services.Werner characterises the adoption of services as a challenging and major learning experience, but it has become one of their main strategic advantages. Key lessons learned:
- service-orientation is an excellent technique to achieve isolation and high levels of ownership and control
- prohibiting direct database access allows scaling and reliability improvements without affecting clients
- a single unified service access mechanism supports service aggregation, routing & tracking
- service orientation improves development and operational processes leading to more agility
- giving developers operational responsibility enhances the quality of the services.
The eBay Architecture (PDF) covers the evolution of eBay from 1998 to 2006. It’s a great example of how continuous reinvention is needed to keep up with rapidly growing scaling requirements. Frank Sommers writes a good summary and discussion where he argues that organizational capability is just as important as technical architecture for scalability. Another key ingredient of the eBay story is the ability to discard “conventional wisdom” when required. This is covered in an interview with Dan Pritchett, revealing some of the “rules” that eBay bends in order to scale.
- eBay.com doesn’t use transactions – mainly for scalability and availability reasons
- different data is treated in different ways – so best effort suffices for some
- references the CAP theorem – consistency, availability, partitioning – pick any two
- many have arrived at the same idea – and transactions are the first to go
Scalable Web Architectures is a great presentation on scalable web architectures in general and Flickr in particular. Also check out Cal Henderson’s list of his other presentations.
Architectures You’ve Always Wondered About provides slides from QCon London 2009 presentations. Case studies about eBay, Second Life, Yahoo!, Linked-In and Orbitz.
Avoiding the Fail Whale is a video in which Robert Scoble interviews architects from FriendFeed, Technorati and iLike.
Improving Running Components at Twitter – Evan Weaver describes how Twitter learned to scale by moving to a messaging architecture.
Real Life Architectures at High Scalability – provides a huge collection of pointers to architecture case studies from around the web.
13 comments ↓
Saul,
kudos is due. Very useful compedium of architectures that includes their broader context.
MarkD.
Thanks Mark, but the real kudos goes to High Scalability which does a great job of summarizing case studies from around the web, and to InfoQ who have a great thread on architecture.
Excellent post Saul. Enjoyed it!
[…] 56 Architecture Case Studies page tags: deliverables examples page_revision: 13, last_edited: 1242308052|%e %b %Y, %H:%M %Z (%O ago) edittags history files print site tools+ options edit sections append backlinks view source parent block rename delete help | terms of service | privacy | report a bug | flag as objectionable Hosted by Wikidot.com — get your free wiki now! Unless stated otherwise Content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License Click here to edit contents of this page. Click here to toggle editing of individual sections of the page (if possible). Watch headings for an “edit” link when available. Append content without editing the whole page source. Check out how this page has evolved in the past. If you want to discuss contents of this page – this is the easiest way to do it. View and manage file attachments for this page. A few useful tools to manage this Site. See pages that link to and include this page. Change the name (also URL address, possibly the category) of the page. View wiki source for this page without editing. View/set parent page (used for creating breadcrumbs and structured layout). Notify administrators if there is objectionable content in this page. Something does not work as expected? Find out what you can do. General Wikidot.com documentation and help section. Wikidot.com Terms of Service – what you can, what you should not etc. Wikidot.com Privacy Policy. _uff = false; _uacct = “UA-68540-5″; _udn=”wikidot.com”; urchinTracker(); _qoptions={ qacct:”p-edL3gsnUjJzw-” }; […]
po0d1d
ixroev
l2gl0s
2xi8ks
191soz
1yq1h6
3h9xeu
jyg2vv
hu3i8m
Leave a Comment