[Author's note: This is one of a series of posts about how developers can design and code differently to take advantage of the cloud. Check out the rest of the series: Part 1: Ender’s Homework, Part 2: Get Out Your CB, Part 3: "Do you want fries with that?", Part 4: Check Those Vitals, and Part 5: Auto Install. Please subscribe, follow me on Twitter or Google+, and check back weekly to explore the topic in full.]
All your (data)base are belong to us
In 2008, I bought the assets of a social networking company that had launched in Silicon Valley 18 months earlier. The system they’d built was designed to compete with Facebook (which, at the time, had only about 10-20 million users and was not yet the 800-lb gorilla of the space). Its engineers had planned for millions of folks to be updating their own private corners of the digital universe, 24x7x365.
As I studied the code in this system, many of my comfortable assumptions about database design were challenged. For example, the system needed to emit emails notifying users that someone had commented on their timeline. I assumed that there’d be some sort of notification table that tracked these events until they turned into outgoing emails, but as I dug in I discovered not one notification table, but 100.
100 tables?! What kind of crazy, denormalized database strategy was that?
Understand what you need from your database
It turns out that notifications were a major bottleneck in the system. When you have hundreds or thousands of edits per second, the overhead required to lock the notification table and perform an atomic update is prohibitive. So the designers had sharded the database such that only 1/100th of it needed to be locked at a time. Sharding also allowed them to support multiple write masters, although it complicated HA and replication. I’m not sure that was an ideal solution, but it certainly scaled higher and was more cloud-savvy than the naive answer.
Cloud changes how applications scale. It also raises the bar for database security. It’s not an accident that cloud computing has stimulated several recent innovations in DB technology, including a wave of NoSQL and NewSQL technologies. Think Cassandra, Mongo, InfiniteGraph, Amazon RDS and DynamoDB…
Your application may be quite different from my overengineered notification mechanism, but if you want to cloudify, it’s still worth spending time to re-examine easy assumptions about what you truly need from a database:
- Do you truly need to run arbitrary queries, or just a small subset that can be carefully optimized to run tens to thousands of times faster?
- Can you use idempotence and eventual consistency to relax demands for globally atomic transactions?
- Can you use a full-text indexing solution like Lucene/Solr (or, better yet, the 100x-faster and more scalable technology from Perfect Search) to offload some query burden?
- Would a document store (possibly implemented inside a traditional RDBMS, like Postgresql’s JSON feature) serve your needs better?
- How does security need to change if your app communicates through less-than-private cloud pipes?
This last question is important. The list of applications that have naively moved to cloud without validating security in their brave new world, and then suffered embarrassing hackage, is all too long.
Cloud doesn’t automatically invalidate what you’re already doing with databases. A lot of applications transfer to the cloud with almost no changes to their database layers, and they run great. However, cloud makes it worth your time to re-examine database assumptions, because it changes how applications (and the database technologies that back them) scale. Latency is different. HA gets simpler in some ways and more complicated in others. You may not need traditional backups of your database, if you can guarantee redundant copies of your data in multiple availability zones.
The moral to the story, then, is not to jump onto some shiny, new, “cloudified” database technology. Rather, it’s to be aware of ramifications and ponder them thoughtfully.
Comment and Share: Have you seen any patterns in how database assumptions change when applications become more cloud-friendly? What works, and what doesn’t?