They don't teach you this in college, but the fundamental theorem of the software industry is the idea that everything needs to be rewritten all the time. As a corollary, web startup engineers believe that there is no problem but scalability, and architecture is its solution. And thus, the NoSQL movement was born.The idea is that object relational databases like MySQL and PostgreSQL have lapsed their useful lifetimes, and that document-based or schemaless databases are the wave of the future. Never mind of course that MySQL was the perfect solution to everything a few years ago when Ruby on Rails was flashing in the pan. Never mind that real businesses track all of their data in SQL databases that scale just fine. (For Silicon Valley readers, Walmart is a real business, Twitter is not.)
Invariably, all web projects start off with something like Rails or Django, most likely backed by MySQL. The data relationships are easy to model, and the application works well. If you are lucky enough that people actually use your application, eventually you will start to see some performance issues. At this point, a developer who values technological purity over gettin' shit done will advocate "rewriting the whole thing in a weekend using Cassandra". And if he's smart enough, he might just pull it off. (Of course, said developer has only migrated the app to use a different data store - all of the ancillary support code was conveniently ignored)
So you've magically changed your backend from MySQL to Cassandra. Stuff will just work now, right? Well, no. Did you know that Cassandra requires a restart when you change the column family definition? Yeah, the MySQL developers actually had to think out how ALTER TABLE works, but according to Cassandra, that's a hard problem that has very little business value. Right.
I'm not just singling out Cassandra - by replacing MySQL or Postgres with a different, new data store, you have traded a well-enumerated list of limitations and warts for a newer, poorly understood list of limitations and warts, and that is a huge business risk.
You Are Not Google
The sooner your company admits this, the sooner you can get down to some real work. Developing the app for Google-sized scale is a waste of your time, plus, there is no way you will get it right. Absolutely none. It's not that you're not smart enough, it's that you do not have the experience to know what problems you will see at scale.
Besides, did you know that Google Adwords is implemented on top of MySQL? What, that business critical code that operates at massive scale doesn't use BigTable? No, in fact there is such enormous value in sticking with what works that Google identifies problems with InnoDB at scale and submits patches, instead of saying "MySQL doesn't scale, let's dump it for something else".
NoSQL will never die, but it will eventually get marginalized, like how Rails was marginalized by NoSQL. In the meantime, DBAs should not be worried, because any company that has the resources to hire a DBA likely has decision makers who understand business reality.
Top photo credit Paul Russell