Programmer Thoughts

By John Dickinson

Democratization of Data

October 28, 2011

You should have control over your data. If you want to host your own data, you should be able to. If you want to pay someone else to host your data, you should be able to interact with their systems in the same way you interact with your local system. You should be able to change hosting providers without changing your interface. You should be able to pull your data from a hosting provider to host it locally. You should be able to host your data across many providers seamlessly. You should be able to move your compute needs to your data storage. You should be able to separate your concerns over distributing your data from who is distributing your data.

You can do three things with data: store it, compute it, and deliver it. You should be in control of where you data is stored, how you compute on it, and how you distribute it. This is the promise of Openstack: a common infrastructure that puts you in control of your data. This promise is the democratization of data.

Ultimately, the democratization of data comes down to storage. Openstack provides a high-quality object storage system called swift. Swift is ideal for unstructured data that can grow without bound. Backups and static web content are perfect examples of good use cases for swift.

Computing on your data is provided by another Openstack project: nova. Nova enables the management of large numbers of dynamic virtual machines. It is directly comparable to AWS EC2 and Rackspace Cloud Servers.

Openstack currently lacks integration with content delivery networks. Such integration should facilitate simple distribution and management of data with exiting CDN providers (or perhaps even allow you to run your own CDN).

These three pillars, along with various complementary projects like identity management, queueing, and block storage, provide a foundation upon which we can build the future.

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

The thoughts expressed here are my own and do not necessarily represent those of my employer.