Storage Virtualization and Deduplication

I have been involved in some Storage projects at HP these days and started to get my foots wet. I can not say I am a real professional on storage virtualization yet, but slowly surely I am emerging. Since I like to share my knowledge, I decided to write this blog post about storage virtualization.

Storage today is typically 10-15 percent of a typical IT budget, but it is only purchasing costs included. To sum it, we need to add personnel costs of operating, support and maintenance and the electricity and cooling costs as well. It obviously varies between different companies, but it is estimated amount of purchases of hardware and software and service on this for about 20-50 percent of the total cost. While the data center power and cooling account for 10-20 percent. Operation and management costs is therefore of 30-50 percent. So, the biggest area to save money is management and administaration areas, where many repetitive data should be automated (Storage Deduplication). Review your processes! Being able to free up staff for other tasks of course leads to cost savings.

Another area to review is the utilization of hardware, which is often surprisingly low. Not infrequently it somewhere between 20 and 40 percent! It is depressingly low numbers. The reason is that traditional storage has many built-in mechanisms to provide redundancy and security. Features such as RAID and mirroring occupy storage space is not used in practice. An advice to companies is to impose thin provisioning. It is a method of optimizing the available storage space by allocating more servers, storage space than actually exists on the disks.

Of course, you can not fill up storage resources to 100 percent, there is always some spare capacity to meet peak loads. But an utilization rate of 70-80 percent is pretty moderate. The key is to learn user patterns for business applications. So that you know how the load looks like and can see that there is extra space for seasonal peaks, transparency is the basis for cost-effective storage.

Good to know about storage vitualization:

Storage is logical: Storage virtualization is a logical view and control of physical storage systems. Enterprise-class virtualization offers high levels of redundancy and advanced data management features, including snapshots and remote mirroring.

Volume management is key: Storage virtualization allows volumes to be managed in a centralized way across multiple distinct pyhsical arrays. For example, you can combine disk capacity from several arrays into a single virtual volume.

Hardware choice is yours: Virtualization makes it easier to use whatever type of storage hardware makes sense for your applications. In example, for testing and development with disposable data, you could store snapshots or copies of production data on inexpensive SATA disks rather than on expensive Fiber Channel gear. But, keep in mind that by virtualization total number of hardwares used is goind down while the strategic importance goes up. Thats why, it is more important to use good hardware.

Virtualization hides SAN Complexity: Storage virtualization is most used in SAN. Management of individual storage devices can be tedious and time-consuming, but storage virtualization helps administrators perform backup, archiving and recovery tasks more easily and faster by disguising the complexity ofthe SAN.

Storage virtualization is not new: Virtualization of data storage capacity goes back to the IBM mainframe days of the 1960s. Virtualization can be as simple as the partitioning of disks for storage and seperation of applications.

Administrative overhead is not reduced: Despite what you may have heard, storage virtualization does not reduce administrative overhead. Instead it moves the work from one place to centralized console of virtualization product. That said, storage virualization can certainly make your life easier.

 In addition to all these, the choice of the clouds are not supposed to be for life, it should be possible to move between different providers. But of course technology is not everything. Everyone wants to buy a box that solves problems, but the major improvements is to modernize management, to understand how applications use the data and to integrate new technologies.