Dynamic Block-level Storage Caching
News:
- Dm-cache-based cloud caching study A cloud caching study based on dm-cache was published on Systor’14 and presetned by Dr. Zhao in the conference at Haifa, Iseral. (2014-07-01)
- CloudVPS activates Linux SSD Caching with dm-cache! Dm-cache has now passed beta-test and is officially in production use to support the virtual machines in CloudVPS!More information can be found here. (2012-10-10)
- SSD-optimized I/O scheduling and cache management! In collaboration with Marvell, a leading storage controller provider, dm-cache researchers are creating new I/O scheduling and cache management techniques optimized for the unique characteristics of flash-based SSDs. VISA PhD student Douglas Otstott worked with Marvell engineers closely to implement this research on Marvell’s SSD controllers as part of his summer internship at Marvell. (2012-09-01)
- Dm-cache deployed in the Cloud! In collaboration with CloudVPS, a leading European cloud provider, dm-cache is now deployed in the clouds to provide scalable virtual machine storage using SSDs. VISA PhD student Dulcardo Arteaga has been working closely with CloudVPS engineers to make this happen as part of his summer internship at CloudVPS. Free Linux VMs are offered to users who are interested inparticipating the beta test! (2012-07-23)
- New performance results! A set of new experiments were done to show case the great speedup from dm-cache on SSDs (See the graphs below). (2012-07-14)
- New patch for kernel 3! New stable dm-cache patch is released for kernel 3 (2012-05-23)
- Dm-cache inspired Facebook FlashCache! The flash-based caching software, FlashCache, developed by Facebook is inspired by the design of dm-cache (2010-07-31)
Project Summary
Storage networking is becoming increasingly important, especially with the emergence of SAN over IP technologies. It plays an important role in resource consolidation and management for various systems, spanning from a large-scale data center to a set of interconnected workstations. However, existing storage systems may not scale well under the load from relatively large number of clients. Local disk caching has the potential to solve this problem by leveraging data locality with client-side storage, but it is lacking for typical storage networks.
Dm-cache solves this problem by providing a generic block-level disk cache for storage networking. It is built upon the Linux device-mapper, a generic block device virtualization infrastructure. It can be transparently plugged into a client of any storage system, including SAN, iSCSI and AoE, and supports dynamic customization for policy-guided optimizations. Experimental evaluation based on file system benchmarks and typical applications show that dm-cache can significantly improve the performance and scalability of a storage system by orders of magnitude.
 |
 |
| Dm-cache Architecture | Dm-cache deployment in cloud systems |
Currently this project is focusing on applying dm-cache to cloud computing. Block-level distributed storage systems (e.g., SAN, iSCSI) are commonly used in the emerging cloud computing systems to provide virtual machine (VM) storage. They allow fast VM migration across different hosts and improved VM availability leveraging typical fault-tolerance measures (e.g., RAID) available in such storage systems. However, as the size of cloud systems and the number of hosted VMs rapidly grow, the scalability of shared block-level storage systems becomes a serious issue. This project proposes to address this issue by using client-side storage to implement block-level caching and exploit the data locality available in VM data accesses. By leveraging the capacity of fast storage devices such as SSD available on the VM hosts, this approach has the potential to substantially improve the performance of VMs and the load on the shared storage system. This approach is implemented upon dm-cache, a generic block-level caching utility. Our current prototype supports cache sharing across different co-hosted VMs in order to maximize cache utilization.
Performance
 |
 |
| VM booting | IOzone |
Performance results on a virtual machine storage system. More results can be found here.
Publications
- Dulcardo Arteaga and Ming Zhao, “Client-side Flash Caching for Cloud Systems,” ACM International Systems and Storage Conference (Systor’14), June 2014. (paper)
- Dulcardo Arteaga, Douglas Otstott, and Ming Zhao, “Dynamic Block-level Cache Management for Cloud Computing Systems,” Conference on File and Storage Technologies (FAST’12), February 2012. (paper, poster, slides)
- Eric Van Hensbergen; Ming Zhao, “Dynamic Policy Disk Caching for Storage Networking,” In IBM Research Report (RC24123), November 2006. (Paper)
Source Code
Contact Us
Acknowledgement
This material is partially sponsored by CloudVPS, a leading cloud service provider from the Netherlands, Marvell, a leading semiconductor company, and the National Science Foundation under grants CCF-0937973 and CCF-0938045. The initial dm-cache code was developed during Dr. Zhao’s summer internship at IBM Research.