VPFS

QoS-driven Storage Management for High-end Computing Systems

---

Project Summary:

In today’s high-end computing (HEC) systems, the parallel file system (PFS) is at the core of the storage infrastructure. PFS deployments are shared by many users and applications, but currently there are no provisions for differentiation of service – data access is provided in a best-effort manner. As systems scale, this limitation can prevent applications from efficiently utilizing the HEC resources while achieving their desired performance and it presents a hurdle to support a large number of data-intensive applications concurrently. This NSF HECURA project tackles the challenges in quality of service (QoS) driven HEC storage management, aiming to support I/O bandwidth guarantees in PFSs.

Participants

• Florida International University
  • Dr. Ming Zhao
  • Dulcardo A. Clavijo
  • Yiqi Xu
  • Xinyue Fan

• University of Florida
  • Dr. Renato Figueiredo
  • Yonggang Liu

• IBM
  • Dr. Seetharami R. Seelam

Publications

• Yiqi Xu, Lixi Wang, Dulcardo Arteaga, Ming Zhao, Yonggang Liu, Renato Figueiredo, and Seelam Seetharami, “Virtualization-based Storage Management for High-end Computing Systems,” Technical Report 2010-05-01, School of Computing and Information Sciences, Florida International University, 2011.
• Yonggang Liu, Renato Figueiredo, Dulcardo Clavijo, Yiqi Xu, and Ming Zhao, “Towards Simulation of Parallel File System Scheduling Algorithms with PFSsim,” 7th IEEE International Workshop on Storage Network Architecture and Parallel I/O (SNAPI, co-held with MSST’11), May 2011.
• Yiqi Xu, Lixi Wang, Dulcardo Arteaga, Ming Zhao, Yonggang Liu, and Renato Figueiredo, “Virtualization-based Storage Management for High-end Computing Systems,” 5th Petascale Data Storage Workshop (PDSW, co-held with SC’10), November 2010.
• Yiqi Xu, Lixi Wang, Dulcardo Arteaga, Ming Zhao, Yonggang Liu, and Renato Figueiredo, “Virtualization-based Storage Management for High-end Computing Systems,” (Poster), 5th Petascale Data Storage Workshop (PDSW, co-held with SC’10), November 2010.
• Yonggang Liu, Renato Figueiredo, Dulcardo Clavijo, Yiqi Xu, and Ming Zhao, “Towards Simulation of Parallel File System Scheduling Algorithms with PFSsim,” (Poster), 5th Petascale Data Storage Workshop (PDSW, co-held with SC’10), November 2010.
• Ming Zhao, Yiqi Xu, Dulcardo Clavijo, Renato Figueiredo, and Yonggang Liu, “QoS-driven Storage Management for High-end Computing Systems,” (Poster), 18th Annual Meeting of the IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), August 2010.
• Ming Zhao and Renato Figueiredo, “Virtualization-based Storage Management for High-end Computing Systems,” Technical Report 2010-05-01, School of Computing and Information Sciences, Florida International University, 2010.

Presentations

• Ming Zhao, “Towards Simulation of Parallel File System Scheduling Algorithms with PFSsim”, 7th IEEE International Workshop on Storage Network Architecture and Parallel I/O (co-held with MSST 2011), May 2011.
• Yiqi Xu, “QoS-driven Storage Management for High-end Computing Systems”, 5th Petascale Data Storage Workshop (co-held with Supercomputing 2010), November 2010.
• Ming Zhao, “QoS-driven Storage Management for High-end Computing Systems”, HEC FSIO Workshop, August 2010.
• Ming Zhao, “QoS-driven Storage Management for High-end Computing Systems”, HEC FSIO Workshop, August 2009.

Source Code

• GitHub (public)
• SVN (authorized users only)

Acknowledgement

This material is based upon work supported by the National Science Foundation under grants CCF-0937973 and CCF-0938045. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.