June 16-20, 2013

Leipzig, Germany

Presentation Details

Name: Scalable Massively Parallel I/O to Task-Local Files
Time: Monday, June 17, 2013
5:00 PM - 5:20 PM
Room:   Hall 2
CCL - Congress Center Leipzig
Speakers:   Wolfgang Frings, JSC
Abstract:   Parallel applications often store data in multiple task-local files, for example, to create checkpoints, to circumvent memory limitations, or to record performance data. When operating at very large processor configurations, such applications often experience scalability limitations when the simultaneous creation of thousands of files causes metadata-server contention or simply when large file counts complicate file management or operations on those files even destabilize the file system. This talk will cover the design principles of SIONlib, a parallel I/O library that addresses this problem by transparently mapping a large number of task-local files onto a small number of physical files via internal metadata handling and block alignment to ensure high performance. Finally, challenges that will arise on upcoming exascale systems and possible solutions based on SIONlib will be discussed.  
Program may be subject to changes.