BeeGFS

BeeGFS is a hardware-independent POSIX parallel file system developed with a strong focus on performance and designed for ease of use, simple installation and easy management. It is designed for all performance-oriented environments, including HPC, AI, deep learning, life sciences, oil, gas, media, and entertainment.

One of the most fundamental concepts of BeeGFS is the strict avoidance of architectural bottlenecks or locking situations in the cluster through the user space architecture. BeeGFS stripes the file content on multiple storage nodes, plus it distributes file system metadata across multiple metadata servers, which benefits both small and large systems as well as metadata-intensive applications.

Western Digital has partnered with BeeGFS to provide a verified, highly scalable shared network file system with striped file contents, high-performance computing and mirroring (HA) for on-premises and cloud systems. xiRAID is used as a high-performance software RAID that provides continued availability and fault tolerance for the storage.

Read the white paper by Western Digital, ThinkParQ & Xinnor:

Solution brief (pdf, 0.6 MB)

White paper (pdf, 1.7 MB)

Challenge

HPC environments have long relied on magnetic disks for storage. However, modern workloads in HPC, such as ML training and big data analytics, demand improved metadata performance. To meet these needs, HPC clusters are evolving their compute, storage, and network components. Distributed parallel file systems now support compute and NVMe-oF storage, utilizing high-performance NVMe SSDs effectively. BeeGFS is one such system, combining storage servers to create a scalable shared network file system that easily handles high throughput for numerous clients.

To meet the increasing need for high-performance parallel file systems in HPC clusters, we developed this solution. It deploys a distributed parallel file system on a composable infrastructure, effectively addressing this demand. Western Digital’s validated design for HPC BeeGFS High-Capacity Storage solution with Xinnor xiRAID provides high throughput, scalability, and a well-documented performance profile for parallel file system storage.

Solution

For a fast and reliable BeeGFS solution, xiRAID is used together with OpenFlex Data24 NVMe-oF Storage Platform by Western Digital. This vertically integrated storage architecture utilizes an Open Composable Infrastructure approach, employing disaggregated data storage with NVMe-over-Fabrics technology.

BeeGFS on the OpenFlex Data24 Storage with xiRAID provides a well-designed and easily manageable solution for HPC that is fully supported. It leverages the benefits of an open composable infrastructure environment and enables disaggregated compute on the NVMe-oF storage platform.

A study by Western Digital demonstrates the superior performance and ease of deployment achieved with BeeGFS, xiRAID, and OpenFlex Data24. It shows that using xiRAID user will be able to achieve significant performance with data protection and high availability. This software technology accelerates storage hardware systems to new levels of performance.

By adopting this solution, enterprises and service providers can minimize installation costs, implement a high-performance file system, and accelerate time to market for new application services. Leveraging the latest NVMe-oF and CDI technologies, organizations can swiftly deploy a proven, self-service, composable infrastructure solution, enabling a more flexible and cost-efficient model.

IOR Test Results

The joint solution demonstrates that by replacing buddy mirroring with more cost and resources efficient xiRAID RAID5, users will gain extra capacity and performance.

IOR Test
Results		 BeeGFS performance on Non-RAID volumes (without Buddy Mirror, not protected against drive failure) BeeGFS performance on Non-RAID volumes with Buddy Mirror BeeGFS performance on Xinnor RAID5 volumes (without Buddy Mirror)
Read 45 GBps 45 GBps 42 GBps
Write 30 GBps 7.7 GBps 24 GBps

Advantages

xiRAID ensures stable performance levels, making it ideal for uninterrupted business operations. Its fast coding and decoding ability, along with fast RAID rebuild, protect against extensive system downtime and minimize workflow disruptions. This is particularly valuable for data-intensive systems and high-density storage infrastructures, where a single drive failure can trigger checksum recalculations for large amounts of data.

By combining BeeGFS and xiRAID with Western Digital OpenFlex Data24, organizations gain several benefits from these parallel file systems:

  • High Performance
  • High Availability
  • Scalability
  • Robustness
  • Easy deployment and integration with existing infrastructure
  • Streamlined data management
  • Optimized for concurrent access
  • BeeGFS software with enterprise-grade features