Databases

Databases play a crucial role in storing, organizing, and managing large amounts of data efficiently. As databases grow in size and complexity, ensuring data availability and protection becomes increasingly important.

RAID offers a method of combining multiple physical disks into a single logical unit, providing increased data redundancy and improved performance. It is a valuable tool in maintaining the integrity and reliability of databases, making RAID storage an indispensable solution for organizations dealing with critical data.

Learn more about how xiRAID has enhanced database performance for two of our partners: Kx and ScaleFlux.

Kx challenge

Kx provides a high-speed streaming analytics platform with kdb+, the fastest time-series database. It optimizes data ingestion, analysis, and storage for large volumes of historical and streaming data. The platform supports data capture, processing, enrichment, analytics, and interactive visualizations. Solutions built on Kx offer redundancy, fault tolerance, query capabilities, filtering, alerting, reporting, and visualizations. Its columnar design and in-memory capabilities provide speed and efficiency compared to relational databases.

Native time-series support enhances speed and performance for streaming and historical data operations. High performance storage subsystem is a must to support very large data volumes and ensure low latency query workloads.

Solution

Kx customers utilize server infrastructure with NVMe and SSD storage to ensure fast data processing and support analytics. xiRAID is specialized software for SSD and NVMe drives, maximizing their performance. Kx uses xiRAID in its R&D labs for testing IoT workloads and large data volumes. xiRAID has boosted Kx's write performance by 88%, sequential read performance by 39%, and reduced latency, enabling more data processing with current infrastructure.

Test Results

  Before (VROC) After
Total Write Rate (async) MiB/s 3,420 6,440
Streaming Read (mapped) Rate MiB/s 14,100 19,565
Streaming ReRead (mapped) Rate MiB/s 47,905 72,461
Random Read 1 MB (mapped) Rate MiB/s 10,892 7,386
Random Read 64K (mapped) Rate MiB/s 6,992 3,542
Random Read 1 MB (unmapped) Rate MiB/s 8,914 6,485
Random Read 64K (unmapped) Rate MiB/s 5,735 3,050

Kx stores data in a columnar database, with most customer workloads involving sequential reading and writing of data. This enables Kx users to benefit from significantly higher performance in reading and writing data than available with alternative storage platforms.

Test setup

  1. 24 NVMe drives 2 TB each, 48 TB RAW24 NVMe drives 2 TB each, 48 TB RAW
  2. RAID 50 configuration with 6 RAID 5 groups
  3. Nano benchmark running 20 threads for read and write operations

Nano benchmark calculates basic raw I/O capability of non-volatile storage, as measured from kdb+ perspective.

ScaleFlux challenge

ScaleFlux develops innovative storage solutions for data centers and cloud computing, with their flagship product being the CSD (Computational Storage Drive) series. The CSD series combines high-performance NVMe SSD with Hardware Compute Engines to offer a flexible, efficient storage solution. It boosts infrastructure efficiency, increases storage density, enhances compute performance, and reduces data storage costs. It combines NVMe SSD features with Hardware Compute Engines, providing a user-friendly solution that unlocks new possibilities for businesses.

ScaleFlux is evaluating the feasibility of using xiRAID in latency-sensitive distributed database applications.

Solution

Assessing a RAID solution's performance in degraded and reconstructing states is crucial. In the degraded state, fewer drives are available for IO requests, and intra-RAID IO adds traffic as data on good disks is used for reconstruction on new disks. xiRAID allows controlled array reconstruction for predictability and efficiency. ScaleFlux CSD 3310 NVMe SSDs offer inline compression to minimize the impact of intra-RAID reconstruction IO. The combination of these technologies creates a top-tier software RAID solution for high-performance workloads in all RAID operating states.

Test Results

During the reconstruction process, there is a minor decrease in CPU utilization, but the overall read and write throughput remains steady at 5.4GB/s and 1.7GB/s, respectively.

ScaleFlux - Test Results - 1

For a more comprehensive analysis of the reconstruction phase, we measured the read and write throughput of both a regular drive and the drive undergoing reconstruction.

ScaleFlux - Test Results - 2

The regular drive maintains a consistent write throughput while supporting a higher read throughput by contributing data to the reconstructing drive. As the reconstruction progresses, the amount of data read from the regular drive decreases, and by the end of the phase, both drives contribute equally to the overall workload.

Download the application note to learn more about the results:

Download

Test setup

CPU: dual-socket Xeon Gold 6342 CPU

Cores: 48 physical cores

DRAM: 512GB

SSD: five 7.68TB ScaleFlux CSD 3310 PCIe Gen 4 SSDs

OS: Ubuntu 22

Kernel version: 5.15.0-70

Certification tool: Aerospike