RAID

In this chapter you will learn how to create and manage xiRAID RAID objects.

Creating the RAID

You can create the RAID with the command

# xicli raid create <args> [optional_args]

For the argument descriptions, see the table below. For recommendations on configuring RAID settings, see the General Configuration Recommendations chapter.

Warning:

Creating xiRAID RAID over xiRAID RAID devices is not allowed. To pool a large number of drives into a single address area, use RAIDs 10, 50, 60, 70.

Minimum number of drives required to create a RAID:

of levels 5, 6, or 7 – at least 4 drives;
of level 10 – at least 2 drives (the number of drives must be even);
of level 0 – at least 1 drive;
of level 1 – at least 2 drives;
of levels 50, 60, or 70 – at least 8 drives (make sure the total drives number is multiple of the --group-size parameter value);
of level N+M – at least 8 drives.

Creating xiRAID RAIDs requires at least 1024 MiB of RAM.

Table 1. Arguments for the create subcommand
Required arguments
-n	--name	The name of the RAID.
-l	--level	The level of the RAID: `0`, `1`, `5`, `6`, `7`, `10`, `50`, `60`, `70`, or `nm`.
-d	--drives	The list of block devices (/dev/sd, /dev/mapper/mpath, /dev/nvme, /dev/dm-) separated by a space.
-gs	--group_size	Only for RAIDs 50, 60, or 70. The number of drives for one RAID group of level 5, 6, or 7.3 of the appropriate RAID 50, 60, or 70. Possible values are integers from `4` to `32`.
-sc	--synd_cnt	Only for RAIDs N+M. The number of syndromes M. Possible values are integers from `4` to `32`. Additional conditions: N+M ≤ 64 and M ≤ N.
Optional arguments
-bs	--block_size	RAID block size: `512` or `4096` bytes. The default: `4096`.
-inp	--init_prio	Except RAID 0. Initialization priority in %. Possible values are from `0` to `100` (maximum rate of initialization). The default: `100`.
-mwe	--merge_write_enabled	Except RAIDs 0, 1, 10. Enable (`1`) or disable (`0`) the Merge function for write operations. The default: `0`.
-mre	--merge_read_enabled	Except RAIDs 0, 1, 10. Enable (`1`) or disable (`0`) the Merge function for read operations. The default: `0`.
-ml	--memory_limit	RAM usage limit in MiB. Possible values: `0` and integers from `1024` to `1048576`. The `0` value sets unlimited RAM usage. The default: `0`.
-mm	--merge_max	Except RAIDs 0, 1, 10. Maximum wait time (in microseconds) for stripe accumulation for the Merge functions. Possible values: integers from `1` to `100000`. The default: `1000`.
-mw	--merge_wait	Except RAIDs 0, 1, 10. Wait time (in microseconds) between requests for the Merge functions. Possible values: integers from `1` to `100000`. The value must be less than the merge_max value. The default: `300`.
-rcp	--recon_prio	Except RAID 0. Reconstruction priority in %. Possible values are from `0` to `100` (maximum rate of reconstruction). The default: `100`.
-re	--resync_enabled	Except RAIDs 0, 1, 10. Enable (`1`) or disable (`0`) the Resync function. The default: `1`.
-rl	--request_limit	Number of simultaneous I/O requests on RAID. Possible values: from `0` (unlimited) to `4294967295`. The `0` value disables the restriction. The default: `0`.
-rsp	--restripe_prio	Restriping priority in %. Possible values are from `0` to `100` (maximum rate of restriping). The default: `100`.
-se	--sched_enabled	Enable (`1`) or disable (`0`) the scheduling function. The default: `0`.
-sp	--sparepool	Name of the spare pool to assign to the RAID.
-ss	--strip_size	Strip size in KiB. Possible values: `16`, `32`, `64`, `128`, or `256`. The default: `16`.

Example: Create the RAID 5 named ”media5” over 4 NVMe drives — ”nvme0n1”, ”nvme1n1”, ”nvme2n1”, ”nvme3n1”, with strip size equal to 64 KiB and enabled Merge function for write operations.

# xicli raid create -n media5 -l 5 -d /dev/nvme0n1 /dev/nvme1n1 
/dev/nvme2n1 /dev/nvme3n1 -ss 64 -mwe 1

Showing RAID State

You can view info about the RAID with the command

# xicli raid show [optional_args]

Table 2. Arguments for the show subcommand
Optional arguments
-n	--name	The name of the RAID. Without the argument, show info on all xiRAID RAIDs.
-o	--online	Only show RAIDs that are in the “online” state (RAIDs that were not unloaded by the raid unload command). The argument takes no value.
-u	--units	Dimension: `s` – sectors (1 sector=512 bytes); `k` – kilobytes; `m` – megabytes; `g` – gigabytes. The default: `g`.
-f	--format	Output format: `table`; `json`; `prettyjson` – human-readable json. The default: `table`.
-e	--extended	Show extended output. The argument takes no value.

Example: Show information on the ”media5” RAID:

# xicli raid show -n media5 -e

Table 3. Description of the show subcommand output
Row	Description
name	RAID name.
static	Static RAID parameters: size. level. synd_cnt – only for RAIDs N+M – number of syndromes. block_size – RAID block size. group_size – only for RAIDs 50, 60, and 70 – size of the corresponding RAID group. strip_size. sparepool – name of the assigned spare pool. active: True, if the RAID’s block device is in the system. False, if: The RAID was not loaded after reboot. The RAID is unloaded. config: True, if the RAID is in the configuration file. False, if the RAID is missing.
state	RAID state: online – the RAID is available and ready to work. initialized – initialization is finished. initing – the RAID is initializing. degraded – the RAID is available and ready for work but some drives are missing or failed. reconstructing – the RAID is reconstructing. offline – the RAID is unavailable. need_recon – the RAID needs reconstruction. need_init – the RAID needs initialization. read_only – the license has expired. The RAID is read-only. unrecovered – RAID can’t complete reconstruction because of unrecoverable sections. none – RAID was unloaded via the unload command or was not restored after reboot. restriping – RAID is restriping. need_resize – restriping was finished, the RAID size increase is available. need_restripe – restriping was stopped and not finished.
devices	The list of devices included in the RAID, and their current states: online – the drive is active. offline – the drive is missing or unavailable. reconstructing – the drive is in process of reconstruction. need_recon – the drive needs reconstruction.
health	To show, use the command with the -e parameter. Percent of valid drive data. When health is 100% – no reconstruction required.
wear	To show, use the command with the -e parameter. The wear percentage of the SSD or NVMe drive. When the drive reaches the 90% threshold, the system sends an error message to the mail. The S.M.A.R.T. values "Percentage used endurance indicator" and "Percentage Used" are used to check SSD and NVMe drives respectively.
serials	To show, use the command with the -e parameter. Serial numbers of drives in RAID.
params	To show, use the command with the -e parameter. Editable RAID parameters: init_prio – (except RAID 0) initialization priority: from 0% to 100%. init_depth – the queue depth of initialization requests. recon_prio – (except RAID 0) reconstruction priority: from 0% to 100%. recon_depth – the queue depth of reconstruction requests. memory_limit_mb – the value limited RAM usage, in megabytes. merge_write_enabled – is the function Merge enabled (1) or disabled (0) for write operations. merge_read_enabled – is the function Merge enabled (1) or disabled (0) for read operations. resync_enabled – is the function Resync enabled (1) or disabled (0). sched_enabled – is the function Scheduling enabled (1) or disabled (0). request_limit – number of simultaneous I/O requests on RAID (0 for no limit). restripe_prio – priority for restriping: from 0% to 100%. merge_wait_usecs – waiting time between requests when Merge is enabled. merge_max_usecs – maximum time to wait for stripe accumulation with Merge enabled.
info	Dynamic RAID values: init_progress – initialization progress: from 0% to 100%. recon_progress – reconstruction progress: from 0% to 100%. memory_usage_mb – amount of RAM usage; if memory_limit_mb = 0 (not limited), then memory_usage_mb is not displayed. restripe_progress – restriping progress: from 0% to 100%.

Deleting the RAID

Warning:

Warning! The result of the command is irreversible. Read the description carefully.

You can delete the RAID without possibility to restore the RAID and data on it with the command

# xicli raid destroy <arg> [optional_args]

Table 4. Arguments for the destroy subcommand
Mutually exclusive required arguments
-n	--name	The name of the RAID.
-a	--all	Delete all the xiRAID RAIDs. The argument takes no value.
Optional arguments
	--force	Force the command execution.
	--config_only	Remove RAID only from config.

Example: Deleting the RAID ”media5”:

# xicli raid destroy -n media5

Unloading the RAID

You can remove (unload) the RAID with possibility to restore the RAID and save data on it with the command

# xicli raid unload <arg>

Table 5. Arguments for the unload subcommand
Mutually exclusive required arguments
-n	--name	The name of the RAID.
-a	--all	Unload all available xiRAID RAIDs. The argument takes no value.

Example: Unloading the RAID ”media5”:

# xicli raid unload -n media5

To restore unloaded RAIDs, run:

# xicli raid restore {-n <raid_name>|-a}

RAID Reconstruction

Reconstruction of a RAID (except RAID 0) starts automatically after a drive has been replaced in the RAID. While a RAID is being reconstructed, the functions initialization and restriping are paused.

Tip:

To improve the system performance under the workload, try decreasing reconstruction priority by changing the corresponding RAID parameter.

To start the RAID reconstruction, run

# xicli raid recon start <arg>

Table 6. Argument for the recon start subcommand
Required argument
-n	--name	The name of the RAID.

To stop the RAID reconstruction, run

# xicli raid recon stop <arg>

Table 7. Argument for the recon stop subcommand
Required argument
-n	--name	The name of the RAID.

Example: Start RAID “media5” reconstruction:

# xicli raid recon start -n media5

RAID Initialization

Initialization will start automatically after a RAID (except RAID 0) is created.

Tip:

To improve the system performance under the load, try decreasing initialization priority by changing the corresponding RAID parameter.

Tip:

The random write performance is higher on an initialized RAID.

To start or continue the RAID initialization, run

# xicli raid init start <arg>

Table 8. Argument for the init start subcommand
Required argument
-n	--name	The name of the RAID.

To stop the RAID initialization, run

# xicli raid init stop <arg>

Table 9. Argument for the init stop subcommand
Required argument
-n	--name	The name of the RAID.

Example: Start initialization of the RAID ”media5”:

# xicli raid init start -n media5

Increasing Size and Changing Level of RAID

In this chapter, you will learn about the following RAID operations:

Changing the RAID level with the addition of new disks.
Increasing the size of a RAID by adding new disks.
Increasing the size of a RAID by replacing its disks with larger disks (vertical scaling RAID operation).

RAID operations that add new disks to a RAID consist of two steps: restriping (raid restripe command), which selects the disks to be added and the RAID level, and resizing (raid resize subcommand), which applies changes to the RAID (which is in need_resize status).

For an operation to increase the size of the RAID through larger disks, only a resize (raid resize command) is required.

Requirements and Specifics of Restriping and Resizing

Requirements and specifics:

Except RAID N+M.
Only one RAID can be restriped at a time.
To improve the performance of your system under workload, try to change the priority of restriping by changing the corresponding RAID parameter.
RAID state must not be one of the following:
- offline;
- need_restripe;
- restriping;
- degraded.

Attention:

The RAID level can only be changed if the number of added drives is sufficient to fit all the data stored prior to the level change. The number of data drives in a group must not decrease. Note that for RAID levels 50, 60, and 70, the total number of drives must be a multiple of the group size. Please keep in mind that RAID 50 includes one syndrome drive per group, while RAID 60 has two and RAID 70 has three. These drives do not store information but are necessary for data recovery in case of loss.

Example: Adding 1 drive to a RAID 60 configuration with 14 disks and 2 groups is not feasible. The only possible configuration for 15 drives is 3 groups of 5 drives, with only 3 of them containing data. Performing this operation would result in a new RAID with fewer data drives, hence it cannot be done. However, it is possible to add 2 drives to create a new RAID configuration consisting of 2 groups, each containing 8 drives (2 syndrome and 6 data drives). This will increase the data storage space.

Table 10. The available options for RAID level changes and the minimum required number of drives
Current level	New level	Requirements	Minimal number of drives you should add
RAID 0	RAID 0		1
	RAID 1	RAID 0 must contain only 1 drive.	1
	RAID 10	RAID 0 contains only 1 drive.	3
	RAID 10	RAID 0 contains more than 1 drive	The number of drives to be added must be equal to the number of drives in the RAID 0.
RAID 1	RAID 1		2
	RAID 10		2
	RAID 5		2
RAID 10	RAID 10		4
	RAID 5		1
	RAID 50	The number of RAID 50 disks must be a multiple of its group size.	1
RAID 5	RAID 5		1
	RAID 6		1
	RAID 10	The number of RAID 10 disks must be an even number. The number of information disks in a group must not decrease.	1
	RAID 50	The number of disks in RAID 50 must be at least 8. The number of disks in RAID 50 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 6	RAID 6		1
	RAID 7.3		1
	RAID 60	The number of disks in RAID 60 must be at least 8. The number of disks in RAID 60 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 7.3	RAID 7.3		1
RAID 7.3	RAID 70	The number of disks in RAID 70 must be at least 8. The number of disks in RAID 70 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 50	RAID 50	The number of disks in RAID 50 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 50	RAID 60	The number of disks in RAID 60 must be at least 8. The number of disks in RAID 60 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 60	RAID 60	The number of disks in RAID 60 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 60	RAID 70	The number of disks in RAID 70 must be at least 12. The number of disks in RAID 70 must be a multiple of its group size. The number of information disks in a group must not decrease.	1
RAID 70	RAID 70	The number of disks in RAID 70 must be a multiple of its group size. The number of information disks in a group must not decrease.	1

raid restripe

The chapter describes the available subcommands for the restripe operation.

Restriping refers to any change in RAID configuration, such as the position of checksums or data drives, with the aim of changing the RAID level or size.

To start RAID restriping, run

# xicli raid restripe start <args>

Table 11. Arguments for the restripe start subcommand
Required arguments
-n	--name	The name of the RAID.
-l	--level	The new level for the RAID. If you are only increasing the RAID size, enter the current RAID level for this argument.
-gs	--group_size	Only for RAIDs 50, 60, and 70. The new group size for the RAID. Possible values: integers from `4` to `32`.
-d	--drives	The list of block devices (/dev/sd, /dev/mapper/mpath, /dev/nvme, /dev/dm-) separated by a space to add to the RAID.

To pause RAID restriping, run

# xicli raid restripe stop <arg>

Table 12. Argument for the restripe stop subcommand
Required argument
-n	--name	The name of the RAID.

To continue RAID restriping, run

# xicli raid restripe continue <arg>

Table 13. Argument for the restripe continue subcommand
Required argument
-n	--name	The name of the RAID.

raid resize

The chapter describes the command for the resizing operation.

Following the restriping process, the RAID goes into the 'need resize' state. We recommend starting the resizing operation in order to restore optimal system speed.

Tip:

We recommend that you run the raid resize command when there is no or minimal workload on the RAID.

# xicli raid resize <arg>

Table 14. Argument for the resize subcommand
Required argument
-n	--name	The name of the RAID.

Examples of Restriping and Resizing

Example: Restripe the RAID “media5” by adding a new drive /dev/sdi without changing the RAID level (increasing the RAID size):

# xicli raid restripe start -n media5 -l 5 -d /dev/sdi

Example: Increasing the RAID "media5" size by replacing the drives (3 GB each) with drives of larger size (5 GB each: /dev/sde, /dev/sdf, /dev/sdg, /dev/sdh):

In this example, the RAID size will be increased from 9 GB to 15 GB.

Perform a RAID replacement and reconstruction for each drive in a turn, waiting for reconstruction to complete:

Change the first drive:
```
# xicli raid replace -n media5 -no 0 -d /dev/sde
```
Reconstruction starts automatically after the drive replacement. Wait for reconstruction to complete.
Change the second drive:
```
# xicli raid replace -n media5 -no 1 -d /dev/sdf
```
Reconstruction starts automatically after the drive replacement. Wait for reconstruction to complete.
Change the third drive:
```
# xicli raid replace -n media5 -no 2 -d /dev/sdg
```
Reconstruction starts automatically after the drive replacement. Wait for reconstruction to complete.
Change the fourth drive:
```
# xicli raid replace -n media5 -no 3 -d /dev/sdh
```
Reconstruction starts automatically after the drive replacement. Wait for reconstruction to complete.
Run resize:
```
# xicli raid resize -n media5
```
RAID size is increased to 15 GB, RAID is in the need_init state.

Example: Restriping of the “media5” RAID with adding new drives /dev/sdf /dev/sdg /dev/sdh and RAID level changing from 5 to 6:

# xicli raid restripe start -n media5 -l 6 -d /dev/sdf /dev/sdg 
/dev/sdh

After restriping is finished, the RAID state is need_resize until you run

# xicli raid resize -n <raid_name>

Changing RAID Parameters

To improve the system performance under the workload, try decreasing initialization, reconstruction, or restriping priorities.

See recommendations on configuring RAID parameters in the chapter RAID and System Setup Recommendations.

To change the RAID dynamic parameters, run:

# xicli raid modify <arg> [optional_args]

Table 15. Arguments for the modify subcommand
Required argument
-n	--name	The name of the RAID.
Optional arguments
-inp	--init_prio	Except RAID 0. Initialization priority in %. Possible values are from `0` to `100` (maximum rate of initialization). The default: `100`.
-mwe	--merge_write_enabled	Except RAIDs 0, 1, 10. Enable (`1`) or disable (`0`) the Merge function for write operations. The default: `0`.
-mre	--merge_read_enabled	Except RAIDs 0, 1, 10. Enable (`1`) or disable (`0`) the Merge function for read operations. The default: `0`.
-ml	--memory_limit	RAM usage limit in MiB. Possible values: `0` and integers from `1024` to `1048576`. The `0` value sets unlimited RAM usage. The default: `0` (unlimited).
-mm	--merge_max	Except RAIDs 0, 1, 10. Maximum wait time (in microseconds) for stripe accumulation for the Merge functions. Possible values: integers from `1` to `100000`. The default: `1000`.
-mw	--merge_wait	Except RAIDs 0, 1, 10. Wait time (in microseconds) between requests for the Merge functions. Possible values: integers from `1` to `100000`. The value must be less than the merge_max value. The default: `300`.
-rcp	--recon_prio	Except RAID 0. Reconstruction priority in %. Possible values: from `0` to `100` (maximum rate of reconstruction). The default: `100`.
-re	--resync_enabled	Except RAIDs 0, 1, 10. Enable (`1`) or disable (`0`) the resync function. The default: `1`.
-rl	--request_limit	Number of simultaneous I/O requests on RAID. Possible values: integers from `0` to `4294967295`. The `0` value disables the restriction. The default: `0`.
-rsp	--restripe_prio	Restriping priority in %. Possible values are from `0` to `100` (maximum rate of restriping). The default: `100`.
-se	--sched_enabled	Enable (`1`) or disable (`0`) the scheduling function. The default: `0`.
-sp	--sparepool	Name of the spare pool to assign to the RAID. The `null` value removes the spare pool from the RAID. Spare pool can not be assigned to RAID 0.
	--force_online	Change RAID state to online if the RAID has unrecoverable sections. I/O operations on unrecoverable sections may lead to data corruption. The argument takes no value.
	--force_resync	Force RAID re-initialization. The argument takes no value.

Example: Setting reconstruction priority for the RAID ”media5” equal to 50%:

# xicli raid modify -n media5 -rcp 50