Chapter 2. Block Storage and Volumes | Red Hat Product Documentation (2024)

Important

Active-active mode is supported only in distributed compute node (DCN) architecture at edge sites.

• The encryption of a volume. For more information, see Configure Volume Type Encryption.
• The back end that a volume uses. For more information, see Specify Back End for Volume Creation and Migrate a Volume.
• Quality-of-Service (QoS) Specs

1

Replace BACKEND with the name of the target back end (specifically, its volume_backend_name value).

2

By default, the Image-Volume cache size is only limited by the back end. Change MAXSIZE to a number in GB.

3

You can also set a maximum number of images using MAXNUMBER.

• QOS Specs are used to apply performance settings, which include limiting read/write operations to disks. Available and supported performance settings vary per storage driver.

  To determine which QOS Specs are supported by your back end, consult the documentation of your back end device’s volume driver.

• Volume types are directly applied to volumes, whereas QOS Specs are not. Rather, QOS Specs are associated to volume types. During volume creation, specifying a volume type also applies the performance settings mapped to the volume type’s associated QOS Specs.

AvailabilityZoneFilter

Filters out all back ends that do not meet the availability zone requirements of the requested volume.

CapacityFilter

Selects only back ends with enough space to accommodate the volume.

CapabilitiesFilter

Selects only back ends that can support any specified settings in the volume.

InstanceLocality

Configures clusters to use volumes local to the same node.

1

You can also add the ControllerExtraConfig: hook and its nested sections to the parameter_defaults: section of an existing environment file.

1. Add the following parameters to the environment file to create two availability zones:

   parameter_defaults: CinderXXXAvailabilityZone: zone1 CinderYYYAvailabilityZone: zone2

   Replace XXX and YYY with supported back-end values, such as:

   CinderISCSIAvailabilityZoneCinderNfsAvailabilityZoneCinderRbdAvailabilityZone

   Note

   Search the /usr/share/openstack-tripleo-heat-templates/deployment/cinder/ directory for the heat template associated with your back end for the correct back-end value.

   The following example deploys two back ends where rbd is zone 1 and iSCSI is zone 2:

   parameter_defaults: CinderRbdAvailabilityZone: zone1 CinderISCSIAvailabilityZone: zone2

2. Deploy the overcloud and include the updated environment file.

Procedure

1. In the dashboard, select Project > Compute > Volumes.

2. Click Create Volume, and edit the following fields:

   Field	Description
   Volume name	Name of the volume.
   Description	Optional, short description of the volume.
   Type	Optional volume type (see Section2.2.2, “Group Volume Settings with Volume Types”). If you have multiple Block Storage back ends, you can use this to select a specific back end. See Section2.3.2, “Specify back end for volume creation”.
   Size (GB)	Volume size (in gigabytes). If you want to create an encrypted volume from an unencrypted image, you must ensure that the volume size is at least 1GB larger than the image size so that the encryption data does not truncate the volume data.
   Availability Zone	Availability zones (logical server groups), along with host aggregates, are a common method for segregating resources within OpenStack. Availability zones are defined during installation. For more information about availability zones and host aggregates, see Creating and managing host aggregates.

3. Specify a Volume Source:

   Source	Description
   No source, empty volume	The volume is empty and does not contain a file system or partition table.
   Snapshot	Use an existing snapshot as a volume source. If you select this option, a new Use snapshot as a source list opens; you can then choose a snapshot from the list. If you want to create a new volume from a snapshot of an encrypted volume, you must ensure that the new volume is at least 1GB larger than the old volume. For more information about volume snapshots, see Section2.3.10, “Create, use, or delete volume snapshots”.
   Image	Use an existing image as a volume source. If you select this option, a new Use snapshot as a source list opens; you can then choose an image from the list.
   Volume	Use an existing volume as a volume source. If you select this option, a new Use snapshot as a source list opens; you can then choose a volume from the list.

4. Click Create Volume. After the volume is created, its name appears in the Volumes table.

1. In the dashboard, select Project > Compute > Volumes.
2. Select the volume’s Edit Volume button.
3. Edit the volume name or description as required.
4. Click Edit Volume to save your changes.

1. List the volumes to retrieve the ID of the volume you want to extend:

   $ cinder list

2. To resize the volume, run the following commands to specify the correct API microversion, then pass the volume ID and the new size (a value greater than the old size) as parameters:

   $ OS_VOLUME_API_VERSION=<API microversion>$ cinder extend <volume ID> <size>

   Replace <API_microversion>, <volume_ID>, and <size> with appropriate values, for example:

   $ OS_VOLUME_API_VERSION=3.42$ cinder extend 573e024d-5235-49ce-8332-be1576d323f8 10

1. In the dashboard, select Project > Compute > Volumes.
2. In the Volumes table, select the volume to delete.
3. Click Delete Volumes.

Limitations of multi-attach volumes

• The Block Storage (cinder) back end must support multi-attach volumes. For information about supported back ends, contact Red Hat Support.

• Your Block Storage (cinder) driver must support multi-attach volumes. Contact Red Hat support to verify that multi-attach is supported for your vendor plugin. For more information about the certification for your vendor plugin, see the following locations:

  • Component, Plug-In, and Driver Support in Red Hat OpenStack Platform
  • Red Hat Ecosystem Catalog
• Read-only multi-attach volumes are not supported.
• Live migration of multi-attach volumes is not available.
• Encryption of multi-attach volumes is not supported.
• Multi-attach volumes are not supported by the Bare Metal Provisioning service (ironic) virt driver. Multi-attach volumes are supported only by the libvirt virt driver.
• You cannot retype an attached volume from a multi-attach type to a non-multi-attach type, and you cannot retype a non-multi-attach type to a multi-attach type.
• You cannot use multi-attach volumes that have multiple read write attachments as the source or destination volume during an attached volume migration.
• You cannot attach multi-attach volumes to shelved offloaded instances.

1. In the dashboard, select Project > Compute > Volumes.
2. Select the Create Snapshot action of the target volume.
3. Provide a Snapshot Name for the snapshot and click Create a Volume Snapshot. The Volume Snapshots tab displays all snapshots.

Warning

The ability to recover the most recent snapshot of a volume is supported but is driver dependent. The correct implementation of this feature is driver assisted. For more information about support for this feature, contact your driver vendor.

Limitations

• There might be limitations to using the revert-to-snapshot feature with multi-attach volumes. Check whether such limitations apply before you use this feature.
• You cannot revert a volume that you resize (extend) after you take a snapshot.
• You cannot use the revert-to-snapshot feature on an attached or in-use volume.

Prerequisites

• Block Storage (cinder) API microversion 3.40 or later.
• You must have created at least one snapshot for the volume.

Procedure

1. Log in to the undercloud as the stack user.

2. Source the overcloudrc file:

   [stack@undercloud ~] $ source overcloudrc

3. Detach your volume:

   $ nova volume-detach <instance_id> <vol_id>

   Replace <instance_id> and <vol_id> with the IDs of the instance and volume that you want to revert.

4. Locate the ID or name of the snapshot that you want to revert. You can only revert the latest snapshot.

   $ cinder snapshot-list

5. Revert the snapshot:

   $ cinder --os-volume-api-version=3.40 revert-to-snapshot <snapshot_id or snapshot_name>

   Replace <snapshot_id or snapshot_name> with the ID or the name of the snapshot.

6. Optional: You can use the cinder snapshot-list command to check that the volume you are reverting is in a reverting state.

   $ cinder snapshot-list

7. Reattach the volume:

   $ nova volume-attach <instance_id> <vol_id>

   Replace <instance_id> and <vol_id> with the IDs of the instance and volume that you reverted.

1. In the dashboard, select Project > Compute > Volumes.
2. Select the target volume’s Upload to Image action.
3. Provide an Image Name for the volume and select a Disk Format from the list.
4. Click Upload.

• To retire storage systems that are no longer supported.
• To change the storage class or tier of a volume.
• To change the availability zone of a volume.

• Retype: The default policy allows volume owners and administrators to retype a volume. The retype operation is the most common way to move volumes between back ends.
• Migrate: The default policy allows only administrators to migrate a volume. Volume migration is reserved for specific use cases, because it is restrictive and requires a clear understanding about how deployments work. For more information, see Migrate a volume.

Restrictions

Red Hat supports moving volumes between back ends within and across availability zones (AZs), but with the following restrictions:

• Volumes must have either available or in-use status to move.
• Support for in-use volumes is driver dependent.
• Volumes cannot have snapshots.
• Volumes cannot belong to a group or consistency group.

Restrictions

• In-use multi-attach volumes cannot be moved while they are attached to more than one nova instance.
• Non block devices are not supported, which limits storage protocols on the target back end to be iSCSI, Fibre Channel (FC), or RBD.

• To move the volume to a different back end.
• To change the storage class or tier of a volume.
• To enable or disable features such as replication.

• The new volume type has a different volume_backend_name defined.
• The volume_backend_name of the current volume type is undefined, and the volume is stored in a different back end than the one specified by the volume_backend_name of the new volume type.

Restrictions

• You cannot retype all volumes. For more information about moving volumes between back ends, see Section2.3.13, “Moving volumes between back ends”.
• You cannot retype unencrypted volumes to be encrypted volume types, but you can retype encrypted volumes to be unencrypted.
• Users with no administrative privileges can only retype volumes that they own.

• LVM appears to see volume groups from guests
• LVM reports duplicate volume group names
• Volume detachments fail because LVM is accessing the storage
• Guests fail to boot due to problems with LVM
• The LVM on the guest machine is in a partial state due to a missing disk that actually exists
• Block Storage service (cinder) actions fail on devices that have LVM
• Block Storage service (cinder) snapshots fail to remove correctly
• Errors during live migration: /etc/multipath.conf does not exist

Prerequisites

• A successful undercloud installation. For more information, see Installing the undercloud.

Procedure

1. Log in to the undercloud host as the stack user.

2. Source the undercloud credentials file:

   $ source ~/stackrc

3. Create a new environment file, or modify an existing environment file. In this example, create a new file lvm2-filtering.yaml:

   $ touch ~/lvm2-filtering.yaml

4. Include the following parameter in the environment file:

   parameter_defaults: LVMFilterEnabled: true

   You can further customize the implementation of the LVM2 filter. For example, to enable filtering only on Compute nodes, use the following configuration:

   parameter_defaults: ComputeParameters: LVMFilterEnabled: true

   These parameters also support regular expression. To enable filtering only on Compute nodes, and ignore all devices that start with /dev/sd, use the following configuration:

   parameter_defaults: ComputeParameters: LVMFilterEnabled: true LVMFilterDenylist: - /dev/sd.*

5. Run the openstack overcloud deploy command and include the environment file that contains the LVM2 filtering configuration, as well as any other environment files that are relevant to your overcloud deployment:

   $ openstack overcloud deploy --templates \<environment-files> \-e lvm2-filtering.yaml

Restrictions

• The volume cannot be replicated.
• The destination back end must be different from the current back end of the volume.

• The existing volume type must be valid for the new back end, which means the following must be true:

  • Volume type must not have the backend_volume_name defined in its extra specs, or both Block Storage back ends must be configured with the same backend_volume_name.
  • Both back ends must support the same features configured in the volume type, such as support for thin provisioning, support for thick provisioning, or other feature configurations.

Prerequisites

• An unencrypted volume that you want to encrypt.

Procedure

1. If the cinder-backup service is available, back up the current unencrypted volume:

   $ cinder backup-create <unencrypted_volume>

   • Replace <unencrypted_volume> with the name or ID of the unencrypted volume.

2. Create a new encrypted volume:

   $ cinder create <encrypted_volume_size> --volume-type <encrypted_volume_type>

   • Replace <encrypted_volume_size> with the size of the new volume in GB. This value must be larger than the size of the unencrypted volume by 1GB to accommodate the encryption metadata.
   • Replace <encrypted_volume_type> with the encryption type that you require.

3. Restore the backup of the unencrypted volume to the new encrypted volume:

   $ cinder backup-restore <backup> --volume <encrypted_volume>

   • Replace <backup> with the name or ID of the unencrypted volume backup.
   • Replace <encrypted_volume> with the ID of the new encrypted volume.

1. Create an Image service image of the unencrypted volume:

   $ cinder upload-to-image <unencrypted_volume> <new_image>

   • Replace <unencrypted_volume> with the name or ID of the unencrypted volume.
   • Replace <new_image> with a name for the new image.

2. Create a new volume from the image that is 1GB larger than the image:

   $ cinder volume create --size <size> --volume-type luks --image <new_image> <encrypted_volume_name>

   • Replace <size> with the size of the new volume. This value must be 1GB larger than the size of the old unencrypted volume.
   • Replace <new_image> with the name of the image that you created from the unencrypted volume.
   • Replace <encrypted_volume_name> with a name for the new encrypted volume.

Prerequisites

The overcloud Controller and Compute nodes must have access to the Red Hat Enterprise Linux server repository. For more information, see Downloading the base cloud image in the Director Installation and Usage guide.

Procedure

1. Configure the overcloud.

   Note

   For more information, see Configuring a basic overcloud with CLI tools in the Director Installation and Usage guide.

2. Update the heat template to enable multipath:

   parameter_defaults: NovaLibvirtVolumeUseMultipath: true NovaComputeOptVolumes: - /etc/multipath.conf:/etc/multipath.conf:ro - /etc/multipath/:/etc/multipath/:rw CinderVolumeOptVolumes: - /etc/multipath.conf:/etc/multipath.conf:ro - /etc/multipath/:/etc/multipath/:rw

3. Optional: If you are using Block Storage (cinder) as an Image service (glance) back end, you must also complete the following steps:

   1. Add the following GlanceApiOptVolumes configuration to the heat template:

      parameter_defaults: GlanceApiOptVolumes: - /etc/multipath.conf:/etc/multipath.conf:ro - /etc/multipath/:/etc/multipath/:rw

   2. Set the ControllerExtraConfig parameter in the following way:

      parameter_defaults: ControllerExtraConfig: glance::config::api_config: default_backend/cinder_use_multipath: value: true

      Note

      Ensure that both default_backend and the GlanceBackendID heat template default value match.

4. For every configured back end, set use_multipath_for_image_xfer to true:

   parameter_defaults: ExtraConfig: cinder::config::cinder_config: <backend>/use_multipath_for_image_xfer: value: true

5. Deploy the overcloud:

   $ openstack overcloud deploy

   Note

   For information about creating the overcloud using overcloud parameters, see Creating the Overcloud with the CLI Tools in the Director Installation and Usage guide.

6. Before containers are running, install multipath on all Controller and Compute nodes:

   $ sudo dnf install -y device-mapper-multipath

   Note

   Director provides a set of hooks to support custom configuration for specific node roles after the first boot completes and before the core configuration begins. For more information about custom overcloud configuration, see Pre-Configuration: Customizing Specific Overcloud Roles in the Advanced Overcloud Customization guide.

7. Configure the multipath daemon on all Controller and Compute nodes:

   $ mpathconf --enable --with_multipathd y --user_friendly_names n --find_multipaths y

   Note

   The example code creates a basic multipath configuration that works for most environments. However, check with your storage vendor for recommendations, because some vendors have optimized configurations that are specific to their hardware. For more information about multipath, see the Configuring device mapper multipath guide.

8. Run the following command on all Controller and Compute nodes to prevent partition creation:

   $ sed -i "s/^defaults {/defaults {\n\tskip_kpartx yes\n\trecheck_wwid yes/" /etc/multipath.conf

   Note

   Setting skip_kpartx to yes prevents kpartx on the Compute node from automatically creating partitions on the device, which prevents unnecessary device mapper entries. For more information about configuration attributes, see Modifying the DM-Multipath configuration file in the Configuring device mapper multipath guide.

9. Start the multipath daemon on all Controller and Compute nodes:

   $ systemctl enable --now multipathd

Prerequisites

The overcloud Controller and Compute nodes must have access to the Red Hat Enterprise Linux server repository. For more information, see Downloading the base cloud image in the Director Installation and Usage guide.

Procedure

1. Verify that multipath is installed on all Controller and Compute nodes:

   $ rpm -qa | grep device-mapper-multipathdevice-mapper-multipath-0.4.9-127.el8.x86_64device-mapper-multipath-libs-0.4.9-127.el8.x86_64

   If multipath is not installed, install it on all Controller and Compute nodes:

   $ sudo dnf install -y device-mapper-multipath

2. Configure the multipath daemon on all Controller and Compute nodes:

   $ mpathconf --enable --with_multipathd y --user_friendly_names n --find_multipaths y

   Note

   The example code creates a basic multipath configuration that works for most environments. However, check with your storage vendor for recommendations, because some vendors have optimized configurations specific to their hardware. For more information about multipath, see the Configuring device mapper multipath guide.

3. Run the following command on all Controller and Compute nodes to prevent partition creation:

   $ sed -i "s/^defaults {/defaults {\n\tskip_kpartx yes\n\trecheck_wwid yes/" /etc/multipath.conf

   Note

   Setting skip_kpartx to yes prevents kpartx on the Compute node from automatically creating partitions on the device, which prevents unnecessary device mapper entries. For more information about configuration attributes, see Modifying the DM-Multipath configuration file in the Configuring device mapper multipath guide.

4. Start the multipath daemon on all Controller and Compute nodes:

   $ systemctl enable --now multipathd

5. Update the heat template to enable multipath:

   parameter_defaults: NovaLibvirtVolumeUseMultipath: true NovaComputeOptVolumes: - /etc/multipath.conf:/etc/multipath.conf:ro - /etc/multipath/:/etc/multipath/:rw CinderVolumeOptVolumes: - /etc/multipath.conf:/etc/multipath.conf:ro - /etc/multipath/:/etc/multipath/:rw

6. Optional: If you are using Block Storage (cinder) as an Image service (glance) back end, you must also complete the following steps:

   1. Add the following GlanceApiOptVolumes configuration to the heat template:

      parameter_defaults: GlanceApiOptVolumes: - /etc/multipath.conf:/etc/multipath.conf:ro - /etc/multipath/:/etc/multipath/:rw

   2. Set the ControllerExtraConfig parameter in the following way:

      parameter_defaults: ControllerExtraConfig: glance::config::api_config: default_backend/cinder_use_multipath: value: true

      Note

      Ensure that both default_backend and the GlanceBackendID heat template default value match.

7. For every configured back end, set use_multipath_for_image_xfer to true:

   parameter_defaults: ExtraConfig: cinder::config::cinder_config: <backend>/use_multipath_for_image_xfer: value: true

8. Run the following command to update the overcloud:

   $ openstack overcloud deploy

   Note

   When you run the openstack overcloud deploy command to install and configure multipath, you must pass all previous roles and environment files that you used to deploy the overcloud, such as --templates, --roles-file, -e for all environment files, and --timeout. Failure to pass all previous roles and environment files can cause problems with your overcloud deployment. For more information about using overcloud parameters, see Creating the Overcloud with the CLI Tools in the Director Installation and Usage guide.

Procedure

1. Create a VM.
2. Attach a non-encrypted volume to the VM.

3. Get the name of the Compute node that contains the instance:

   $ nova show INSTANCE | grep OS-EXT-SRV-ATTR:host

   Replace INSTANCE with the name of the VM that you booted.

4. Retrieve the virsh name of the instance:

   $ nova show INSTANCE | grep instance_name

   Replace INSTANCE with the name of the VM that you booted.

5. Get the IP address of the Compute node:

   $ . stackrc$ nova list | grep compute_name

   Replace compute_name with the name from the output of the nova show INSTANCE command.

6. SSH into the Compute node that runs the VM:

   $ ssh heat-admin@COMPUTE_NODE_IP

   Replace COMPUTE_NODE_IP with the IP address of the Compute node.

7. Log in to the container that runs virsh:

   $ podman exec -it nova_libvirt /bin/bash

8. Enter the following command on a Compute node instance to verify that it is using multipath in the cinder volume host location:

   virsh domblklist VIRSH_INSTANCE_NAME | grep /dev/dm

   Replace VIRSH_INSTANCE_NAME with the output of the nova show INSTANCE | grep instance_name command.

   If the instance shows a value other than /dev/dm-, the connection is non-multipath and you must refresh the connection info with the nova shelve and nova unshelve commands:

   $ nova shelve <instance>$ nova unshelve <instance>

   Note

   If you have more than one type of back end, you must verify the instances and volumes on all back ends, because connection info that each back end returns might vary.