# Managing Ozone storage

Cloudera Migration Assistant provides comprehensive management of Apache Ozone storage structures on Cloudera Base clusters. You can scan existing Ozone storage, browse volumes and buckets, create new storage structures manually or by importing a YAML layout definition, and migrate data between HDFS and Ozone or between Ozone locations.

## About this task

Ozone is a scalable, distributed object store optimized for big data workloads. In CMA, Ozone storage is organized in a three-level hierarchy:

- **Volumes** -- Top-level containers that provide namespace isolation. Each volume has an owner, a namespace quota (number of buckets), and a space quota.
- **Buckets** -- Storage containers within a volume that hold keys (files and folders). Buckets have configurable layout types, replication settings, quotas, encryption, and snapshot options.
- **Folders** -- Directory structures within a bucket that organize data. Folders can be nested to create complex hierarchies.

CMA allows you to manage this hierarchy through the UI and API, track changes via server-sent events (SSE), and apply the storage layout to the actual Ozone cluster through the CMA Agent.

## Before you begin

- Ensure that Cloudera Migration Assistant is set up correctly using the steps in [Deploying Cloudera Migration Assistant](cma-server-setup.md).
- Ensure that the destination cluster has Apache Ozone installed and the Ozone service is running.
- Ensure that you have a Cloudera Base cluster registered in CMA. If you do not have a cluster yet, complete the steps in [Registering clusters](../migration/cma-cluster-registration.md).
- Ensure that Kerberos authentication is configured correctly if the cluster is secured. CMA uses the configured migration user principal for Ozone operations.

## Scanning Ozone storage

Before managing Ozone storage, scan the cluster to discover existing volumes, buckets, and folders.

1. Click on the cluster you want to scan on the **Clusters** page.
2. Click **Start Scanning** to open the **Scan Settings**.
3. Select **Ozone scan**.

    !!! important
        Ensure that the Ozone service is available on the cluster before starting the scan. CMA checks service availability at scan time and reports an error if the Ozone service is not responding.

4. Click **Scan selected**.

    You are redirected to the scanning progress page where you can monitor whether the scan completed successfully or encountered an error.

5. When the scan is finished, browse the Ozone storage structure in the cluster view.

    The scan results display volumes, buckets, and folder hierarchies. For each bucket, CMA shows the layout type, replication settings, quotas, and size information. For each volume, CMA shows the namespace quota, space quota, and owner.

### Browsing Ozone storage

After scanning, you can browse the Ozone storage hierarchy:

- View all volumes on the cluster with their attributes (owner, quotas, bucket count)
- Drill into a volume to see its buckets with attributes (layout, replication type, quotas, encryption key, GDPR enforcement)
- Drill into a bucket to browse its folder hierarchy, including file sizes and ownership
- View Ozone snapshots for buckets that have snapshots enabled
- Filter folders by keyword, label, or related service (such as Hive tables or databases that reference specific Ozone paths)

## Managing volumes

You can create, update, and delete Ozone volumes through the CMA UI or API.

### Creating a volume

1. Navigate to the Ozone storage view for the cluster.
2. Click **Add Volume**.
3. Provide the following properties:

    | Property | Description |
    | --- | --- |
    | **Name** | Name of the volume (required) |
    | **Namespace quota** | Maximum number of buckets allowed in the volume |
    | **Space quota** | Maximum storage space for the volume (for example, `10TB`) |
    | **User** | Owner of the volume |

4. Click **Create**.

### Updating a volume

1. Select the volume you want to modify.
2. Edit the mutable properties (namespace quota, space quota, user).
3. Click **Save**.

### Deleting a volume

1. Select the volume you want to delete.
2. Click **Delete**.

    !!! warning
        Deleting a volume removes all its child buckets and folders. This action cannot be undone.

## Managing buckets

You can create, update, and delete Ozone buckets within a volume.

### Creating a bucket

1. Navigate to the volume where you want to create the bucket.
2. Click **Add Bucket**.
3. Provide the following properties:

    | Property | Description |
    | --- | --- |
    | **Name** | Name of the bucket (required) |
    | **Layout** | Bucket layout type: `FILE_SYSTEM_OPTIMIZED`, `OBJECT_STORE`, or `LEGACY` |
    | **Replication type** | `RATIS` or `EC` (Erasure Coding) |
    | **Replication** | For RATIS: `ONE` or `THREE`. For EC: codec-data-parity-chunksize format (for example, `RS-3-2-1024k`) |
    | **Namespace quota** | Maximum number of keys allowed in the bucket |
    | **Space quota** | Maximum storage space for the bucket (for example, `1TB`) |
    | **User** | Owner of the bucket |
    | **Bucket encryption key** | Name of the encryption key registered in the Ozone Key Management Server |
    | **Enforce GDPR** | Enable Transparent Data Encryption for GDPR compliance |
    | **Create snapshot** | Enable snapshot creation for the bucket |

4. Click **Create**.

### Updating a bucket

1. Select the bucket you want to modify.
2. Edit the mutable properties.
3. Click **Save**.

### Deleting a bucket

1. Select the bucket you want to delete.
2. Click **Delete**.

    !!! warning
        Deleting a bucket removes all its child folders. This action cannot be undone.

## Managing folders

You can create, rename, and delete folders within a bucket.

### Creating folders

1. Navigate to the bucket where you want to create folders.
2. Click **Add Folder**.
3. Provide the folder name.
4. Click **Create**.

    Non-existent intermediate folders are created automatically.

### Renaming a folder

1. Select the folder you want to rename.
2. Provide the new name.
3. Click **Save**.

### Deleting folders

1. Select the folder you want to delete.
2. Click **Delete**.

## Planning and deploying an Ozone storage layout

CMA provides a **Storage Layout Editor** for designing and deploying Ozone storage structures. Access it from the **Layout** tab on the cluster view. The editor has two tabs: **Plan** (for designing the structure) and **Deploy** (for applying it to the cluster).

### Using the Storage Layout Editor

The **Plan** tab shows a two-panel editor:

- **Left panel (Available Templates)** — reusable volume structures you can drag into the editor.
- **Right panel (Editor)** — the working area where you build the target Ozone structure.

#### Adding volumes and buckets manually

1. Click on the cluster on the **Clusters** page.
2. Click the **Layout** tab.
3. In the **Plan** tab, click the add volume button (the plus icon at the top of the editor panel).
4. Provide the volume properties in the drawer that opens, then click **Create**.
5. To add buckets to a volume, select the volume node and click **Add Bucket**.
6. To add folders to a bucket, select the bucket node and click **Add Folder**.
7. Click **Save Plan** in the toolbar to save all pending changes to the database.

#### Using templates

The **Available Templates** panel on the left shows reusable volume structures. There are two types of templates:

- **System templates** — built-in templates provided by Cloudera Migration Assistant. These cannot be deleted.
- **User templates** — templates you create from volumes in the editor. These can be deleted.

##### Searching and filtering templates

Use the **Search** field at the top of the **Available Templates** panel to filter templates by name. Use the **Type** dropdown to filter by template type.

##### Dragging a template into the editor

You can drag any node from a template directly into the editor to add it to the layout. Volumes, buckets, and folders are all draggable individually using the drag handle (the grip icon on the right side of each node). When you drag a node, the full subtree (all its children) moves with it.

To use a template:

1. In the **Available Templates** panel, click the template name to expand it and see its volumes, buckets, and folders.
2. Drag the node you want (volume, bucket, or folder) by its drag handle into the editor panel on the right.

##### Saving a volume as a template

You can save any volume in the editor as a template to reuse it later. Saving a volume as a template stores the entire volume structure (including its buckets and folders) in the **Available Templates** panel.

1. In the editor, hover over the volume node you want to save.
2. Click **Add to Template** (the download icon that appears on hover).
3. Provide a name for the template.
4. Click **Save**.

The template appears in the **Available Templates** panel and is stored in the database.

##### Deleting a user template

System templates cannot be deleted. To delete a user template:

1. In the **Available Templates** panel, hover over the template name.
2. Click the delete icon (trash) next to the template name.
3. Confirm the deletion.

#### Importing a layout from YAML / JSON

You can define a complete Ozone storage structure in a YAML /JSON file and import it into the editor.

The YAML /JSON file defines volumes, buckets, and folders in a hierarchical structure:

```yaml
volumes:
  - name: "my-volume"
    namespace-quota: 100
    space-quota: "10TB"
    user: "admin"
    buckets:
      - name: "my-bucket"
        create-snapshot: true
        namespace-quota: 1000
        space-quota: "1TB"
        user: "admin"
        layout: "FILE_SYSTEM_OPTIMIZED"
        replication-type: "RATIS"
        replication: "THREE"
        bucketkey: "my-encryption-key"
        enforcegdpr: false
        folders:
          - name: "data"
            folders:
              - name: "raw"
              - name: "processed"
          - name: "logs"
```

To import the layout:

1. In the **Plan** tab, click **Import Structure** in the toolbar.
2. Upload the YAML /JSON file or paste the YAML /JSON  content.
3. Click **Upload**.

    CMA validates the structure and loads the volumes, buckets, and folders into the editor. Validation errors appear if the YAML /JSON is malformed or contains invalid values.

4. Click **Save Plan** to save the imported structure to the database.

### Deploying the layout to the cluster

After saving a plan, deploy it to the Ozone cluster from the **Deploy** tab.

1. Click the **Deploy** tab in the **Layout** page.

    The **Deployment Queue** table shows all saved volumes, buckets, and folders that have pending changes (CREATE, MODIFY, or DELETE events not yet applied to the cluster).

2. Review the pending changes in the **Volumes**, **Buckets**, and **Folders** tables.

    !!! warning
        Deployment is irreversible. After clicking **Deploy to Ozone**, rollback is not possible. Return to the **Plan** tab to make modifications before deploying.

3. Click **Deploy to Ozone**.

    CMA sends the deploy command to the CMA Agent on the cluster. The agent processes the changes in the following order:

    a. Create, update, or delete volumes.

    b. Create, update, or delete buckets.

    c. Create, update, or delete folders.

    Monitor progress in the **Commands** tab on the cluster view.

## Understanding node status colors and styles

Both the **Storage Layout Editor** and the **Ozone browser** use colors and text styles to indicate the status of volumes, buckets, and folders. The two views show different stages of the same lifecycle.

### Storage Layout Editor

The editor shows both unsaved (local) changes and changes that have been saved to the database but not yet deployed to the cluster.

| Status | Color     | Style         | Description |
| --- |-----------|---------------| --- |
| New or locally modified (unsaved) | Light red | Normal        | Node was created or edited in the editor but not yet saved to the database. |
| Locally deleted (unsaved) | Light red | Strikethrough | Node was deleted in the editor but the deletion has not been saved yet. |
| Saved, pending CREATE | Red       | Normal        | Node was saved to the database and is queued for deployment to the cluster. |
| Saved, pending MODIFY | Default  | Italic        | Node was saved to the database and is queued for deployment to the cluster. |
| Saved, pending DELETE | Default   | Strikethrough | Node was saved as a deletion and is queued for removal from the cluster. |
| Saved, no pending event | Default     | Normal        | Node is deployed and up to date. |

### Ozone browser

The browser only shows nodes that have been saved to the database. Unsaved editor changes are not visible here. If you have unsaved changes in the editor, the browser shows the last saved state of those nodes until you save.

| Status | Color   | Style | Description |
| --- |---------| --- | --- |
| Pending CREATE (saved, not yet deployed) | Red     | Normal | Node was created and saved, awaiting deployment to the cluster. |
| Pending MODIFY (saved, not yet deployed) | Default | Italic | Node was modified and saved, awaiting deployment to the cluster. |
| Pending DELETE (saved, not yet deployed) | Default | Strikethrough | Node is queued for deletion. All children of a pending-delete node also appear with strikethrough. |
| No pending event | Default | Normal | Node is deployed and up to date. |

!!! note
    The strikethrough style in the browser propagates to all children of a node marked for deletion, so you can see the full impact of a pending delete operation before it is deployed.

## Using collections with Ozone locations

You can organize Ozone locations into collections using labels for visual organization. This is useful when preparing for migration:

1. Navigate to the Ozone storage view and browse to the folders you want to organize.
2. Select the Ozone locations.
3. Click **Add to collection** and select or create a collection.

Collections with Ozone locations can be added to migration plans for HDFS-to-Ozone or Ozone-to-Ozone migration.

## HDFS to Ozone migration

CMA supports migrating data from HDFS to Ozone storage using Replication Manager policies.

1. Ensure that the source cluster HDFS data is scanned and the destination cluster Ozone storage is set up.
2. Create a migration plan that includes HDFS locations from the source cluster.
3. In the structure mapping, map the source HDFS paths to destination Ozone paths (volume/bucket/folder).
4. Create an execution from the plan and click **Run**.

    CMA uses the Replication Manager Java API to create replication policies that copy data from HDFS to the mapped Ozone locations on the destination cluster.

## Creating folders on the destination cluster

During migration planning, you can create new folders on the destination Ozone storage directly from the CMA UI:

1. In the migration plan split view, navigate to the destination cluster Ozone storage.
2. Click **Create Folder** on the destination side.
3. Provide the folder path.
4. Click **Create**.

    The folder is created in the CMA data model and applied to the cluster during execution.

## Related information

- [Scanning the source cluster](../migration/cma-scanning.md)
- [Creating collections](../migration/cma-collections.md)
