# Migrating HDFS Data to Ozone

This guide walks you through migrating data from HDFS on a source cluster to Apache Ozone on a target cluster using Cloudera Migration Assistant (CMA).

## What You Will Achieve

By the end of this guide, CMA will have copied your selected HDFS directories to Ozone storage on the target cluster using Replication Manager policies. The source HDFS data is not modified or deleted during this process.

## Before You Begin

Before you start, confirm the following:

- Both the source cluster (with HDFS) and the target cluster (with Ozone) are registered in CMA. See [Registering Clusters](cma-cluster-registration.md) for details.
- The target cluster has Apache Ozone installed and the Ozone service is running.
- The Ozone storage layout on the target cluster has been created **before** starting the migration. You must define the target volumes, buckets, and folders first using the Storage Layout Editor, then deploy them to the cluster. See [Managing Ozone Storage](../administrator/cma-ozone-storage.md) for the complete procedure. Without a deployed Ozone storage layout, there is no destination for the migrated data.
- The CMA Agent is deployed on both the source and target clusters.
- Replication Manager is configured and accessible. CMA uses the Replication Manager Java API to create the replication policies that perform the actual data copy.
- Kerberos authentication is configured correctly if the clusters are secured. CMA uses the configured migration user principal for HDFS and Ozone operations.

---

## Step 1 — Scan the Source Cluster (HDFS)

Scan the source cluster to discover the HDFS directories available for migration.

1. On the **Clusters** page, click the source cluster name.
2. Click **Start Scanning** to open the **Scan Settings**.
3. Select **Hdfs data scan**.
4. Optionally configure the scan path to limit the scan to a specific HDFS directory (for example, `/tmp` or `/user`). This reduces scan time on large clusters.
5. Click **Scan selected**. You are redirected to the scanning progress view.
6. Wait for the scan to complete. The status indicator turns green when it finishes.

After the scan, the **Discovery** tab shows the HDFS directory tree with file sizes and ownership information.

---

## Step 2 — Scan the Target Cluster (Ozone)

Scan the target cluster so CMA can display its Ozone storage structure.

1. On the **Clusters** page, click the target cluster name.
2. Click **Start Scanning** to open the **Scan Settings**.
3. Select **Ozone data scan**.

    !!! important
        Ensure that the Ozone service is available on the target cluster before starting the scan.

4. Click **Scan selected** and wait for the scan to complete.

After the scan, the **Discovery** tab shows the Ozone storage hierarchy: volumes, buckets, and folders.

---

## Step 3 — Create a Migration Plan

A migration plan connects the source cluster to the target cluster and holds all your mapping and configuration choices.

1. Navigate to the **Plans** page.
2. Click **Create Plan**.
3. Fill in the plan details:
   - **Name** — a descriptive name for the migration plan.
   - **Source cluster** — select the cluster with HDFS data.
   - **Target cluster** — select the cluster with Ozone storage.
4. Click through the wizard steps and click **Create**. The plan opens on its detail page.

---

## Step 4 — Map HDFS Directories to Ozone Paths (Data Tab)

The **Data** tab shows a split view with the source HDFS browser on the left and the target storage browser on the right. You will create structure mappings that tell CMA which HDFS directories should be copied to which Ozone locations.

1. Inside the migration plan, click the **Data** tab.
2. In the right (target) panel, switch the storage browser from **HDFS** to **Ozone** using the service selector dropdown.
3. Navigate the target Ozone browser to the destination volume, bucket, and folder where you want the data to land.
4. In the left (source) panel, navigate the HDFS browser to the parent directory containing the data you want to migrate.
5. Hover over the source directory you want to migrate and click the **Copy** button that appears.
6. CMA creates a structure mapping linking the source HDFS path to the target Ozone path.

You can verify the mapping was created by switching to the **Review** view on the Data tab, which lists all structure mappings in the plan.

### Creating Target Folders

If the target Ozone location does not have the folder structure you need, you can create folders directly from the CMA UI:

1. In the right (target) panel, navigate to the bucket or folder where you want to create a new folder.
2. Click **Create Folder**.
3. Provide the folder name and click **Create**.

The folder is created in the CMA data model and applied to the cluster during execution.

### Mapping Multiple Directories

Repeat Step 4 for each HDFS directory you want to migrate. Each directory can be mapped to a different Ozone location. You can also map multiple source directories to the same target bucket but different folders.

---

## Step 5 — Create and Run an Execution

When your structure mappings are complete, create an execution to start the data migration.

1. Click the **Execution** tab inside the plan.
2. Click **Create Execution**.
3. The execution wizard opens:
   - **General step** — provide a name for the execution, then click **Next**.
   - **Scope step** — accept the defaults (all collections) or select specific collections, then click **Next**.
   - **Review step** — review the execution configuration, then click **Finish Execution Setup**.
4. Wait for the execution to be created. Click **View Execution** to open the execution detail page.
5. Click **Run All** to start the migration.

### What Happens During Execution

CMA creates Replication Manager policies for each structure mapping. The Replication Manager handles the actual data copy from HDFS to Ozone using DistCp under the hood. The execution steps run in sequence:

1. **Replication policy creation** — CMA calls the Replication Manager Java API to create policies that map source HDFS paths to destination Ozone paths.
2. **Data replication** — Replication Manager copies the data from HDFS to the mapped Ozone locations on the target cluster.
3. **Verification** — CMA monitors the replication status and reports completion.

Monitor progress on the execution detail page. Each step shows its status (PENDING, IN_PROGRESS, COMPLETED, or FAILED). The overall execution status turns to **COMPLETED** when all steps finish successfully.

---

## Step 6 — Verify the Migration

After the execution completes:

1. On the execution detail page, confirm that all steps show **COMPLETED** status.
2. Navigate to the target cluster's **Discovery** tab and browse the Ozone storage to verify the migrated data is present at the expected paths.
3. Optionally, verify the data on the target cluster via the command line:

    ```shell
    ozone fs -ls ofs://<service-id>/<volume>/<bucket>/<path>
    ```

---

## Troubleshooting

| Symptom | Likely Cause | Resolution |
|---|---|---|
| No HDFS directories appear after scanning | The scan path may be too narrow, or the scan did not complete. | Check the scan command log. Retry with a broader scan path (for example, `/`). |
| Ozone scan shows no volumes | The Ozone service may not be running on the target cluster. | Verify the Ozone service is running in Cloudera Manager, then rescan. |
| Structure mapping fails to create | The target Ozone path may not exist. | Create the required volumes, buckets, and folders in the [Ozone Storage Manager](../administrator/cma-ozone-storage.md) before mapping. |
| Execution step fails with authentication error | Kerberos credentials may be expired or misconfigured. | Verify the CMA Agent's Kerberos configuration and keytab. Restart the CMA Agent if needed. |
| Execution fails with Replication Manager error | Replication Manager may not be accessible or configured. | Verify that the Replication Manager URL and credentials are configured in CMA. Check the CMA Master log for details. |
| Data appears in Ozone but files are missing | DistCp may have skipped files due to permission issues. | Check the Replication Manager job logs for skipped files. Ensure the migration user has read access to all source HDFS files. |

## Related Documentation

- [Registering Clusters](cma-cluster-registration.md)
- [Scanning Clusters](cma-scanning.md)
- [Creating Collections](cma-collections.md)
- [Managing Ozone Storage](../administrator/cma-ozone-storage.md)
