# CMA Deployment Architecture Overview

Cloudera Migration Assistant (CMA) uses a **Master-Agent architecture** to orchestrate migrations across distributed Hadoop clusters. This document explains the key components, their roles, and how they communicate.

## Architecture Diagram

```mermaid
graph TD
    User["Administrator / User"]
    Vault["HashiCorp Vault<br/>(Optional)"]

    subgraph Master["CMA Master (Control Node)"]
        GW["Gateway Server<br/>(API proxy + OAuth2 login)"]
        UI["Web UI<br/>(Angular)"]
        API["REST API<br/>(Spring Boot)"]
        AuthSrv["Auth Server<br/>(OAuth2 / JWT / LDAP)"]
        DB["Database<br/>(H2 or PostgreSQL)"]
        GRPC_S["gRPC Server"]
    end

    subgraph AgentSrc["CMA Agent (Source Cluster)"]
        GRPC_C1["gRPC Client"]
        Exec1["Command Executor"]
        Scan1["Cluster Scanner"]
    end

    subgraph AgentTgt["CMA Agent (Target Cluster)"]
        GRPC_C2["gRPC Client"]
        Exec2["Command Executor"]
        Scan2["Cluster Scanner"]
    end

    CM_Src["Cloudera Manager<br/>(Source: CDH / CDP)"]
    CM_Tgt["Cloudera Manager<br/>(Target: CDP)"]
    RM["Replication Manager<br/>(Target Cluster)"]

    User -->|"HTTPS"| GW
    GW --> UI
    GW --> API
    GW --> AuthSrv
    API --> DB
    AuthSrv --> DB
    API --> GRPC_S
    API -.->|"HTTPS"| Vault

    GRPC_C1 -->|"gRPC<br/>(agent-initiated)"| GRPC_S
    GRPC_C2 -->|"gRPC<br/>(agent-initiated)"| GRPC_S

    Exec1 --> CM_Src
    Scan1 --> CM_Src
    Exec2 --> CM_Tgt
    Scan2 --> CM_Tgt
    Exec2 -.->|"REST"| RM

    style Master fill:#e8f4fd,stroke:#1a73e8
    style AgentSrc fill:#e6f4ea,stroke:#1e8e3e
    style AgentTgt fill:#fef7e0,stroke:#f9ab00
```

## Component Overview

### CMA Master

The **CMA Master** is the central control node deployed on a dedicated host or on a cluster node via Cloudera Manager.

| Component | Description |
|-----------|-------------|
| **Gateway Server** | Entry point for all traffic. Routes API requests to Master, authentication to Auth Server. Handles OAuth2 login and session management. |
| **Web UI** | Angular-based interface for managing clusters, creating migration plans, and monitoring execution. |
| **REST API** | Spring Boot endpoints for all migration operations. |
| **Auth Server** | OAuth2/JWT-based authentication with LDAP integration and role-based access control (RBAC). |
| **Database** | Stores cluster metadata, migration plans, scan results, credentials, and execution history. H2 (embedded) for evaluation, PostgreSQL for production. |
| **gRPC Server** | Handles bidirectional streaming communication with CMA Agents. |

**Key Responsibilities:**
- Provides the administrative web interface
- Manages cluster registration (source and target)
- Stores and displays scan results
- Orchestrates migration execution plans
- Dispatches commands to agents and tracks execution status
- Manages user authentication and authorization
- Optionally stores secrets in HashiCorp Vault

### CMA Agent

The **CMA Agent** runs on or near the source/target clusters and executes commands locally. It is deployed as a Cloudera Manager add-on service.

| Component | Description |
|-----------|-------------|
| **gRPC Client** | Maintains persistent outbound connection to the Master |
| **Command Executor** | Executes scanning and migration commands, runs Ansible playbooks |
| **Cluster Scanner** | Scans cluster resources (HDFS, Hive, HBase, Spark, Oozie, Ozone) |
| **Cluster Service** | Interfaces with Cloudera Manager APIs |

**Key Responsibilities:**
- Connects to and registers with the CMA Master (outbound connection)
- Executes commands received from the Master (scanning, migration, data replication)
- Uploads scan results to the Master
- Reports command execution status
- Interfaces directly with cluster services via Cloudera Manager

### Replication Manager

For **CDP Base to CDP Base** migrations, CMA requires a **Replication Manager** (RM) service deployed via Cloudera Manager parcel on the **target cluster**. The CMA Agent on the target cluster communicates with Replication Manager over REST to create and manage replication policies for data and metadata migration (HDFS, Hive, HBase).

Replication Manager must be installed on the **same node as Cloudera Manager** on the target cluster. It is deployed as a separate parcel alongside the CMA parcel. For **HDFS to Ozone** migrations, the target cluster must run **Cloudera Manager 7.13.2** or later.

> **Note:** Replication Manager is required for CDP Base to CDP Base migrations (including HDFS to Ozone). For detailed setup and integration information, see [Replication Manager Integration](cma-replication-manager.md).

## Communication Model

```mermaid
sequenceDiagram
    participant Agent
    participant Master

    Agent->>Master: Connect + register (Instance ID)
    Master-->>Agent: Connection acknowledged

    Note over Master,Agent: Persistent bidirectional gRPC stream

    Master->>Agent: Command (scan / migrate / refresh)
    Agent-->>Master: Command acknowledgment

    loop Status updates
        Agent-->>Master: Progress update (streaming)
    end

    Agent-->>Master: Scan results (streaming)
    Agent-->>Master: Command completed
```

**Key characteristics:**
- **Agent-initiated**: Agents connect outbound to the Master (simplifies firewall rules)
- **Persistent**: Connection is maintained and automatically reconnects on failure
- **Bidirectional**: Commands flow Master → Agent; results and status flow Agent → Master
- **Concurrent**: Supports multiple agents connected simultaneously

### gRPC Services

| Service | Purpose |
|---------|---------|
| `CommandService` | Bidirectional streaming for command dispatch and acknowledgment |
| `CommandStateService` | Streaming for command status updates |
| `ScanService` | Streaming for scan result uploads |
| `CmaInstanceService` | Agent registration and instance management |
| `ManagedClusterService` | Cluster metadata synchronization |

## Deployment Models

### Standalone Deployment

In standalone mode, Master and Agent run together on the same host. Suitable for single-cluster migrations, evaluation, and testing.

```mermaid
graph LR
    subgraph Host["Single Host"]
        Master["CMA Master"]
        Agent["CMA Agent"]
    end

    Cluster["Source / Target Cluster"]

    Master <--> Agent
    Agent --> Cluster

    style Host fill:#e8f4fd,stroke:#1a73e8
```

**To deploy in standalone mode:**
```shell
cma-4.1.0.4.1.0-b5/bin/cma start
```

### Distributed Deployment (Parcel)

In distributed mode, the Master runs on a control node while Agents are deployed on each cluster via Cloudera Manager parcels. This is the recommended production deployment.

```mermaid
graph TD
    Master["CMA Master<br/>(Control Node)"]

    subgraph Source1["Source Cluster 1 (CDH)"]
        Agent1["CMA Agent"]
        CM1["Cloudera Manager"]
    end

    subgraph Source2["Source Cluster 2 (CDH)"]
        Agent2["CMA Agent"]
        CM2["Cloudera Manager"]
    end

    subgraph Target["Target Cluster (CDP)"]
        Agent3["CMA Agent"]
        CM3["Cloudera Manager"]
        RM["Replication Manager"]
    end

    Agent1 -->|"gRPC :8091"| Master
    Agent2 -->|"gRPC :8091"| Master
    Agent3 -->|"gRPC :8091"| Master
    Agent3 -.->|"REST :9011"| RM

    Agent1 --> CM1
    Agent2 --> CM2
    Agent3 --> CM3

    style Master fill:#e8f4fd,stroke:#1a73e8
    style Source1 fill:#e6f4ea,stroke:#1e8e3e
    style Source2 fill:#e6f4ea,stroke:#1e8e3e
    style Target fill:#fef7e0,stroke:#f9ab00
```

**Benefits:**
- Scalability for multiple clusters
- Reduced network latency for local operations
- Isolated execution environments
- Parallel scanning and migration across clusters

## Data Flow

### Cluster Registration

**For distributed deployments (CMA Agent deployed via Cloudera Manager):**

1. Administrator deploys CMA Agent service on the cluster via Cloudera Manager
2. Configures the required properties (see table below)
3. Starts the CMA Agent service
4. Agent automatically connects to Master via gRPC
5. Agent registers the managed cluster with Master
6. Cluster appears in CMA Master UI ready for scanning

**Required CMA Agent Configuration in Cloudera Manager:**

| Property | Description |
|----------|-------------|
| `cma_gateway_url` | URL of the CMA Gateway (e.g., `https://master-host:8093`). **Required when the CMA Agent is on a different cluster from the CMA Master.** The agent uses this URL to discover the Master host, gRPC port, and Auth Server URI automatically via the discovery endpoint (`GET /api/agent/config`). When connecting from a different cluster, also set the **CMA Master** service dependency to **none** in Cloudera Manager. |
| `cm_username` | Cloudera Manager admin username |
| `cm_password` | Cloudera Manager admin password |
| `cma_user_home` | Home directory for the cma-agent user (e.g., `/home/cma-agent`) |
| `cma_agent_client_id` | OAuth2 client ID for agent-to-master authentication (default: `cma-agent`). Must match the value configured on the CMA Master. |
| `cma_agent_client_secret` | OAuth2 client secret for agent-to-master authentication. Must match the value configured on the CMA Master. |

> **Finding the Gateway URL:** In Cloudera Manager on the cluster where CMA Master is installed, open the **CMA Master** service and click the **CMA UI** link — the URL shown in your browser (up to the port number) is the Gateway URL. The format is `https://<master-host>:8093` (HTTPS) or `http://<master-host>:8090` (HTTP). You can verify connectivity from the agent host by calling the discovery endpoint:
> ```shell
> curl -sk https://<master-host>:8093/api/agent/config
> ```
> The response returns the Auth Server URI and gRPC endpoint that the agent will use.

> **Note:** The gateway URL is the only connection parameter needed. The agent discovers all other coordinates automatically. You can still override individual values using `cma_master_url` (gRPC endpoint), `--master-host`, `--grpc-port`, or `--auth-server-uri` if automatic discovery is not suitable for your environment.

**Optional Configuration:**

| Property | Default | Description |
|----------|---------|-------------|
| `cma_java_home` | Auto-detected | JDK 17+ home directory |
| `cma_python3_executable` | `python3` | Python 3.11 executable path |
| `cma_agent_http_port` | `9090` | Agent HTTP port |
| `cma_agent_https_port` | `9093` | Agent HTTPS port |
| `kerberos.auth.enable` | `false` | Enable Kerberos authentication |
| `external_vault_url` | - | External Vault URL (if not using CM-managed Vault) |

### Scanning Flow

```mermaid
sequenceDiagram
    actor Admin
    participant UI as CMA Web UI
    participant Master as CMA Master
    participant Agent as CMA Agent
    participant CM as Cloudera Manager

    Admin->>UI: Initiate scan
    UI->>Master: POST /api/scan
    Master->>Agent: gRPC: ScanCommand
    Agent->>CM: Query cluster services
    CM-->>Agent: Service metadata
    Agent->>CM: Query HDFS, Hive, HBase, etc.
    CM-->>Agent: Resource metadata
    Agent-->>Master: Stream scan results
    Master->>Master: Store in database
    Master-->>UI: Update scan status (SSE)
    UI-->>Admin: Display scan results
```

### Migration Flow

```mermaid
sequenceDiagram
    actor Admin
    participant UI as CMA Web UI
    participant Master as CMA Master
    participant Agent as CMA Agent
    participant Cluster as Cluster Services

    Admin->>UI: Create migration plan
    Admin->>UI: Start execution
    UI->>Master: POST /api/transitions
    Master->>Agent: gRPC: MigrationCommand

    loop For each migration step
        Agent->>Cluster: Execute step (replicate / convert / transfer)
        Cluster-->>Agent: Step result
        Agent-->>Master: Progress update (streaming)
        Master-->>UI: Status update (SSE)
    end

    Agent-->>Master: Execution completed
    Master-->>UI: Final status
    UI-->>Admin: Display results
```

## Network Requirements

### Service Ports

| Service | HTTP Port | HTTPS Port | Purpose |
|---------|-----------|------------|---------|
| Gateway Server | 8090 | 8093 | Web UI and REST API (entry point for users) |
| Master Server | 8890 | 8893 | Internal REST API |
| Master gRPC | 8091 | 8091 | Agent-to-Master communication (plaintext or TLS on the same port, depending on configuration) |
| Auth Server | 9000 | 9443 | Authentication and token issuance |
| Agent Server | 9090 | 9093 | Agent local REST API |
| Replication Manager | 9011 | - | Data replication policies (CDP Base to Base) |

### Agent Network Requirements

| Requirement | Description |
|-------------|-------------|
| Outbound to Master | gRPC connection to Master port 8091 |
| Access to Cloudera Manager | HTTP/HTTPS to CM API (typically port 7180/7183) |
| SSH access | To cluster nodes for Ansible-based operations |
| Cluster service access | HDFS, Hive Metastore, HBase, Ozone as needed |

### Firewall Considerations

- **Agents initiate all connections** to the Master (outbound from agent network)
- Master does **not** need to initiate connections to agents
- Only the Gateway Server port (8090/8093) needs to be exposed to administrators
- Internal ports (Master, Auth Server) can be restricted to the CMA host
- All traffic is encrypted when TLS is enabled (recommended for production)

## Security Model

### Authentication

- **User Authentication**: OAuth2/JWT via CMA Auth Server, with optional LDAP/Active Directory integration
- **Agent Communication**: gRPC with TLS encryption (recommended for production; always enabled in Docker; auto-bootstrapped in CSD deployments). Agents authenticate to the Master using OAuth2 client credentials (`cma_agent_client_id` / `cma_agent_client_secret`). These must match on both the Master and Agent services.
- **Internal Service Authentication**: The Gateway Server authenticates to the Auth Server using OAuth2 client credentials (`cma_client_id` / `cma_client_secret`), configured on the CMA Master.
- **Cluster Authentication**: Cloudera Manager credentials stored in the database or HashiCorp Vault

### Authorization

CMA implements role-based access control (RBAC):

| Role | Permissions |
|------|-------------|
| `ROLE_CMA_ADMIN` | Full access to all features |
| `ROLE_CMA_OPERATOR` | Execute migrations, view all data |
| `ROLE_CMA_MIGRATOR` | Create and modify migrations |
| `ROLE_CMA_VIEWER` | Read-only access |

### Credential Management

- Credentials can be stored encrypted in the database or in **HashiCorp Vault** (recommended for production)
- Vault integration uses KV v2 secret engine with short-lived access tokens
- SSH keys used for Ansible-based migration operations
- Cloudera Manager API credentials stored per-cluster

## High Availability Considerations

### Master HA

- Database can be externalized to PostgreSQL for durability and backup
- Multiple Master instances not currently supported (roadmap)
- Recommend regular backup of database and configuration

### Agent HA

- Agents automatically reconnect on connection loss
- Commands are idempotent where possible
- Status tracking allows resumption of interrupted operations

## Troubleshooting

### Common Issues

| Issue | Possible Cause | Resolution |
|-------|----------------|------------|
| Agent not connecting to Master | Firewall blocking gRPC port 8091 | Ensure outbound connectivity from agent to master:8091 |
| Agent disconnects frequently | Network instability | Check network, review agent logs for reconnect errors |
| Agent port conflict (9090) | HBase ThriftServer on same port | Set `cma_agent_http_port` to 9190 in CM config |
| gRPC port conflict (8091) | CM Firehose Host Monitoring | Set `cma_grpc_port` to 8081 in CMA Master config |
| Commands not executing | Agent not registered | Verify cluster registration in CMA UI |
| Scan results missing | Upload failed | Check agent logs, retry scan |
| CMA Master slow to start | Liquibase DB migration | Wait 60-90 seconds after start command, check `/var/log/cma/cma-server.log` |

### Log Locations

**Parcel deployment (Cloudera Manager):**

| Service | Log Path |
|---------|----------|
| CMA Master Server | `/var/log/cma/cma-server.log` |
| CMA Agent Server | `/var/log/cma-agent/cma-agent-server.log` |
| CMA Auth Server | `/var/log/cma/cma-auth-server.log` |
| CMA Gateway Server | `/var/log/cma/cma-gateway-server.log` |

**Standalone deployment:**

| Service | Log Path |
|---------|----------|
| Master | `~/.cma/cma-server/logs/cma-server.log` |
| Auth Server | `~/.cma/cma-server/logs/cma-auth-server.log` |
| Gateway Server | `~/.cma/cma-server/logs/cma-gateway-server.log` |

## Related Documentation

- [CMA Deployment](cma-server-setup.md) - Setup instructions (local, Docker, parcel)
- [Role-based Access Control](cma-role-access.md) - RBAC configuration
- [Enable TLS/SSL](cma-enabling-tls-ssl.md) - Security configuration
- [Vault Setup](cma-vault-setup.md) - HashiCorp Vault integration
- [Scanning Clusters](../migration/cma-scanning.md) - Scanning operations
