# ApiRangerReplicationArgs Data Model

Replication arguments for replicating services defined in Ranger. It works the following way:

1. EXPORT the given services, policies and roles from the source Ranger, then COPY the exported result to destination, see available arguments in ApiRangerReplicationExportArgs. Note that if getExportArgs not provided, then services defined in Ranger won't be replicated, it will be skipped.
2. TRANSFORM the exported services, policies and roles from the source Ranger on the destination, see available arguments in ApiRangerReplicationTransformArgs. Includes arguments for transforming HDFS Ranger service policies to cloud policies.
3. IMPORT the exported and transformed services, policies and roles from source to destination Ranger, see available arguments in ApiRangerReplicationImportArgs
4. For replicating the Ranger HDFS audit logs, there are 2 options:
   - If getAuditLogDistCpArgs is provided then the replication will distributed copy Ranger HDFS audit logs from source HDFS to destination HDFS, see available arguments in ApiHdfsReplicationArguments. If provided, then the mandatory fields are getUserName (destination user) and getSourceUser (source user). Any other ApiHdfsReplicationArguments fields could be provided, sourcePath and destinationPath will be set appropriately by CM, when not specified.
   - Alternatively, you can provide getAuditLogDistCpCloudArgs, in this case distributed copy for Ranger HDFS audit logs from source HDFS to destination cloud storage, see available arguments in ApiHdfsCloudReplicationArguments. Mandatory fields are getSourceUser (source user), getDestinationAccount (destination user) and getDestinationPath. Any other ApiHdfsCloudReplicationArguments fields could be provided, sourcePath will be set appropriately by CM, when not specified.

Note that at least one of getExportArgs, getAuditLogDistCpArgs, getAuditLogDistCpCloudArgs must be provided, otherwise the replication would not do anything.

Properties
name | data type | description
--- | --- | ---
sourceRangerService | ApiServiceRef | The source service to replicate from.
exportArgs | ApiRangerReplicationExportArgs | If provided then replicate services defined in Ranger from source to destination. If not provided then it will be skipped. Mandatory field is getRangerServiceNames.
transformArgs | ApiRangerReplicationTransformArgs | Replication arguments for the TRANSFORM operation. Relevant only if getExportArgs was provided.
importArgs | ApiRangerReplicationImportArgs | Replication arguments for the IMPORT operation. Relevant only if getExportArgs was provided.

auditLogDistCpArgs

ApiHdfsReplicationArguments

If provided then the replication will distributed copy Ranger HDFS audit logs from source to destination HDFS. If not provided then it will be skipped. Mandatory fields are getUserName and getSourceUser. Any other ApiHdfsReplicationArguments fields could be provided, sourcePath and destinationPath will be set appropriately by CM, when not specified.

Note that auditLogDistCpArgs and auditLogDistCpCloudArgs are mutually exclusive fields, so only one can be provided.

auditLogDistCpCloudArgs

ApiHdfsCloudReplicationArguments

If provided then the replication will distributed copy Ranger HDFS audit logs from source HDFS to cloud. If not provided then it will be skipped. Mandatory fields are getSourceUser (source user), getDestinationAccount (destination user) and getDestinationPath. Any other ApiHdfsReplicationArguments fields could be provided, sourcePath will be set appropriately by CM, when not specified.

Note that auditLogDistCpArgs and auditLogDistCpCloudArgs are mutually exclusive fields, so only one can be provided.

Example

    {
      "sourceRangerService" : {
        "peerName" : "...",
        "clusterName" : "...",
        "clusterDisplayName" : "...",
        "serviceName" : "...",
        "serviceDisplayName" : "...",
        "serviceType" : "..."
      },
      "exportArgs" : {
        "rangerServiceNames" : [ "...", "..." ]
      },
      "transformArgs" : {
        "rangerServiceNamesMapping" : {
          "property1" : "...",
          "property2" : "..."
        },
        "rangerUsersMapping" : {
          "property1" : "...",
          "property2" : "..."
        },
        "rangerResourcesMapping" : {
          "property1" : "...",
          "property2" : "..."
        },
        "hiveUrlsMapping" : {
          "property1" : "...",
          "property2" : "..."
        },
        "hdfsToAdlsArgs" : {
          "adlsStorageAccount" : "...",
          "adlsStorageAccountContainer" : "..."
        },
        "hdfsToS3Args" : {
          "s3BucketName" : "..."
        },
        "hdfsToGcsArgs" : {
          "gcsBucketName" : "..."
        }
      },
      "importArgs" : {
        "importStrategy" : "OVERRIDE",
        "extraAuthSiteProperties" : {
          "property1" : "...",
          "property2" : "..."
        }
      },
      "auditLogDistCpArgs" : {
        "sourceService" : {
          "peerName" : "...",
          "clusterName" : "...",
          "clusterDisplayName" : "...",
          "serviceName" : "...",
          "serviceDisplayName" : "...",
          "serviceType" : "..."
        },
        "sourcePath" : "...",
        "destinationPath" : "...",
        "mapreduceServiceName" : "...",
        "schedulerPoolName" : "...",
        "userName" : "...",
        "sourceUser" : "...",
        "numMaps" : 12345,
        "dryRun" : true,
        "bandwidthPerMap" : 12345,
        "abortOnError" : true,
        "removeMissingFiles" : true,
        "preserveReplicationCount" : true,
        "preserveBlockSize" : true,
        "preservePermissions" : true,
        "logPath" : "...",
        "skipChecksumChecks" : true,
        "skipListingChecksumChecks" : true,
        "skipTrash" : true,
        "replicationStrategy" : "STATIC",
        "preserveXAttrs" : true,
        "exclusionFilters" : [ "...", "..." ],
        "raiseSnapshotDiffFailures" : true,
        "deleteLatestSourceSnapshotOnJobFailure" : true,
        "numFetchThreads" : 12345,
        "destinationCloudAccount" : "..."
      },
      "auditLogDistCpCloudArgs" : {
        "sourceAccount" : "...",
        "destinationAccount" : "...",
        "sourceService" : {
          "peerName" : "...",
          "clusterName" : "...",
          "clusterDisplayName" : "...",
          "serviceName" : "...",
          "serviceDisplayName" : "...",
          "serviceType" : "..."
        },
        "sourcePath" : "...",
        "destinationPath" : "...",
        "mapreduceServiceName" : "...",
        "schedulerPoolName" : "...",
        "userName" : "...",
        "sourceUser" : "...",
        "numMaps" : 12345,
        "dryRun" : true,
        "bandwidthPerMap" : 12345,
        "abortOnError" : true,
        "removeMissingFiles" : true,
        "preserveReplicationCount" : true,
        "preserveBlockSize" : true,
        "preservePermissions" : true,
        "logPath" : "...",
        "skipChecksumChecks" : true,
        "skipListingChecksumChecks" : true,
        "skipTrash" : true,
        "replicationStrategy" : "STATIC",
        "preserveXAttrs" : true,
        "exclusionFilters" : [ "...", "..." ],
        "raiseSnapshotDiffFailures" : true,
        "deleteLatestSourceSnapshotOnJobFailure" : true,
        "numFetchThreads" : 12345,
        "destinationCloudAccount" : "..."
      }
    }