ApiRangerReplicationTransformArgs Data Model

Arguments for TRANSFORMING Ranger services and policies, executed after EXPORT (see ApiRangerReplicationExportArgs).

Note that if replication is from on-prem to cloud, and it contains a Ranger service with type HDFS, then depending on the cloud provider, one of the following must be provided: getHdfsToAdlsArgs, getHdfsToS3Args, getHdfsToGcsArgs. Otherwise, providing this information is not required!

Properties
name data type description
rangerServiceNamesMapping map of string A mapping between source to destination ranger services.

Note that if a mapping is not given for a service then it will stay the same on the destination.

Example: {"cm_hdfs":"dr_hdfs", "cm_hive":"dr_hive"}
Default: empty or null, every service name will stay the same on the destination

rangerUsersMapping map of string A mapping between source to destination ranger usernames.

Note that if a mapping is not given for a username then it will stay the same on the destination.

Example: {"testuser1":"testuser1@host.com", "testuser2":"testuser2@host.com" }
Default: empty or null, every username will stay the same on the destination

rangerResourcesMapping map of string A mapping between source to destination ranger resources.

If a mapping is not given for a resource then it will stay the same on the destination.

Example:
{"/srcdir1/srcfile1": "/destdir1/destfile1", "src_url.com": "dest_url.com", "source_db": "dest_db", "source_table":"dest_table"} }
Default: empty or null, every resource will remain the same on destination

If replication contains HDFS Ranger service type and replication is done from on-prem to cloud, then for transforming HDFS Ranger service policies to Cloud policies NO resource mapping is needed. Instead, provide one of the following: getHdfsToAdlsArgs, getHdfsToS3Args, getHdfsToGcsArgs

HIVE prefixed based resource URL replacement is supported, but it should be provided in a different mapping, for more info see getHiveUrlsMapping.

hiveUrlsMapping map of string Used for HIVE prefixed based resource URL replacement.

For instance, if Hive resource URLs on source are:

  • hdfs://namenodehost:8020/user/hive/warehouse/table1
  • hdfs://namenodehost:8020/user/hive/warehouse/table2
and destination is Amazon S3, and the resource mapping contains the following entry:
"hdfs://namenodehost:8020/user/hive/warehouse": "s3a://bucket"
then after transformation hive urls will be:
  • s3a://bucket/table1
  • s3a://bucket/table2
Default: empty or null, every HIVE URL will remain the same on destination.
hdfsToAdlsArgs ApiRangerReplicationHdfsToAdlsArgs Arguments for transforming HDFS Ranger service policies to Azure Data Lake Storage (ADLS) policies. Note that it's relevant only if any of the Ranger services to be replicated contain HDFS Ranger service type and replication will be done from on-prem to ADLS. Otherwise, do not provide this field!

Note that hdfsToAdlsArgs, hdfsToS3Args, hdfsToGcsArgs are mutually exclusive fields, so only one can be provided.

hdfsToS3Args ApiRangerReplicationHdfsToS3Args Arguments for transforming HDFS Ranger service policies to Amazon S3 policies. It's relevant only if any of the Ranger services to be replicated contain HDFS Ranger service type and replication will be done from on-prem to S3. Otherwise, do not provide this field!

Note that hdfsToAdlsArgs, hdfsToS3Args, hdfsToGcsArgs are mutually exclusive fields, so only one can be provided.

hdfsToGcsArgs ApiRangerReplicationHdfsToGcsArgs Arguments for transforming HDFS Ranger service policies to Google Cloud Storage (GCS) policies. Note that it's relevant only if any of the Ranger services to be replicated contain HDFS Ranger service type and replication will be done from on-prem to GCS. Otherwise, do not provide this field!

Note that hdfsToAdlsArgs, hdfsToS3Args, hdfsToGcsArgs are mutually exclusive fields, so only one can be provided.

Example

{
  "rangerServiceNamesMapping" : {
    "property1" : "...",
    "property2" : "..."
  },
  "rangerUsersMapping" : {
    "property1" : "...",
    "property2" : "..."
  },
  "rangerResourcesMapping" : {
    "property1" : "...",
    "property2" : "..."
  },
  "hiveUrlsMapping" : {
    "property1" : "...",
    "property2" : "..."
  },
  "hdfsToAdlsArgs" : {
    "adlsStorageAccount" : "...",
    "adlsStorageAccountContainer" : "..."
  },
  "hdfsToS3Args" : {
    "s3BucketName" : "..."
  },
  "hdfsToGcsArgs" : {
    "gcsBucketName" : "..."
  }
}