ApiTransferFileReq Data Model

Represents the arguments required for transferring a file.

Properties
name data type description
sourceService ApiServiceRef (Possibly remote) service from which the source file should be transferred.
sourceProxyUser string User on the source cluster accessing the source file.
sourcePath string Path of the source file to be transferred.

After a successful transfer, this file will be deleted.

sourceAdditionalConfigs map of string A map of additional configuration options for the source file system. These will override the values gathered automatically by getting the client configuration of the source service.

null values are not allowed; use sourceUnsetConfigs to remove configuration properties.

The configuration of the source file system is built this way:

  1. Gather the client configuration for the source service
  2. Add the content of sourceAdditionalConfigs to each xml files present in the client configuration, possibly overriding properties that have been already present.
  3. Remove the properties present in sourceUnsetConfigs from each xml files present in the client configuration.
sourceUnsetConfigs array of string A set of configuration properties to be unset for the source file system.

The properties listed here will be removed from the source file system configuration.

destinationProxyUser string User on the executor cluster accessing the destination file.
destinationPath string Path of the destination file to be created.
additionalConfigs map of string A map of additional configuration options. These will override the values gathered automatically by getting the client configuration of the destination service.

null values are not allowed; use unsetConfigs to remove configuration properties.

This configuration will be used for everything apart from the source file system; it is used e.g. for the destination file system and for the underlying implementation of the FS copy (org.apache.hadoop.fs.FileUtil#copy).

The configuration is built this way:

  1. Gather the client configuration for the destination service
  2. Add the content of additionalConfigs to each xml files present in the client configuration, possibly overriding properties that have been already present.
  3. Remove the properties present in unsetConfigs from each xml files present in the client configuration.
unsetConfigs array of string A set of configuration properties to be unset.

The properties listed here will be removed from the configuration.

additionalEnvironmentVariables map of string A map of additional environment variables to be used in the one-off process

This is the place to add environment variables related to e.g. kinit that acquires the TGT on the execution-side (usually the destination) or JVM options.

null values are not allowed; this interface must be changed should that be necessary.

Example

{
  "sourceService" : {
    "peerName" : "...",
    "clusterName" : "...",
    "clusterDisplayName" : "...",
    "serviceName" : "...",
    "serviceDisplayName" : "...",
    "serviceType" : "..."
  },
  "sourceProxyUser" : "...",
  "sourcePath" : "...",
  "sourceAdditionalConfigs" : {
    "property1" : "...",
    "property2" : "..."
  },
  "sourceUnsetConfigs" : [ "...", "..." ],
  "destinationProxyUser" : "...",
  "destinationPath" : "...",
  "additionalConfigs" : {
    "property1" : "...",
    "property2" : "..."
  },
  "unsetConfigs" : [ "...", "..." ],
  "additionalEnvironmentVariables" : {
    "property1" : "...",
    "property2" : "..."
  }
}