com.cloudera.sqoop.shims
Class HadoopShim

java.lang.Object
  extended by com.cloudera.sqoop.shims.HadoopShim

public abstract class HadoopShim
extends java.lang.Object

In order to be compatible with multiple versions of Hadoop, all parts of the Hadoop interface that are not cross-version compatible are encapsulated in an implementation of this class. Users should use the ShimLoader class as a factory to obtain an implementation of HadoopShims corresponding to the version of Hadoop currently on the classpath.


Constructor Summary
protected HadoopShim()
           
 
Method Summary
static HadoopShim get()
           
abstract  java.lang.String getDbInputClassProperty()
          Returns the Configuration property identifying a DBWritable to use.
abstract  java.lang.String getDbInputConditionsProperty()
          Returns the Configuration property specifying WHERE conditions for the db table.
abstract  java.lang.String getDbInputTableNameProperty()
          Returns the Configuration property identifying the DB input table.
abstract  java.lang.String getDbPasswordProperty()
          Returns the Configuration property identifying the DB password.
abstract  java.lang.String getDbUrlProperty()
          Returns the Configuration property identifying the DB connect string.
abstract  java.lang.String getDbUsernameProperty()
          Returns the Configuration property identifying the DB username.
abstract  java.lang.String getJobLocalDirProperty()
           
abstract  org.apache.hadoop.mapreduce.MapContext getMapContextForIOPath(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path p)
          Returns a mock MapContext that has both an OutputCommitter and an InputSplit wired to the specified path.
abstract  long getNumMapInputRecords(org.apache.hadoop.mapreduce.Job job)
           
abstract  long getNumMapOutputRecords(org.apache.hadoop.mapreduce.Job job)
           
abstract  java.lang.String getTaskIdProperty()
           
abstract  java.lang.String[] parseGenericOptions(org.apache.hadoop.conf.Configuration conf, java.lang.String[] args)
          Parse arguments in 'args' via the GenericOptionsParser and embed the results in the supplied configuration.
abstract  void setJobMapSpeculativeExecution(org.apache.hadoop.mapreduce.Job job, boolean isEnabled)
          Set the mapper speculative execution property for a job.
abstract  void setJobNumMaps(org.apache.hadoop.mapreduce.Job job, int numMapTasks)
          Set the (hinted) number of map tasks for a job.
abstract  void setJobtrackerAddr(org.apache.hadoop.conf.Configuration conf, java.lang.String addr)
          Sets the Jobtracker address to use for a job.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

HadoopShim

protected HadoopShim()
Method Detail

parseGenericOptions

public abstract java.lang.String[] parseGenericOptions(org.apache.hadoop.conf.Configuration conf,
                                                       java.lang.String[] args)
                                                throws java.io.IOException
Parse arguments in 'args' via the GenericOptionsParser and embed the results in the supplied configuration.

Parameters:
conf - the configuration to populate with generic options.
args - the arguments to process.
Returns:
the unused args to be passed to the application itself.
Throws:
java.io.IOException

getNumMapOutputRecords

public abstract long getNumMapOutputRecords(org.apache.hadoop.mapreduce.Job job)
                                     throws java.io.IOException,
                                            java.lang.InterruptedException
Returns:
the number of mapper output records from a job using its counters.
Throws:
java.io.IOException
java.lang.InterruptedException

getNumMapInputRecords

public abstract long getNumMapInputRecords(org.apache.hadoop.mapreduce.Job job)
                                    throws java.io.IOException,
                                           java.lang.InterruptedException
Returns:
the number of mapper input records from a job using its counters.
Throws:
java.io.IOException
java.lang.InterruptedException

getTaskIdProperty

public abstract java.lang.String getTaskIdProperty()
Returns:
the Configuration property identifying the current task id.

getJobLocalDirProperty

public abstract java.lang.String getJobLocalDirProperty()
Returns:
the Configuration property identifying the job's local dir.

setJobNumMaps

public abstract void setJobNumMaps(org.apache.hadoop.mapreduce.Job job,
                                   int numMapTasks)
Set the (hinted) number of map tasks for a job.


setJobMapSpeculativeExecution

public abstract void setJobMapSpeculativeExecution(org.apache.hadoop.mapreduce.Job job,
                                                   boolean isEnabled)
Set the mapper speculative execution property for a job.


setJobtrackerAddr

public abstract void setJobtrackerAddr(org.apache.hadoop.conf.Configuration conf,
                                       java.lang.String addr)
Sets the Jobtracker address to use for a job.


getDbInputClassProperty

public abstract java.lang.String getDbInputClassProperty()
Returns the Configuration property identifying a DBWritable to use.


getDbUsernameProperty

public abstract java.lang.String getDbUsernameProperty()
Returns the Configuration property identifying the DB username.


getDbPasswordProperty

public abstract java.lang.String getDbPasswordProperty()
Returns the Configuration property identifying the DB password.


getDbUrlProperty

public abstract java.lang.String getDbUrlProperty()
Returns the Configuration property identifying the DB connect string.


getDbInputTableNameProperty

public abstract java.lang.String getDbInputTableNameProperty()
Returns the Configuration property identifying the DB input table.


getDbInputConditionsProperty

public abstract java.lang.String getDbInputConditionsProperty()
Returns the Configuration property specifying WHERE conditions for the db table.


getMapContextForIOPath

public abstract org.apache.hadoop.mapreduce.MapContext getMapContextForIOPath(org.apache.hadoop.conf.Configuration conf,
                                                                              org.apache.hadoop.fs.Path p)
Returns a mock MapContext that has both an OutputCommitter and an InputSplit wired to the specified path. Used for testing LargeObjectLoader.


get

public static final HadoopShim get()


Copyright © 2010 Cloudera, Inc.