com.cloudera.sqoop.mapreduce
Class ImportJobBase

java.lang.Object
  extended by com.cloudera.sqoop.mapreduce.JobBase
      extended by com.cloudera.sqoop.mapreduce.ImportJobBase
Direct Known Subclasses:
DataDrivenImportJob, MySQLDumpImportJob

public class ImportJobBase
extends JobBase

Base class for running an import MapReduce job. Allows dependency injection, etc, for easy customization of import job types.


Field Summary
static org.apache.commons.logging.Log LOG
           
 
Fields inherited from class com.cloudera.sqoop.mapreduce.JobBase
inputFormatClass, mapperClass, options, outputFormatClass
 
Constructor Summary
ImportJobBase()
           
ImportJobBase(SqoopOptions opts)
           
ImportJobBase(SqoopOptions opts, java.lang.Class<? extends org.apache.hadoop.mapreduce.Mapper> mapperClass, java.lang.Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass, java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass, ImportJobContext context)
           
 
Method Summary
protected  void configureOutputFormat(org.apache.hadoop.mapreduce.Job job, java.lang.String tableName, java.lang.String tableClassName)
          Configure the output format to use for the job.
 void runImport(java.lang.String tableName, java.lang.String ormJarFile, java.lang.String splitByCol, org.apache.hadoop.conf.Configuration conf)
          Run an import job to read a table in to HDFS.
protected  boolean runJob(org.apache.hadoop.mapreduce.Job job)
          Actually run the MapReduce job.
 
Methods inherited from class com.cloudera.sqoop.mapreduce.JobBase
configureInputFormat, configureMapper, configureNumTasks, getInputFormatClass, getMapperClass, getOutputFormatClass, loadJars, setInputFormatClass, setMapperClass, setOptions, setOutputFormatClass, unloadJars
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Constructor Detail

ImportJobBase

public ImportJobBase()

ImportJobBase

public ImportJobBase(SqoopOptions opts)

ImportJobBase

public ImportJobBase(SqoopOptions opts,
                     java.lang.Class<? extends org.apache.hadoop.mapreduce.Mapper> mapperClass,
                     java.lang.Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass,
                     java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass,
                     ImportJobContext context)
Method Detail

configureOutputFormat

protected void configureOutputFormat(org.apache.hadoop.mapreduce.Job job,
                                     java.lang.String tableName,
                                     java.lang.String tableClassName)
                              throws java.lang.ClassNotFoundException,
                                     java.io.IOException
Configure the output format to use for the job.

Overrides:
configureOutputFormat in class JobBase
Throws:
java.lang.ClassNotFoundException
java.io.IOException

runJob

protected boolean runJob(org.apache.hadoop.mapreduce.Job job)
                  throws java.lang.ClassNotFoundException,
                         java.io.IOException,
                         java.lang.InterruptedException
Actually run the MapReduce job.

Overrides:
runJob in class JobBase
Throws:
java.lang.ClassNotFoundException
java.io.IOException
java.lang.InterruptedException

runImport

public void runImport(java.lang.String tableName,
                      java.lang.String ormJarFile,
                      java.lang.String splitByCol,
                      org.apache.hadoop.conf.Configuration conf)
               throws java.io.IOException,
                      ImportException
Run an import job to read a table in to HDFS.

Parameters:
tableName - the database table to read
ormJarFile - the Jar file to insert into the dcache classpath. (may be null)
splitByCol - the column of the database table to use to split the import
conf - A fresh Hadoop Configuration to use to build an MR job.
Throws:
java.io.IOException - if the job encountered an IO problem
ImportException - if the job failed unexpectedly or was misconfigured.


Copyright © 2010 Cloudera, Inc.