com.cloudera.sqoop.mapreduce
Class ImportJobBase
java.lang.Object
com.cloudera.sqoop.mapreduce.JobBase
com.cloudera.sqoop.mapreduce.ImportJobBase
- Direct Known Subclasses:
- DataDrivenImportJob, MySQLDumpImportJob
public class ImportJobBase
- extends JobBase
Base class for running an import MapReduce job.
Allows dependency injection, etc, for easy customization of import job types.
Field Summary |
static org.apache.commons.logging.Log |
LOG
|
Method Summary |
protected void |
configureOutputFormat(org.apache.hadoop.mapreduce.Job job,
java.lang.String tableName,
java.lang.String tableClassName)
Configure the output format to use for the job. |
void |
runImport(java.lang.String tableName,
java.lang.String ormJarFile,
java.lang.String splitByCol,
org.apache.hadoop.conf.Configuration conf)
Run an import job to read a table in to HDFS. |
protected boolean |
runJob(org.apache.hadoop.mapreduce.Job job)
Actually run the MapReduce job. |
Methods inherited from class com.cloudera.sqoop.mapreduce.JobBase |
configureInputFormat, configureMapper, configureNumTasks, getInputFormatClass, getMapperClass, getOutputFormatClass, loadJars, setInputFormatClass, setMapperClass, setOptions, setOutputFormatClass, unloadJars |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LOG
public static final org.apache.commons.logging.Log LOG
ImportJobBase
public ImportJobBase()
ImportJobBase
public ImportJobBase(SqoopOptions opts)
ImportJobBase
public ImportJobBase(SqoopOptions opts,
java.lang.Class<? extends org.apache.hadoop.mapreduce.Mapper> mapperClass,
java.lang.Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass,
java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass,
ImportJobContext context)
configureOutputFormat
protected void configureOutputFormat(org.apache.hadoop.mapreduce.Job job,
java.lang.String tableName,
java.lang.String tableClassName)
throws java.lang.ClassNotFoundException,
java.io.IOException
- Configure the output format to use for the job.
- Overrides:
configureOutputFormat
in class JobBase
- Throws:
java.lang.ClassNotFoundException
java.io.IOException
runJob
protected boolean runJob(org.apache.hadoop.mapreduce.Job job)
throws java.lang.ClassNotFoundException,
java.io.IOException,
java.lang.InterruptedException
- Actually run the MapReduce job.
- Overrides:
runJob
in class JobBase
- Throws:
java.lang.ClassNotFoundException
java.io.IOException
java.lang.InterruptedException
runImport
public void runImport(java.lang.String tableName,
java.lang.String ormJarFile,
java.lang.String splitByCol,
org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException,
ImportException
- Run an import job to read a table in to HDFS.
- Parameters:
tableName
- the database table to readormJarFile
- the Jar file to insert into the dcache classpath.
(may be null)splitByCol
- the column of the database table to use to split
the importconf
- A fresh Hadoop Configuration to use to build an MR job.
- Throws:
java.io.IOException
- if the job encountered an IO problem
ImportException
- if the job failed unexpectedly or was
misconfigured.
Copyright © 2010 Cloudera, Inc.