org.apache.hadoop.hive.ql.io
Class HiveInputFormat<K extends WritableComparable,V extends Writable>
java.lang.Object
org.apache.hadoop.hive.ql.io.HiveInputFormat<K,V>
- Direct Known Subclasses:
- BucketizedHiveInputFormat, CombineHiveInputFormat, HiveIndexedInputFormat
public class HiveInputFormat<K extends WritableComparable,V extends Writable>
- extends Object
HiveInputFormat is a parameterized InputFormat which looks at the path name
and determine the correct InputFormat for that path name from
mapredPlan.pathToPartitionInfo(). It can be used to read files with different
input format in the same map-reduce job.
Method Summary |
void |
configure(JobConf job)
|
static |
getInputFormatFromCache(Class inputFormatClass,
JobConf job)
|
protected static PartitionDesc |
getPartitionDescFromPath(Map<String,PartitionDesc> pathToPartitionInfo,
Path dir)
|
RecordReader |
getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
InputSplit[] |
getSplits(JobConf job,
int numSplits)
|
protected void |
init(JobConf job)
|
static void |
pushFilters(JobConf jobConf,
TableScanOperator tableScan)
|
protected void |
pushProjectionsAndFilters(JobConf jobConf,
Class inputFormatClass,
String splitPath,
String splitPathWithNoSchema)
|
protected void |
pushProjectionsAndFilters(JobConf jobConf,
Class inputFormatClass,
String splitPath,
String splitPathWithNoSchema,
boolean nonNative)
|
void |
validateInput(JobConf job)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LOG
public static final org.apache.commons.logging.Log LOG
inputFormats
protected static Map<Class,> inputFormats
- A cache of InputFormat instances.
pathToPartitionInfo
protected Map<String,PartitionDesc> pathToPartitionInfo
HiveInputFormat
public HiveInputFormat()
configure
public void configure(JobConf job)
getInputFormatFromCache
public static getInputFormatFromCache(Class inputFormatClass,
JobConf job)
throws IOException
- Throws:
IOException
getRecordReader
public RecordReader getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
throws IOException
- Throws:
IOException
init
protected void init(JobConf job)
getSplits
public InputSplit[] getSplits(JobConf job,
int numSplits)
throws IOException
- Throws:
IOException
validateInput
public void validateInput(JobConf job)
throws IOException
- Throws:
IOException
getPartitionDescFromPath
protected static PartitionDesc getPartitionDescFromPath(Map<String,PartitionDesc> pathToPartitionInfo,
Path dir)
throws IOException
- Throws:
IOException
pushFilters
public static void pushFilters(JobConf jobConf,
TableScanOperator tableScan)
pushProjectionsAndFilters
protected void pushProjectionsAndFilters(JobConf jobConf,
Class inputFormatClass,
String splitPath,
String splitPathWithNoSchema)
pushProjectionsAndFilters
protected void pushProjectionsAndFilters(JobConf jobConf,
Class inputFormatClass,
String splitPath,
String splitPathWithNoSchema,
boolean nonNative)
Copyright © 2012 The Apache Software Foundation