org.apache.hadoop.hive.ql.io
Class BucketizedHiveInputFormat<K extends WritableComparable,V extends Writable>

java.lang.Object
  extended by org.apache.hadoop.hive.ql.io.HiveInputFormat<K,V>
      extended by org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat<K,V>

public class BucketizedHiveInputFormat<K extends WritableComparable,V extends Writable>
extends HiveInputFormat<K,V>

BucketizedHiveInputFormat serves the similar function as hiveInputFormat but its getSplits() always group splits from one input file into one wrapper split. It is useful for the applications that requires input files to fit in one mapper.


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat
HiveInputFormat.HiveInputSplit
 
Field Summary
static org.apache.commons.logging.Log LOG
           
 
Fields inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat
inputFormats, pathToPartitionInfo
 
Constructor Summary
BucketizedHiveInputFormat()
           
 
Method Summary
 RecordReader getRecordReader(InputSplit split, JobConf job, Reporter reporter)
           
 InputSplit[] getSplits(JobConf job, int numSplits)
           
protected  FileStatus[] listStatus(JobConf job, Path path)
           
 
Methods inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat
configure, getInputFormatFromCache, getPartitionDescFromPath, init, pushFilters, pushProjectionsAndFilters, pushProjectionsAndFilters, validateInput
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Constructor Detail

BucketizedHiveInputFormat

public BucketizedHiveInputFormat()
Method Detail

getRecordReader

public RecordReader getRecordReader(InputSplit split,
                                    JobConf job,
                                    Reporter reporter)
                             throws IOException
Overrides:
getRecordReader in class HiveInputFormat<K extends WritableComparable,V extends Writable>
Throws:
IOException

listStatus

protected FileStatus[] listStatus(JobConf job,
                                  Path path)
                           throws IOException
Throws:
IOException

getSplits

public InputSplit[] getSplits(JobConf job,
                              int numSplits)
                       throws IOException
Overrides:
getSplits in class HiveInputFormat<K extends WritableComparable,V extends Writable>
Throws:
IOException


Copyright © 2012 The Apache Software Foundation