org.apache.hadoop.hive.ql.exec
Class FileSinkOperator
java.lang.Object
org.apache.hadoop.hive.ql.exec.Operator<T>
org.apache.hadoop.hive.ql.exec.TerminalOperator<fileSinkDesc>
org.apache.hadoop.hive.ql.exec.FileSinkOperator
- All Implemented Interfaces:
- Serializable, Node
public class FileSinkOperator
- extends TerminalOperator<fileSinkDesc>
- implements Serializable
File Sink operator implementation
- See Also:
- Serialized Form
Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator |
alias, childOperators, childOperatorsArray, childOperatorsTag, colExprMap, conf, done, id, inputObjInspectors, LOG, out, outputObjInspector, parentOperators, reporter, state, statsMap |
Method Summary |
void |
closeOp(boolean abort)
Operator specific close routine. |
String |
getName()
Implements the getName function for the Node Interface. |
static FileSinkOperator.RecordWriter |
getRecordWriter(org.apache.hadoop.mapred.JobConf jc,
HiveOutputFormat<?,?> hiveOutputFormat,
Class<? extends org.apache.hadoop.io.Writable> valueClass,
boolean isCompressed,
Properties tableProp,
org.apache.hadoop.fs.Path outPath)
|
protected void |
initializeOp(org.apache.hadoop.conf.Configuration hconf)
Operator specific initialization. |
void |
jobClose(org.apache.hadoop.conf.Configuration hconf,
boolean success)
Unlike other operator interfaces which are called from map or reduce task,
jobClose is called from the jobclient side once the job has completed |
void |
process(Object row,
int tag)
Process the row. |
Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator |
areAllParentsInitialized, close, dump, endGroup, forward, getChildOperators, getChildren, getColumnExprMap, getConf, getDone, getIdentifier, getParentOperators, getSchema, getStats, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initializeChildren, logStats, preorderMap, removeChild, replaceChild, replaceParent, resetStats, setAlias, setChildOperators, setColumnExprMap, setConf, setDone, setId, setOutputCollector, setParentOperators, setReporter, setSchema, startGroup |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
outWriter
protected transient FileSinkOperator.RecordWriter outWriter
fs
protected transient org.apache.hadoop.fs.FileSystem fs
outPath
protected transient org.apache.hadoop.fs.Path outPath
finalPath
protected transient org.apache.hadoop.fs.Path finalPath
serializer
protected transient Serializer serializer
commonKey
protected transient org.apache.hadoop.io.BytesWritable commonKey
tabIdEnum
protected transient FileSinkOperator.TableIdEnum tabIdEnum
autoDelete
protected transient boolean autoDelete
FileSinkOperator
public FileSinkOperator()
initializeOp
protected void initializeOp(org.apache.hadoop.conf.Configuration hconf)
throws HiveException
- Description copied from class:
Operator
- Operator specific initialization.
- Overrides:
initializeOp
in class Operator<fileSinkDesc>
- Throws:
HiveException
getRecordWriter
public static FileSinkOperator.RecordWriter getRecordWriter(org.apache.hadoop.mapred.JobConf jc,
HiveOutputFormat<?,?> hiveOutputFormat,
Class<? extends org.apache.hadoop.io.Writable> valueClass,
boolean isCompressed,
Properties tableProp,
org.apache.hadoop.fs.Path outPath)
throws IOException,
HiveException
- Throws:
IOException
HiveException
process
public void process(Object row,
int tag)
throws HiveException
- Description copied from class:
Operator
- Process the row.
- Specified by:
process
in class Operator<fileSinkDesc>
- Parameters:
row
- The object representing the row.tag
- The tag of the row usually means which parent this row comes from.
Rows with the same tag should have exactly the same rowInspector all the time.
- Throws:
HiveException
closeOp
public void closeOp(boolean abort)
throws HiveException
- Description copied from class:
Operator
- Operator specific close routine. Operators which inherents this
class should overwrite this funtion for their specific cleanup
routine.
- Overrides:
closeOp
in class Operator<fileSinkDesc>
- Throws:
HiveException
getName
public String getName()
- Description copied from class:
Operator
- Implements the getName function for the Node Interface.
- Specified by:
getName
in interface Node
- Overrides:
getName
in class Operator<fileSinkDesc>
- Returns:
- the name of the operator
jobClose
public void jobClose(org.apache.hadoop.conf.Configuration hconf,
boolean success)
throws HiveException
- Description copied from class:
Operator
- Unlike other operator interfaces which are called from map or reduce task,
jobClose is called from the jobclient side once the job has completed
- Overrides:
jobClose
in class Operator<fileSinkDesc>
- Parameters:
hconf
- Configuration with with which job was submittedsuccess
- whether the job was completed successfully or not
- Throws:
HiveException
Copyright © 2009 The Apache Software Foundation