org.apache.hadoop.hive.ql.exec
Class FileSinkOperator
java.lang.Object
org.apache.hadoop.hive.ql.exec.Operator<T>
org.apache.hadoop.hive.ql.exec.TerminalOperator<fileSinkDesc>
org.apache.hadoop.hive.ql.exec.FileSinkOperator
- All Implemented Interfaces:
- Serializable, Node
public class FileSinkOperator
- extends TerminalOperator<fileSinkDesc>
- implements Serializable
File Sink operator implementation
- See Also:
- Serialized Form
Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator |
alias, childOperators, conf, done, gWork, id, joinAlias, LOG, out, parentOperators, reporter, statsMap |
Method Summary |
void |
close(boolean abort)
|
String |
getName()
Implements the getName function for the Node Interface. |
void |
initialize(org.apache.hadoop.conf.Configuration hconf,
org.apache.hadoop.mapred.Reporter reporter)
|
void |
jobClose(org.apache.hadoop.conf.Configuration hconf,
boolean success)
Unlike other operator interfaces which are called from map or reduce task,
jobClose is called from the jobclient side once the job has completed |
void |
process(Object row,
ObjectInspector rowInspector)
|
static void |
renameOrMoveFiles(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path src,
org.apache.hadoop.fs.Path dst)
Rename src to dst, or in the case dst already exists, move files in src
to dst. |
Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator |
dump, endGroup, forward, getChildOperators, getChildren, getConf, getDone, getParentOperators, getSchema, getStats, logStats, preorderMap, resetStats, setAlias, setChildOperators, setConf, setDone, setId, setJoinAlias, setMapredWork, setOutputCollector, setParentOperators, setSchema, startGroup |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
outWriter
protected transient FileSinkOperator.RecordWriter outWriter
fs
protected transient org.apache.hadoop.fs.FileSystem fs
outPath
protected transient org.apache.hadoop.fs.Path outPath
finalPath
protected transient org.apache.hadoop.fs.Path finalPath
serializer
protected transient Serializer serializer
commonKey
protected transient org.apache.hadoop.io.BytesWritable commonKey
tabIdEnum
protected transient FileSinkOperator.TableIdEnum tabIdEnum
autoDelete
protected transient boolean autoDelete
FileSinkOperator
public FileSinkOperator()
close
public void close(boolean abort)
throws HiveException
- Overrides:
close
in class Operator<fileSinkDesc>
- Throws:
HiveException
initialize
public void initialize(org.apache.hadoop.conf.Configuration hconf,
org.apache.hadoop.mapred.Reporter reporter)
throws HiveException
- Overrides:
initialize
in class Operator<fileSinkDesc>
- Throws:
HiveException
process
public void process(Object row,
ObjectInspector rowInspector)
throws HiveException
- Specified by:
process
in class Operator<fileSinkDesc>
- Throws:
HiveException
getName
public String getName()
- Description copied from class:
Operator
- Implements the getName function for the Node Interface.
- Specified by:
getName
in interface Node
- Overrides:
getName
in class Operator<fileSinkDesc>
- Returns:
- the name of the operator
jobClose
public void jobClose(org.apache.hadoop.conf.Configuration hconf,
boolean success)
throws HiveException
- Description copied from class:
Operator
- Unlike other operator interfaces which are called from map or reduce task,
jobClose is called from the jobclient side once the job has completed
- Overrides:
jobClose
in class Operator<fileSinkDesc>
- Parameters:
hconf
- Configuration with with which job was submittedsuccess
- whether the job was completed successfully or not
- Throws:
HiveException
renameOrMoveFiles
public static void renameOrMoveFiles(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path src,
org.apache.hadoop.fs.Path dst)
throws IOException
- Rename src to dst, or in the case dst already exists, move files in src
to dst. If there is an existing file with the same name, the new file's
name will be appended with "_1", "_2", etc.
- Parameters:
fs
- the FileSystem where src and dst are on.src
- the src directorydst
- the target directory
- Throws:
IOException
Copyright © 2009 The Apache Software Foundation