org.apache.hadoop.hive.ql.exec
Class PTFOperator

java.lang.Object
  extended by org.apache.hadoop.hive.ql.exec.Operator<PTFDesc>
      extended by org.apache.hadoop.hive.ql.exec.PTFOperator
All Implemented Interfaces:
Serializable, Cloneable, Node

public class PTFOperator
extends Operator<PTFDesc>
implements Serializable

See Also:
Serialized Form

Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator
Operator.OperatorFunc, Operator.ProgressCounter, Operator.State
 
Field Summary
protected  KeyWrapper currentKeys
           
protected  KeyWrapper newKeys
           
 
Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator
alias, beginTime, childOperators, childOperatorsArray, childOperatorsTag, colExprMap, conf, counterNames, counterNameToEnum, counters, done, fatalErrorCntr, groupKeyObject, id, inputObjInspectors, inputRows, isLogInfoEnabled, LOG, numInputRowsCntr, numOutputRowsCntr, operatorId, out, outputObjInspector, outputRows, parentOperators, reporter, state, statsMap, timeTakenCntr, totalTime
 
Constructor Summary
PTFOperator()
           
 
Method Summary
protected  void closeOp(boolean abort)
          Operator specific close routine.
static void connectLeadLagFunctionsToPartition(PTFDesc ptfDesc, PTFPartition.PTFPartitionIterator<Object> pItr)
           
 PTFPartition createFirstPartitionForChain(ObjectInspector oi, HiveConf hiveConf, boolean isMapSide)
          Create a new Partition.
 String getName()
          Implements the getName function for the Node Interface.
static String getOperatorName()
           
 OperatorType getType()
          Return the type of the specific operator among the types in OperatorType.
protected  void initializeOp(Configuration jobConf)
          Operator specific initialization.
protected  void processInputPartition()
           
protected  void processMapFunction()
           
 void processOp(Object row, int tag)
          Process the row.
protected  void reconstructQueryDef(HiveConf hiveConf)
          Initialize the visitor to use the QueryDefDeserializer Use the order defined in QueryDefWalker to visit the QueryDef
protected  void setupKeysWrapper(ObjectInspector inputOI)
           
 
Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator
acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, assignCounterNameToEnum, augmentPlan, checkFatalErrors, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, close, columnNamesRowResolvedCanBeObtained, dump, dump, endGroup, fatalErrorMessage, flush, forward, getAdditionalCounters, getChildOperators, getChildren, getColumnExprMap, getConf, getConfiguration, getCounterNames, getCounterNameToEnum, getCounters, getDone, getExecContext, getGroupKeyObject, getIdentifier, getInputObjInspectors, getNextCntr, getNumChild, getNumParent, getOperatorId, getParentOperators, getSchema, getStats, getWrappedCounterName, incrCounter, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeCounters, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, process, processGroup, removeChild, removeChildAndAdoptItsChildren, removeChildren, removeParent, replaceChild, replaceParent, reset, resetId, resetLastEnumUsed, resetStats, setAlias, setChildOperators, setColumnExprMap, setConf, setCounterNames, setCounterNameToEnum, setDone, setExecContext, setGroupKeyObject, setId, setInputObjInspectors, setOperatorId, setOutputCollector, setParentOperators, setReporter, setSchema, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString, updateCounters
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

currentKeys

protected transient KeyWrapper currentKeys

newKeys

protected transient KeyWrapper newKeys
Constructor Detail

PTFOperator

public PTFOperator()
Method Detail

initializeOp

protected void initializeOp(Configuration jobConf)
                     throws HiveException
Description copied from class: Operator
Operator specific initialization.

Overrides:
initializeOp in class Operator<PTFDesc>
Throws:
HiveException

closeOp

protected void closeOp(boolean abort)
                throws HiveException
Description copied from class: Operator
Operator specific close routine. Operators which inherents this class should overwrite this funtion for their specific cleanup routine.

Overrides:
closeOp in class Operator<PTFDesc>
Throws:
HiveException

processOp

public void processOp(Object row,
                      int tag)
               throws HiveException
Description copied from class: Operator
Process the row.

Specified by:
processOp in class Operator<PTFDesc>
Parameters:
row - The object representing the row.
tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.
Throws:
HiveException

reconstructQueryDef

protected void reconstructQueryDef(HiveConf hiveConf)
                            throws HiveException
Initialize the visitor to use the QueryDefDeserializer Use the order defined in QueryDefWalker to visit the QueryDef

Parameters:
hiveConf -
Throws:
HiveException

setupKeysWrapper

protected void setupKeysWrapper(ObjectInspector inputOI)
                         throws HiveException
Throws:
HiveException

processInputPartition

protected void processInputPartition()
                              throws HiveException
Throws:
HiveException

processMapFunction

protected void processMapFunction()
                           throws HiveException
Throws:
HiveException

getName

public String getName()
Description copied from class: Operator
Implements the getName function for the Node Interface.

Specified by:
getName in interface Node
Overrides:
getName in class Operator<PTFDesc>
Returns:
the name of the operator

getOperatorName

public static String getOperatorName()

getType

public OperatorType getType()
Description copied from class: Operator
Return the type of the specific operator among the types in OperatorType.

Specified by:
getType in class Operator<PTFDesc>
Returns:
OperatorType.*

createFirstPartitionForChain

public PTFPartition createFirstPartitionForChain(ObjectInspector oi,
                                                 HiveConf hiveConf,
                                                 boolean isMapSide)
                                          throws HiveException
Create a new Partition. A partition has 2 OIs: the OI for the rows being put in and the OI for the rows coming out. You specify the output OI by giving the Serde to use to Serialize. Typically these 2 OIs are the same; but not always. For the first PTF in a chain the OI of the incoming rows is dictated by the Parent Op to this PTFOp. The output OI from the Partition is typically LazyBinaryStruct, but not always. In the case of Noop/NoopMap we keep the Strcuture the same as what is given to us.

The Partition we want to create here is for feeding the First table function in the chain. So for map-side processing use the Serde from the output Shape its InputDef. For reduce-side processing use the Serde from its RawInputShape(the shape after map-side processing).

Parameters:
oi -
hiveConf -
isMapSide -
Returns:
Throws:
HiveException

connectLeadLagFunctionsToPartition

public static void connectLeadLagFunctionsToPartition(PTFDesc ptfDesc,
                                                      PTFPartition.PTFPartitionIterator<Object> pItr)
                                               throws HiveException
Throws:
HiveException


Copyright © 2012 The Apache Software Foundation