|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.hive.ql.exec.Operator<MuxDesc>
org.apache.hadoop.hive.ql.exec.MuxOperator
public class MuxOperator
MuxOperator is used in the Reduce side of MapReduce jobs optimized by Correlation Optimizer. Correlation Optimizer will remove unnecessary ReduceSinkOperaotrs, and MuxOperators are used to replace those ReduceSinkOperaotrs. Example: The original operator tree is ... JOIN2 / \ RS4 RS5 / \ GBY1 JOIN1 | / \ RS1 RS2 RS3 If GBY1, JOIN1, and JOIN2 can be executed in the same reducer (optimized by Correlation Optimizer). The new operator tree will be ... JOIN2 | MUX / \ GBY1 JOIN1 \ / DEMUX / | \ / | \ / | \ RS1 RS2 RS3 A MuxOperator has two functions. First, it will construct key, value and tag structure for the input of Join Operators. Second, it is a part of operator coordination mechanism which makes sure the operator tree in the Reducer can work correctly.
Nested Class Summary | |
---|---|
protected static class |
MuxOperator.Handler
Handler is used to construct the key-value structure. |
Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator |
---|
Operator.OperatorFunc, Operator.ProgressCounter, Operator.State |
Field Summary | |
---|---|
protected static org.apache.commons.logging.Log |
LOG
|
Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator |
---|
alias, beginTime, childOperators, childOperatorsArray, childOperatorsTag, colExprMap, conf, counterNames, counterNameToEnum, counters, done, fatalErrorCntr, groupKeyObject, id, inputObjInspectors, inputRows, isLogInfoEnabled, numInputRowsCntr, numOutputRowsCntr, operatorId, out, outputObjInspector, outputRows, parentOperators, reporter, state, statsMap, timeTakenCntr, totalTime |
Constructor Summary | |
---|---|
MuxOperator()
|
Method Summary | |
---|---|
protected void |
closeOp(boolean abort)
Operator specific close routine. |
void |
endGroup()
|
void |
forward(Object row,
ObjectInspector rowInspector)
|
String |
getName()
Implements the getName function for the Node Interface. |
static String |
getOperatorName()
|
OperatorType |
getType()
Return the type of the specific operator among the types in OperatorType. |
protected void |
initializeChildren(Configuration hconf)
Calls initialize on each of the children with outputObjetInspector as the output row format. |
protected void |
initializeOp(Configuration hconf)
Operator specific initialization. |
void |
processGroup(int tag)
|
void |
processOp(Object row,
int tag)
Process the row. |
void |
startGroup()
|
Methods inherited from class java.lang.Object |
---|
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Field Detail |
---|
protected static final org.apache.commons.logging.Log LOG
Constructor Detail |
---|
public MuxOperator()
Method Detail |
---|
protected void initializeOp(Configuration hconf) throws HiveException
Operator
initializeOp
in class Operator<MuxDesc>
HiveException
protected void initializeChildren(Configuration hconf) throws HiveException
initializeChildren
in class Operator<MuxDesc>
HiveException
public void processOp(Object row, int tag) throws HiveException
Operator
processOp
in class Operator<MuxDesc>
row
- The object representing the row.tag
- The tag of the row usually means which parent this row comes from.
Rows with the same tag should have exactly the same rowInspector
all the time.
HiveException
public void forward(Object row, ObjectInspector rowInspector) throws HiveException
forward
in class Operator<MuxDesc>
HiveException
public void startGroup() throws HiveException
startGroup
in class Operator<MuxDesc>
HiveException
public void endGroup() throws HiveException
endGroup
in class Operator<MuxDesc>
HiveException
public void processGroup(int tag) throws HiveException
processGroup
in class Operator<MuxDesc>
HiveException
protected void closeOp(boolean abort) throws HiveException
Operator
closeOp
in class Operator<MuxDesc>
HiveException
public String getName()
Operator
getName
in interface Node
getName
in class Operator<MuxDesc>
public static String getOperatorName()
public OperatorType getType()
Operator
getType
in class Operator<MuxDesc>
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |