org.apache.hadoop.hive.ql.optimizer.correlation
Class ReduceSinkDeDuplication.AbsctractReducerReducerProc
java.lang.Object
org.apache.hadoop.hive.ql.optimizer.correlation.ReduceSinkDeDuplication.AbsctractReducerReducerProc
- All Implemented Interfaces:
- NodeProcessor
- Enclosing class:
- ReduceSinkDeDuplication
public abstract static class ReduceSinkDeDuplication.AbsctractReducerReducerProc
- extends Object
- implements NodeProcessor
Method Summary |
protected Integer |
checkNumReducer(int creduce,
int preduce)
If number of reducers for RS is -1, the RS can have any number of reducers. |
protected Integer |
checkOrder(String corder,
String porder)
|
protected boolean |
merge(ReduceSinkOperator cRS,
JoinOperator pJoin,
int minReducer)
|
protected boolean |
merge(ReduceSinkOperator cRS,
ReduceSinkOperator pRS,
int minReducer)
Current RSDedup remove/replace child RS. |
Object |
process(Node nd,
Stack<Node> stack,
NodeProcessorCtx procCtx,
Object... nodeOutputs)
Generic process for all ops that don't have specific implementations. |
protected abstract Object |
process(ReduceSinkOperator cRS,
GroupByOperator cGBY,
ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx)
|
protected abstract Object |
process(ReduceSinkOperator cRS,
ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx)
|
protected Integer |
sameKeys(List<ExprNodeDesc> cexprs,
List<ExprNodeDesc> pexprs,
Operator<?> child,
Operator<?> parent)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
ReduceSinkDeDuplication.AbsctractReducerReducerProc
public ReduceSinkDeDuplication.AbsctractReducerReducerProc()
process
public Object process(Node nd,
Stack<Node> stack,
NodeProcessorCtx procCtx,
Object... nodeOutputs)
throws SemanticException
- Description copied from interface:
NodeProcessor
- Generic process for all ops that don't have specific implementations.
- Specified by:
process
in interface NodeProcessor
- Parameters:
nd
- operator to processprocCtx
- operator processor contextnodeOutputs
- A variable argument list of outputs from other nodes in the walk
- Returns:
- Object to be returned by the process call
- Throws:
SemanticException
process
protected abstract Object process(ReduceSinkOperator cRS,
ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx)
throws SemanticException
- Throws:
SemanticException
process
protected abstract Object process(ReduceSinkOperator cRS,
GroupByOperator cGBY,
ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx)
throws SemanticException
- Throws:
SemanticException
merge
protected boolean merge(ReduceSinkOperator cRS,
JoinOperator pJoin,
int minReducer)
throws SemanticException
- Throws:
SemanticException
merge
protected boolean merge(ReduceSinkOperator cRS,
ReduceSinkOperator pRS,
int minReducer)
throws SemanticException
- Current RSDedup remove/replace child RS. For key columns,
sorting order, and the number of reducers, copy
more specific part of configurations of child RS to that of parent RS.
For partitioning columns, if both child RS and parent RS have been assigned
partitioning columns, we will choose the more general partitioning columns.
If parent RS has not been assigned any partitioning column, we will use
partitioning columns (if exist) of child RS.
- Throws:
SemanticException
sameKeys
protected Integer sameKeys(List<ExprNodeDesc> cexprs,
List<ExprNodeDesc> pexprs,
Operator<?> child,
Operator<?> parent)
throws SemanticException
- Throws:
SemanticException
checkOrder
protected Integer checkOrder(String corder,
String porder)
checkNumReducer
protected Integer checkNumReducer(int creduce,
int preduce)
- If number of reducers for RS is -1, the RS can have any number of reducers.
It's generally true except for order-by or forced bucketing cases.
if both of num-reducers are not -1, those number should be the same.
Copyright © 2012 The Apache Software Foundation