|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.hive.ql.optimizer.GenMapRedUtils
public final class GenMapRedUtils
General utility common functions for the Processor to convert operator into map-reduce tasks.
Method Summary | |
---|---|
static MapredWork |
getMapRedWork(ParseContext parseCtx)
create a new plan and return. |
static MapredWork |
getMapRedWorkFromConf(HiveConf conf)
create a new plan and return. |
static void |
initPlan(ReduceSinkOperator op,
GenMRProcContext opProcCtx)
Initialize the current plan by adding it to root tasks. |
static void |
initUnionPlan(GenMRProcContext opProcCtx,
UnionOperator currUnionOp,
Task<? extends Serializable> currTask,
boolean local)
|
static void |
initUnionPlan(ReduceSinkOperator op,
UnionOperator currUnionOp,
GenMRProcContext opProcCtx,
Task<? extends Serializable> unionTask)
Initialize the current union plan. |
static void |
joinPlan(Task<? extends Serializable> currTask,
Task<? extends Serializable> oldTask,
GenMRProcContext opProcCtx)
Merge the current task into the old task for the reducer |
static void |
joinUnionPlan(GenMRProcContext opProcCtx,
UnionOperator currUnionOp,
Task<? extends Serializable> currentUnionTask,
Task<? extends Serializable> existingTask,
boolean local)
|
static Operator<? extends OperatorDesc> |
putOpInsertMap(Operator<? extends OperatorDesc> op,
RowResolver rr,
ParseContext parseCtx)
insert in the map for the operator to row resolver. |
static void |
replaceMapWork(String sourceAlias,
String targetAlias,
MapWork source,
MapWork target)
Replace the Map-side operator tree associated with targetAlias in target with the Map-side operator tree associated with sourceAlias in source. |
static void |
setKeyAndValueDesc(ReduceWork plan,
Operator<? extends OperatorDesc> topOp)
set key and value descriptor. |
static void |
setKeyAndValueDescForTaskTree(Task<? extends Serializable> task)
Set the key and value description for all the tasks rooted at the given task. |
static void |
setTaskPlan(String alias_id,
Operator<? extends OperatorDesc> topOp,
Task<?> task,
boolean local,
GenMRProcContext opProcCtx)
set the current task in the mapredWork. |
static void |
setTaskPlan(String alias_id,
Operator<? extends OperatorDesc> topOp,
Task<?> task,
boolean local,
GenMRProcContext opProcCtx,
PrunedPartitionList pList)
set the current task in the mapredWork. |
static void |
setTaskPlan(String path,
String alias,
Operator<? extends OperatorDesc> topOp,
MapWork plan,
boolean local,
TableDesc tt_desc)
set the current task in the mapredWork. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Method Detail |
---|
public static void initPlan(ReduceSinkOperator op, GenMRProcContext opProcCtx) throws SemanticException
op
- the reduce sink operator encounteredopProcCtx
- processing context
SemanticException
public static void initUnionPlan(ReduceSinkOperator op, UnionOperator currUnionOp, GenMRProcContext opProcCtx, Task<? extends Serializable> unionTask) throws SemanticException
op
- the reduce sink operator encounteredopProcCtx
- processing context
SemanticException
public static void initUnionPlan(GenMRProcContext opProcCtx, UnionOperator currUnionOp, Task<? extends Serializable> currTask, boolean local) throws SemanticException
SemanticException
public static void joinUnionPlan(GenMRProcContext opProcCtx, UnionOperator currUnionOp, Task<? extends Serializable> currentUnionTask, Task<? extends Serializable> existingTask, boolean local) throws SemanticException
SemanticException
public static void joinPlan(Task<? extends Serializable> currTask, Task<? extends Serializable> oldTask, GenMRProcContext opProcCtx) throws SemanticException
currTask
- the current task for the current reduceroldTask
- the old task for the current reduceropProcCtx
- processing context
SemanticException
public static void setTaskPlan(String alias_id, Operator<? extends OperatorDesc> topOp, Task<?> task, boolean local, GenMRProcContext opProcCtx) throws SemanticException
alias_id
- current aliastopOp
- the top operator of the stackplan
- current planlocal
- whether you need to add to map-reduce or local workopProcCtx
- processing context
SemanticException
public static void setTaskPlan(String alias_id, Operator<? extends OperatorDesc> topOp, Task<?> task, boolean local, GenMRProcContext opProcCtx, PrunedPartitionList pList) throws SemanticException
alias_id
- current aliastopOp
- the top operator of the stackplan
- current planlocal
- whether you need to add to map-reduce or local workopProcCtx
- processing contextpList
- pruned partition list. If it is null it will be computed on-the-fly.
SemanticException
public static void setTaskPlan(String path, String alias, Operator<? extends OperatorDesc> topOp, MapWork plan, boolean local, TableDesc tt_desc) throws SemanticException
alias
- current aliastopOp
- the top operator of the stackplan
- current planlocal
- whether you need to add to map-reduce or local worktt_desc
- table descriptor
SemanticException
public static void setKeyAndValueDesc(ReduceWork plan, Operator<? extends OperatorDesc> topOp)
plan
- current plantopOp
- current top operator in the pathpublic static void setKeyAndValueDescForTaskTree(Task<? extends Serializable> task)
task
- public static MapredWork getMapRedWork(ParseContext parseCtx)
public static MapredWork getMapRedWorkFromConf(HiveConf conf)
public static Operator<? extends OperatorDesc> putOpInsertMap(Operator<? extends OperatorDesc> op, RowResolver rr, ParseContext parseCtx)
op
- operator createdrr
- row resolverparseCtx
- parse contextpublic static void replaceMapWork(String sourceAlias, String targetAlias, MapWork source, MapWork target)
sourceAlias
- targetAlias
- source
- target
-
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |