Modifier and Type | Method and Description |
---|---|
void |
ExecutionEngine.explain(LogicalPlan lp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose,
File dir,
String suffix)
This method handles the backend processing of the Explain command.
|
Modifier and Type | Method and Description |
---|---|
void |
HExecutionEngine.explain(LogicalPlan lp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose,
File file,
String suffix) |
abstract void |
Launcher.explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose)
Explain how a pig job will be executed on the underlying infrastructure.
|
Modifier and Type | Method and Description |
---|---|
void |
FetchLauncher.explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format)
Creates an empty MR plan
|
static boolean |
FetchOptimizer.isPlanFetchable(PigContext pc,
PhysicalPlan pp)
Visits the plan with
FetchablePlanVisitor and checks whether the
plan is fetchable. |
Modifier and Type | Class and Description |
---|---|
class |
MRCompilerException |
Modifier and Type | Method and Description |
---|---|
MROperPlan |
MRCompiler.compile()
The front-end method that the user calls to compile
the plan.
|
MROperPlan |
MapReduceLauncher.compile(PhysicalPlan php,
PigContext pc) |
void |
MapReduceLauncher.explain(PhysicalPlan php,
PigContext pc,
PrintStream ps,
String format,
boolean verbose) |
PigStats |
MapReduceLauncher.launchPig(PhysicalPlan php,
String grpName,
PigContext pc) |
void |
SampleOptimizer.visit() |
void |
NativeMapReduceOper.visit(MROpPlanVisitor v) |
void |
MapReduceOper.visit(MROpPlanVisitor v) |
void |
PhyPlanSetter.visitAdd(Add add) |
void |
PhyPlanSetter.visitAnd(POAnd and) |
void |
PhyPlanSetter.visitCollectedGroup(POCollectedGroup mg) |
void |
MRCompiler.visitCollectedGroup(POCollectedGroup op) |
void |
PhyPlanSetter.visitComparisonFunc(POUserComparisonFunc compFunc) |
void |
PhyPlanSetter.visitConstant(ConstantExpression cnst) |
void |
MRCompiler.visitCounter(POCounter op)
For the counter job, it depends if it is row number or not.
|
void |
MRCompiler.visitCross(POCross op) |
void |
PhyPlanSetter.visitDemux(PODemux demux) |
void |
PhyPlanSetter.visitDistinct(PODistinct distinct) |
void |
MRCompiler.visitDistinct(PODistinct op) |
void |
PhyPlanSetter.visitDivide(Divide dv) |
void |
PhyPlanSetter.visitEqualTo(EqualToExpr eq) |
void |
PhyPlanSetter.visitFilter(POFilter fl) |
void |
MRCompiler.visitFilter(POFilter op) |
void |
PhyPlanSetter.visitFRJoin(POFRJoin join) |
void |
MRCompiler.visitFRJoin(POFRJoin op)
This is an operator which will have multiple inputs(= to number of join inputs)
But it prunes off all inputs but the fragment input and creates separate MR jobs
for each of the replicated inputs and uses these as the replicated files that
are configured in the POFRJoin operator.
|
void |
PhyPlanSetter.visitGlobalRearrange(POGlobalRearrange gr) |
void |
MRCompiler.visitGlobalRearrange(POGlobalRearrange op) |
void |
PhyPlanSetter.visitGreaterThan(GreaterThanExpr grt) |
void |
PhyPlanSetter.visitGTOrEqual(GTOrEqualToExpr gte) |
void |
PhyPlanSetter.visitIsNull(POIsNull isNull) |
void |
PhyPlanSetter.visitLessThan(LessThanExpr lt) |
void |
PhyPlanSetter.visitLimit(POLimit lim) |
void |
MRCompiler.visitLimit(POLimit op) |
void |
PhyPlanSetter.visitLoad(POLoad ld) |
void |
MRCompiler.visitLoad(POLoad op) |
void |
PhyPlanSetter.visitLocalRearrange(POLocalRearrange lr) |
void |
MRCompiler.visitLocalRearrange(POLocalRearrange op) |
void |
PhyPlanSetter.visitLTOrEqual(LTOrEqualToExpr lte) |
void |
PhyPlanSetter.visitMergeCoGroup(POMergeCogroup mergeCoGrp) |
void |
MRCompiler.visitMergeCoGroup(POMergeCogroup poCoGrp)
Leftmost relation is referred as base relation (this is the one fed into mappers.)
First, close all MROpers except for first one (referred as baseMROPer)
Then, create a MROper which will do indexing job (idxMROper)
Connect idxMROper before the mappedMROper in the MRPlan.
|
void |
PhyPlanSetter.visitMergeJoin(POMergeJoin join) |
void |
MRCompiler.visitMergeJoin(POMergeJoin joinOp)
Since merge-join works on two inputs there are exactly two MROper predecessors identified as left and right.
|
void |
PhyPlanSetter.visitMod(Mod mod) |
void |
SecondaryKeyOptimizerMR.visitMROp(MapReduceOper mr) |
void |
SecondaryKeyOptimizer.visitMROp(MapReduceOper mr) |
void |
SampleOptimizer.visitMROp(MapReduceOper mr) |
void |
LimitAdjuster.visitMROp(MapReduceOper mr) |
void |
KeyTypeDiscoveryVisitor.visitMROp(MapReduceOper mr) |
void |
CombinerOptimizer.visitMROp(MapReduceOper mr) |
void |
AccumulatorOptimizer.visitMROp(MapReduceOper mr) |
void |
PhyPlanSetter.visitMultiply(Multiply mul) |
void |
PhyPlanSetter.visitNative(PONative nt) |
void |
MRCompiler.visitNative(PONative op) |
void |
PhyPlanSetter.visitNot(PONot not) |
void |
PhyPlanSetter.visitNotEqualTo(NotEqualToExpr eq) |
void |
PhyPlanSetter.visitOr(POOr or) |
void |
PhyPlanSetter.visitPackage(POPackage pkg) |
void |
MRCompiler.visitPackage(POPackage op) |
void |
PhyPlanSetter.visitPartialAgg(POPartialAgg poPartialAgg) |
void |
PhyPlanSetter.visitPOForEach(POForEach nfe) |
void |
MRCompiler.visitPOForEach(POForEach op) |
void |
PhyPlanSetter.visitPoissonSample(POPoissonSample poissonSample) |
void |
PhyPlanSetter.visitPOOptimizedForEach(POOptimizedForEach optimizedForEach) |
void |
PhyPlanSetter.visitPreCombinerLocalRearrange(POPreCombinerLocalRearrange preCombinerLocalRearrange) |
void |
PhyPlanSetter.visitProject(POProject proj) |
void |
PhyPlanSetter.visitRank(PORank rank) |
void |
MRCompiler.visitRank(PORank op)
In case of PORank, it is closed any other previous job (containing
POCounter as a leaf) and PORank is added on map phase.
|
void |
PhyPlanSetter.visitRegexp(PORegexp re) |
void |
PhyPlanSetter.visitReservoirSample(POReservoirSample reservoirSample) |
void |
PhyPlanSetter.visitSkewedJoin(POSkewedJoin join) |
void |
MRCompiler.visitSkewedJoin(POSkewedJoin op) |
void |
PhyPlanSetter.visitSort(POSort sort) |
void |
MRCompiler.visitSort(POSort op) |
void |
PhyPlanSetter.visitSplit(POSplit spl) |
void |
MRCompiler.visitSplit(POSplit op)
Compiles a split operator.
|
void |
PhyPlanSetter.visitStore(POStore st) |
void |
MRCompiler.visitStore(POStore op) |
void |
PhyPlanSetter.visitStream(POStream stream) |
void |
MRCompiler.visitStream(POStream op) |
void |
PhyPlanSetter.visitSubtract(Subtract sub) |
void |
PhyPlanSetter.visitUnion(POUnion un) |
void |
MRCompiler.visitUnion(POUnion op) |
void |
UDFFinishVisitor.visitUserFunc(POUserFunc userFunc) |
void |
UDFEndOfAllInputNeededVisitor.visitUserFunc(POUserFunc userFunc) |
void |
PhyPlanSetter.visitUserFunc(POUserFunc userFunc) |
Modifier and Type | Method and Description |
---|---|
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitCollectedGroup(POCollectedGroup mg) |
void |
UDFFinder.visitComparisonFunc(POUserComparisonFunc compFunc) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitLocalRearrange(POLocalRearrange lr) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitMergeCoGroup(POMergeCogroup mergeCoGrp) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitMergeJoin(POMergeJoin join) |
void |
XMLMRPrinter.visitMROp(MapReduceOper mr) |
void |
POPackageAnnotator.visitMROp(MapReduceOper mr) |
void |
MRPrinter.visitMROp(MapReduceOper mr) |
void |
MROpPlanVisitor.visitMROp(MapReduceOper mr) |
void |
MRIntermediateDataVisitor.visitMROp(MapReduceOper mr) |
void |
EndOfAllInputSetter.visitMROp(MapReduceOper mr) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitPartialAgg(POPartialAgg partAgg) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitPOForEach(POForEach foreach) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitPoissonSample(POPoissonSample poissonSample) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitReservoirSample(POReservoirSample reservoirSample) |
void |
UDFFinder.visitSort(POSort op) |
void |
EndOfAllInputSetter.EndOfAllInputChecker.visitStream(POStream stream) |
void |
UDFFinder.visitUserFunc(POUserFunc userFunc) |
void |
ScalarPhyFinder.visitUserFunc(POUserFunc userFunc) |
Modifier and Type | Class and Description |
---|---|
class |
LogicalToPhysicalTranslatorException |
Modifier and Type | Method and Description |
---|---|
abstract void |
PhysicalOperator.visit(PhyPlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
void |
Subtract.visit(PhyPlanVisitor v) |
void |
POUserFunc.visit(PhyPlanVisitor v) |
void |
POUserComparisonFunc.visit(PhyPlanVisitor v) |
void |
PORelationToExprProject.visit(PhyPlanVisitor v) |
void |
PORegexp.visit(PhyPlanVisitor v) |
void |
POProject.visit(PhyPlanVisitor v) |
void |
POOr.visit(PhyPlanVisitor v) |
void |
PONot.visit(PhyPlanVisitor v) |
void |
PONegative.visit(PhyPlanVisitor v) |
void |
POMapLookUp.visit(PhyPlanVisitor v) |
void |
POIsNull.visit(PhyPlanVisitor v) |
void |
POCast.visit(PhyPlanVisitor v) |
void |
POBinCond.visit(PhyPlanVisitor v) |
void |
POAnd.visit(PhyPlanVisitor v) |
void |
NotEqualToExpr.visit(PhyPlanVisitor v) |
void |
Multiply.visit(PhyPlanVisitor v) |
void |
Mod.visit(PhyPlanVisitor v) |
void |
LessThanExpr.visit(PhyPlanVisitor v) |
void |
LTOrEqualToExpr.visit(PhyPlanVisitor v) |
void |
GreaterThanExpr.visit(PhyPlanVisitor v) |
void |
GTOrEqualToExpr.visit(PhyPlanVisitor v) |
abstract void |
ExpressionOperator.visit(PhyPlanVisitor v) |
void |
EqualToExpr.visit(PhyPlanVisitor v) |
void |
Divide.visit(PhyPlanVisitor v) |
void |
ConstantExpression.visit(PhyPlanVisitor v) |
void |
Add.visit(PhyPlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
protected void |
PlanPrinter.breadthFirst() |
protected String |
PlanPrinter.depthFirstPP() |
protected void |
XMLPhysicalPlanPrinter.depthFirstPP(Element parentNode) |
void |
XMLPhysicalPlanPrinter.print(OutputStream printer) |
void |
PlanPrinter.print(OutputStream printer) |
void |
XMLPhysicalPlanPrinter.visit() |
void |
PlanPrinter.visit() |
void |
PhyPlanVisitor.visitAdd(Add add) |
void |
PhyPlanVisitor.visitAnd(POAnd and) |
void |
PhyPlanVisitor.visitCollectedGroup(POCollectedGroup mg) |
void |
PhyPlanVisitor.visitComparisonFunc(POUserComparisonFunc compFunc) |
void |
PhyPlanVisitor.visitConstant(ConstantExpression cnst) |
void |
PhyPlanVisitor.visitCounter(POCounter poCounter) |
void |
PhyPlanVisitor.visitCross(POCross cross) |
void |
PhyPlanVisitor.visitDemux(PODemux demux) |
void |
PhyPlanVisitor.visitDistinct(PODistinct distinct) |
void |
PhyPlanVisitor.visitDivide(Divide dv) |
void |
PhyPlanVisitor.visitEqualTo(EqualToExpr eq) |
void |
PhyPlanVisitor.visitFilter(POFilter fl) |
void |
PhyPlanVisitor.visitFRJoin(POFRJoin join) |
void |
PhyPlanVisitor.visitGlobalRearrange(POGlobalRearrange gr) |
void |
PhyPlanVisitor.visitGreaterThan(GreaterThanExpr grt) |
void |
PhyPlanVisitor.visitGTOrEqual(GTOrEqualToExpr gte) |
void |
PhyPlanVisitor.visitIsNull(POIsNull isNull) |
void |
PhyPlanVisitor.visitLessThan(LessThanExpr lt) |
void |
PhyPlanVisitor.visitLimit(POLimit lim) |
void |
UdfCacheShipFilesVisitor.visitLoad(POLoad ld) |
void |
PhyPlanVisitor.visitLoad(POLoad ld) |
void |
PhyPlanVisitor.visitLocalRearrange(POLocalRearrange lr) |
void |
PhyPlanVisitor.visitLTOrEqual(LTOrEqualToExpr lte) |
void |
PhyPlanVisitor.visitMergeCoGroup(POMergeCogroup mergeCoGrp) |
void |
PhyPlanVisitor.visitMergeJoin(POMergeJoin join) |
void |
PhyPlanVisitor.visitMod(Mod mod) |
void |
PhyPlanVisitor.visitMultiply(Multiply mul) |
void |
PhyPlanVisitor.visitNative(PONative nat) |
void |
PhyPlanVisitor.visitNot(PONot not) |
void |
PhyPlanVisitor.visitNotEqualTo(NotEqualToExpr eq) |
void |
PhyPlanVisitor.visitOr(POOr or) |
void |
PhyPlanVisitor.visitPackage(POPackage pkg) |
void |
PhyPlanVisitor.visitPartialAgg(POPartialAgg poPartialAgg) |
void |
PhyPlanVisitor.visitPartitionRearrange(POPartitionRearrange pr) |
void |
PhyPlanVisitor.visitPOForEach(POForEach nfe) |
void |
PhyPlanVisitor.visitPoissonSample(POPoissonSample poissonSample) |
void |
PhyPlanVisitor.visitPOOptimizedForEach(POOptimizedForEach optimizedForEach) |
void |
PhyPlanVisitor.visitPreCombinerLocalRearrange(POPreCombinerLocalRearrange preCombinerLocalRearrange) |
void |
PhyPlanVisitor.visitProject(POProject proj) |
void |
PhyPlanVisitor.visitRank(PORank rank) |
void |
PhyPlanVisitor.visitRegexp(PORegexp re) |
void |
PhyPlanVisitor.visitReservoirSample(POReservoirSample reservoirSample) |
void |
PhyPlanVisitor.visitSkewedJoin(POSkewedJoin sk) |
void |
PhyPlanVisitor.visitSort(POSort sort) |
void |
PhyPlanVisitor.visitSplit(POSplit spl) |
void |
UdfCacheShipFilesVisitor.visitStore(POStore st) |
void |
PhyPlanVisitor.visitStore(POStore st) |
void |
PhyPlanVisitor.visitStream(POStream stream) |
void |
PhyPlanVisitor.visitSubtract(Subtract sub) |
void |
PhyPlanVisitor.visitUnion(POUnion un) |
void |
UdfCacheShipFilesVisitor.visitUserFunc(POUserFunc udf) |
void |
PhyPlanVisitor.visitUserFunc(POUserFunc userFunc) |
Modifier and Type | Method and Description |
---|---|
void |
POUnion.visit(PhyPlanVisitor v) |
void |
POStream.visit(PhyPlanVisitor v) |
void |
POStore.visit(PhyPlanVisitor v) |
void |
POSplit.visit(PhyPlanVisitor v) |
void |
POSort.visit(PhyPlanVisitor v) |
void |
POSkewedJoin.visit(PhyPlanVisitor v) |
void |
POReservoirSample.visit(PhyPlanVisitor v) |
void |
PORank.visit(PhyPlanVisitor v) |
void |
POPreCombinerLocalRearrange.visit(PhyPlanVisitor v) |
void |
POPoissonSample.visit(PhyPlanVisitor v) |
void |
POPartialAgg.visit(PhyPlanVisitor v) |
void |
POPackage.visit(PhyPlanVisitor v) |
void |
POOptimizedForEach.visit(PhyPlanVisitor v) |
void |
PONative.visit(PhyPlanVisitor v) |
void |
POMergeJoin.visit(PhyPlanVisitor v) |
void |
POMergeCogroup.visit(PhyPlanVisitor v) |
void |
POLocalRearrange.visit(PhyPlanVisitor v) |
void |
POLoad.visit(PhyPlanVisitor v) |
void |
POLimit.visit(PhyPlanVisitor v) |
void |
POGlobalRearrange.visit(PhyPlanVisitor v) |
void |
POForEach.visit(PhyPlanVisitor v) |
void |
POFilter.visit(PhyPlanVisitor v) |
void |
POFRJoin.visit(PhyPlanVisitor v) |
void |
PODistinct.visit(PhyPlanVisitor v) |
void |
PODemux.visit(PhyPlanVisitor v) |
void |
POCross.visit(PhyPlanVisitor v) |
void |
POCounter.visit(PhyPlanVisitor v) |
void |
POCollectedGroup.visit(PhyPlanVisitor v) |
void |
POBroadcastSpark.visit(PhyPlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
static <C extends PhysicalOperator> |
PlanHelper.containsPhysicalOperator(PhysicalPlan plan,
Class<C> opClass) |
static PhysicalPlan |
PlanHelper.getLocalRearrangePlanFromSplit(PhysicalPlan plan,
OperatorKey rearrangeKey)
Finds POLocalRearrange from POSplit sub-plan
|
static <C> LinkedList<C> |
PlanHelper.getPhysicalOperators(PhysicalPlan plan,
Class<C> opClass)
Returns a LinkedList of operators contained within the physical plan which implement the supplied class, in dependency order.
|
Modifier and Type | Method and Description |
---|---|
SparkOperPlan |
SparkLauncher.compile(PhysicalPlan physicalPlan,
PigContext pigContext) |
void |
UDFJarsFinder.visitSparkOp(SparkOperator sparkOp) |
void |
SparkPOUserFuncVisitor.visitSparkOp(SparkOperator sparkOperator) |
void |
JobGraphBuilder.visitSparkOp(SparkOperator sparkOp) |
Modifier and Type | Method and Description |
---|---|
void |
POSampleSortSpark.visit(PhyPlanVisitor v) |
void |
POJoinGroupSpark.visit(PhyPlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
void |
SecondaryKeyOptimizerSpark.visitSparkOp(SparkOperator sparkOperator)
Secondary key sort optimization is enabled in group + foreach nested situation, like TestAccumlator#testAccumWithSort
After calling SecondaryKeyOptimizerUtil.applySecondaryKeySort, the POSort in the POForeach will be deleted in the spark plan.
|
void |
ParallelismSetter.visitSparkOp(SparkOperator sparkOp) |
void |
NoopFilterRemover.visitSparkOp(SparkOperator sparkOp) |
void |
MultiQueryOptimizerSpark.visitSparkOp(SparkOperator sparkOp) |
void |
JoinGroupOptimizerSpark.visitSparkOp(SparkOperator sparkOp) |
void |
CombinerOptimizer.visitSparkOp(SparkOperator sparkOp) |
void |
AccumulatorOptimizer.visitSparkOp(SparkOperator sparkOperator) |
Modifier and Type | Class and Description |
---|---|
class |
SparkCompilerException
Create a new SparkCompilerException with null as the error message.
|
Modifier and Type | Method and Description |
---|---|
void |
SparkCompiler.compile() |
void |
SparkOperator.visit(SparkOpPlanVisitor v) |
void |
SparkCompiler.visitCollectedGroup(POCollectedGroup op) |
void |
SparkCompiler.visitCounter(POCounter op) |
void |
SparkCompiler.visitCross(POCross op) |
void |
SparkCompiler.visitDistinct(PODistinct op) |
void |
SparkCompiler.visitFilter(POFilter op) |
void |
SparkCompiler.visitFRJoin(POFRJoin op) |
void |
SparkCompiler.visitGlobalRearrange(POGlobalRearrange op) |
void |
SparkCompiler.visitLimit(POLimit op) |
void |
SparkCompiler.visitLoad(POLoad op) |
void |
SparkCompiler.visitLocalRearrange(POLocalRearrange op) |
void |
SparkCompiler.visitMergeCoGroup(POMergeCogroup poCoGrp) |
void |
SparkCompiler.visitMergeJoin(POMergeJoin joinOp) |
void |
SparkCompiler.visitNative(PONative op) |
void |
SparkCompiler.visitPackage(POPackage op) |
void |
SparkCompiler.visitPOForEach(POForEach op) |
void |
SparkCompiler.visitRank(PORank op) |
void |
SparkCompiler.visitSkewedJoin(POSkewedJoin op)
currently use regular join to replace skewedJoin
Skewed join currently works with two-table inner join.
|
void |
SparkCompiler.visitSort(POSort op) |
void |
XMLSparkPrinter.visitSparkOp(SparkOperator so) |
void |
SparkPrinter.visitSparkOp(SparkOperator sparkOp) |
void |
SparkPOPackageAnnotator.visitSparkOp(SparkOperator sparkOp) |
void |
SparkOpPlanVisitor.visitSparkOp(SparkOperator sparkOperator) |
void |
SparkCompiler.visitSplit(POSplit op) |
void |
SparkCompiler.visitStore(POStore op) |
void |
SparkCompiler.visitStream(POStream op) |
void |
SparkCompiler.visitUnion(POUnion op) |
Modifier and Type | Method and Description |
---|---|
TezPlanContainer |
TezLauncher.compile(PhysicalPlan php,
PigContext pc) |
void |
TezLauncher.explain(PhysicalPlan php,
PigContext pc,
PrintStream ps,
String format,
boolean verbose) |
static void |
TezLauncher.processLoadAndParallelism(TezOperPlan tezPlan,
PigContext pc) |
void |
TezDagBuilder.visitTezOp(TezOperator tezOp) |
Modifier and Type | Class and Description |
---|---|
class |
TezCompilerException |
Modifier and Type | Method and Description |
---|---|
TezOperPlan |
TezCompiler.compile()
The front-end method that the user calls to compile the plan.
|
double |
TezOperator.getParallelismFactor(TezOperator successor) |
void |
TezOperator.visit(TezOpPlanVisitor v) |
void |
TezPlanContainerNode.visit(TezPlanContainerVisitor v) |
void |
TezCompiler.visitCollectedGroup(POCollectedGroup op) |
void |
TezCompiler.visitCounter(POCounter op) |
void |
TezCompiler.visitCross(POCross op) |
void |
TezCompiler.visitDistinct(PODistinct op) |
void |
TezCompiler.visitFilter(POFilter op) |
void |
TezCompiler.visitFRJoin(POFRJoin op) |
void |
TezCompiler.visitGlobalRearrange(POGlobalRearrange op) |
void |
TezCompiler.visitLimit(POLimit op) |
void |
TezCompiler.visitLoad(POLoad op) |
void |
TezPOPackageAnnotator.LoRearrangeDiscoverer.visitLocalRearrange(POLocalRearrange lrearrange) |
void |
TezCompiler.visitLocalRearrange(POLocalRearrange op) |
void |
TezCompiler.visitMergeCoGroup(POMergeCogroup poCoGrp) |
void |
TezCompiler.visitMergeJoin(POMergeJoin joinOp)
Since merge-join works on two inputs there are exactly two TezOper predecessors identified as left and right.
|
void |
TezCompiler.visitNative(PONative op) |
void |
TezCompiler.visitPackage(POPackage op) |
void |
TezCompiler.visitPOForEach(POForEach op) |
void |
TezCompiler.visitRank(PORank op) |
void |
TezCompiler.visitSkewedJoin(POSkewedJoin op) |
void |
TezCompiler.visitSort(POSort op) |
void |
TezPOPackageAnnotator.LoRearrangeDiscoverer.visitSplit(POSplit spl) |
void |
TezCompiler.visitSplit(POSplit op) |
void |
TezCompiler.visitStore(POStore op) |
void |
TezCompiler.visitStream(POStream op) |
void |
TezPrinter.visitTezOp(TezOperator tezOper) |
void |
TezPrinter.TezVertexGraphPrinter.visitTezOp(TezOperator tezOper) |
void |
TezPOUserFuncVisitor.visitTezOp(TezOperator tezOp) |
void |
TezPOStreamVisitor.visitTezOp(TezOperator tezOp) |
void |
TezPOPackageAnnotator.visitTezOp(TezOperator tezOp) |
void |
TezOpPlanVisitor.visitTezOp(TezOperator tezOperator) |
void |
TezPrinter.TezDAGGraphPrinter.visitTezPlanContainerNode(TezPlanContainerNode tezPlanContainerNode) |
void |
TezPlanContainerVisitor.visitTezPlanContainerNode(TezPlanContainerNode tezPlanContainerNode) |
void |
TezPlanContainerUDFCollector.visitTezPlanContainerNode(TezPlanContainerNode tezPlanContainerNode) |
void |
TezPlanContainerPrinter.visitTezPlanContainerNode(TezPlanContainerNode tezPlanContainerNode) |
void |
TezCompiler.visitUnion(POUnion op) |
Modifier and Type | Method and Description |
---|---|
void |
POValueOutputTez.visit(PhyPlanVisitor v) |
void |
POValueInputTez.visit(PhyPlanVisitor v) |
void |
POShuffledValueInputTez.visit(PhyPlanVisitor v) |
void |
PORankTez.visit(PhyPlanVisitor v) |
void |
POCounterTez.visit(PhyPlanVisitor v) |
void |
POCounterStatsTez.visit(PhyPlanVisitor v) |
void |
NativeTezOper.visit(TezOpPlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
void |
UnionOptimizer.connectPredecessorsToVertexGroups(TezOperator unionOp,
TezOperator pred,
PhysicalPlan predClonedUnionPlan,
TezOperator[] storeVertexGroupOps,
TezOperator[] outputVertexGroupOps)
Connects the unionOp predecessor to the store vertex groups and the output vertex groups
and disconnects it from the unionOp.
|
static PhysicalPlan |
UnionOptimizer.getUnionPredPlanFromSplit(PhysicalPlan plan,
String unionOpKey) |
static boolean |
UnionOptimizer.isOptimizable(TezOperator tezOp) |
static boolean |
UnionOptimizer.isOptimizableStoreFunc(TezOperator tezOp,
List<String> supportedStoreFuncs,
List<String> unsupportedStoreFuncs) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitFilter(POFilter fl) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitFRJoin(POFRJoin join) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitLimit(POLimit lim) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitMergeJoin(POMergeJoin join) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitPackage(POPackage pkg) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitPOForEach(POForEach nfe) |
void |
TezOperDependencyParallelismEstimator.TezParallelismFactorVisitor.visitSplit(POSplit sp) |
void |
UnionOptimizer.visitTezOp(TezOperator tezOp) |
void |
TezEstimatedParallelismClearer.visitTezOp(TezOperator tezOp) |
void |
SecondaryKeyOptimizerTez.visitTezOp(TezOperator to) |
void |
ParallelismSetter.visitTezOp(TezOperator tezOp) |
void |
NoopFilterRemover.visitTezOp(TezOperator tezOp) |
void |
MultiQueryOptimizerTez.visitTezOp(TezOperator tezOp) |
void |
LoaderProcessor.visitTezOp(TezOperator tezOp) |
void |
CombinerOptimizer.visitTezOp(TezOperator to) |
void |
AccumulatorOptimizer.visitTezOp(TezOperator tezOp) |
Constructor and Description |
---|
LoaderProcessor(TezOperPlan plan,
PigContext pigContext) |
Modifier and Type | Method and Description |
---|---|
void |
TezUDFContextSeparator.visitTezOp(TezOperator tezOperator) |
Modifier and Type | Method and Description |
---|---|
static void |
CombinerOptimizerUtil.addCombiner(PhysicalPlan mapPlan,
PhysicalPlan reducePlan,
PhysicalPlan combinePlan,
CompilationMessageCollector messageCollector,
boolean doMapAgg)
Algebraic functions and distinct in nested plan of a foreach are
partially computed in the map and combine phase.
|
SecondaryKeyOptimizerUtil.SecondaryKeyOptimizerInfo |
SecondaryKeyOptimizerUtil.applySecondaryKeySort(PhysicalPlan mapPlan,
PhysicalPlan reducePlan) |
static List<Pair<PhysicalOperator,PhysicalPlan>> |
CombinerOptimizerUtil.findAlgebraicOps(List<PhysicalPlan> feInners)
find algebraic operators and also check if the foreach statement is
suitable for combiner use
|
void |
ParallelConstantVisitor.visitConstant(ConstantExpression cnst) |
void |
CombinerOptimizerUtil.DistinctPatcher.visitProject(POProject proj) |
void |
CombinerOptimizerUtil.fixMapProjects.visitProject(POProject proj) |
Modifier and Type | Class and Description |
---|---|
class |
TypeCheckerException |
Modifier and Type | Class and Description |
---|---|
class |
PlanValidationException |
Modifier and Type | Method and Description |
---|---|
protected void |
DependencyOrderWalker.doAllPredecessors(O node,
Set<O> seen,
Collection<O> fifo) |
protected void |
ReverseDependencyOrderWalker.doAllSuccessors(O node,
Set<O> seen,
Collection<O> fifo) |
protected void |
PlanVisitor.popWalker()
Pop the next to previous walker off of the stack and set it as the current
walker.
|
void |
PlanVisitor.visit()
Entry point for visiting the plan.
|
abstract void |
Operator.visit(V v)
Visit this node with the provided visitor.
|
void |
ReverseDependencyOrderWalker.walk(PlanVisitor<O,P> visitor)
Begin traversing the graph.
|
abstract void |
PlanWalker.walk(PlanVisitor<O,P> visitor)
Begin traversing the graph.
|
void |
DepthFirstWalker.walk(PlanVisitor<O,P> visitor)
Begin traversing the graph.
|
void |
DependencyOrderWalker.walk(PlanVisitor<O,P> visitor)
Begin traversing the graph.
|
Modifier and Type | Class and Description |
---|---|
class |
OptimizerException |
Modifier and Type | Method and Description |
---|---|
void |
UDFContextSeparator.visitLoad(POLoad ld) |
void |
UDFContextSeparator.visitStore(POStore st) |
void |
UDFContextSeparator.visitUserFunc(POUserFunc userFunc) |
Modifier and Type | Method and Description |
---|---|
void |
TypeCheckingRelVisitor.visit(LOCross cs)
Return concatenated of all fields from all input operators
If one of the inputs have no schema then we cannot construct
the output schema.
|
void |
TypeCheckingRelVisitor.visit(LODistinct op)
LODistinct, output schema should be the same as input
|
void |
TypeCheckingRelVisitor.visit(LOSplit split)
The schema of split output will be the same as split input
|
Modifier and Type | Method and Description |
---|---|
void |
IllustratorAttacher.revisit(PhysicalPlan plan)
revisit an enhanced physical plan from MR compilation
|
void |
IllustratorAttacher.visitAnd(POAnd and) |
void |
IllustratorAttacher.visitComparisonFunc(POUserComparisonFunc compFunc) |
void |
IllustratorAttacher.visitCounter(POCounter counter) |
void |
IllustratorAttacher.visitDemux(PODemux demux) |
void |
IllustratorAttacher.visitDistinct(PODistinct distinct) |
void |
IllustratorAttacher.visitEqualTo(EqualToExpr eq) |
void |
IllustratorAttacher.visitFilter(POFilter fl) |
void |
IllustratorAttacher.visitGreaterThan(GreaterThanExpr grt) |
void |
IllustratorAttacher.visitGTOrEqual(GTOrEqualToExpr gte) |
void |
IllustratorAttacher.visitIsNull(POIsNull isNull) |
void |
IllustratorAttacher.visitLessThan(LessThanExpr lt) |
void |
IllustratorAttacher.visitLimit(POLimit lim) |
void |
IllustratorAttacher.visitLoad(POLoad ld) |
void |
IllustratorAttacher.visitLocalRearrange(POLocalRearrange lr) |
void |
IllustratorAttacher.visitLTOrEqual(LTOrEqualToExpr lte) |
void |
IllustratorAttacher.visitNot(PONot not) |
void |
IllustratorAttacher.visitNotEqualTo(NotEqualToExpr eq) |
void |
IllustratorAttacher.visitOr(POOr or) |
void |
PhysicalPlanResetter.visitPackage(POPackage pkg) |
void |
IllustratorAttacher.visitPackage(POPackage pkg) |
void |
IllustratorAttacher.visitPOForEach(POForEach nfe) |
void |
IllustratorAttacher.visitPOOptimizedForEach(POOptimizedForEach optimizedForEach) |
void |
IllustratorAttacher.visitProject(POProject proj) |
void |
IllustratorAttacher.visitRank(PORank rank) |
void |
IllustratorAttacher.visitRegexp(PORegexp re) |
void |
IllustratorAttacher.visitSort(POSort sort) |
void |
IllustratorAttacher.visitSplit(POSplit spl) |
void |
IllustratorAttacher.visitStore(POStore st) |
void |
IllustratorAttacher.visitStream(POStream stream) |
void |
IllustratorAttacher.visitUnion(POUnion un) |
void |
IllustratorAttacher.visitUserFunc(POUserFunc userFunc) |
Constructor and Description |
---|
IllustratorAttacher(PhysicalPlan plan,
LineageTracer lineage,
int maxRecords,
Map<POLoad,LogicalSchema> poLoadToSchemaMap,
PigContext hadoopPigContext) |
Modifier and Type | Method and Description |
---|---|
void |
ScriptState.FeatureVisitor.visitCollectedGroup(POCollectedGroup mg) |
void |
ScriptState.AliasVisitor.visitCollectedGroup(POCollectedGroup mg) |
void |
ScriptState.FeatureVisitor.visitDemux(PODemux demux) |
void |
ScriptState.FeatureVisitor.visitDistinct(PODistinct distinct) |
void |
ScriptState.AliasVisitor.visitDistinct(PODistinct distinct) |
void |
ScriptState.AliasVisitor.visitFilter(POFilter fl) |
void |
ScriptState.FeatureVisitor.visitFRJoin(POFRJoin join) |
void |
ScriptState.AliasVisitor.visitFRJoin(POFRJoin join) |
void |
ScriptState.AliasVisitor.visitLimit(POLimit lim) |
void |
ScriptState.AliasVisitor.visitLoad(POLoad load) |
void |
ScriptState.AliasVisitor.visitLocalRearrange(POLocalRearrange lr) |
void |
ScriptState.FeatureVisitor.visitMergeCoGroup(POMergeCogroup mergeCoGrp) |
void |
ScriptState.AliasVisitor.visitMergeCoGroup(POMergeCogroup mergeCoGrp) |
void |
ScriptState.FeatureVisitor.visitMergeJoin(POMergeJoin join) |
void |
ScriptState.AliasVisitor.visitMergeJoin(POMergeJoin join) |
void |
ScriptState.AliasVisitor.visitPOForEach(POForEach nfe) |
void |
ScriptState.AliasVisitor.visitSkewedJoin(POSkewedJoin sk) |
void |
ScriptState.AliasVisitor.visitSort(POSort sort) |
void |
ScriptState.FeatureVisitor.visitSplit(POSplit split) |
void |
ScriptState.FeatureVisitor.visitStream(POStream stream) |
void |
ScriptState.AliasVisitor.visitStream(POStream stream) |
void |
ScriptState.AliasVisitor.visitUnion(POUnion un) |
Copyright © 2007-2017 The Apache Software Foundation