Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
HExecutionEngine.compile(LogicalPlan plan,
Properties properties) |
Modifier and Type | Method and Description |
---|---|
abstract void |
Launcher.explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose)
Explain how a pig job will be executed on the underlying infrastructure.
|
abstract PigStats |
Launcher.launchPig(PhysicalPlan php,
String grpName,
PigContext pc)
Method to launch pig for hadoop either for a cluster's job tracker or for
a local job runner.
|
Modifier and Type | Method and Description |
---|---|
void |
FetchLauncher.explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format)
Creates an empty MR plan
|
static boolean |
FetchOptimizer.isPlanFetchable(PigContext pc,
PhysicalPlan pp)
Visits the plan with
FetchablePlanVisitor and checks whether the
plan is fetchable. |
PigStats |
FetchLauncher.launchPig(PhysicalPlan pp)
Runs the fetch task by executing chain of calls on the PhysicalPlan from the leaf
up to the LoadFunc
|
Modifier and Type | Field and Description |
---|---|
PhysicalPlan |
MapReduceOper.combinePlan |
PhysicalPlan |
MapReduceOper.mapPlan |
protected PhysicalPlan |
PigGenericMapBase.mp |
PhysicalPlan |
MapReduceOper.reducePlan |
protected PhysicalPlan |
PigGenericMapReduce.Reduce.rp |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
MRCompiler.getPlan()
Used to get the plan that was compiled
|
Modifier and Type | Method and Description |
---|---|
MROperPlan |
MapReduceLauncher.compile(PhysicalPlan php,
PigContext pc) |
void |
MapReduceLauncher.explain(PhysicalPlan php,
PigContext pc,
PrintStream ps,
String format,
boolean verbose) |
PigStats |
MapReduceLauncher.launchPig(PhysicalPlan php,
String grpName,
PigContext pc) |
static void |
NoopFilterRemoverUtil.removeFilter(POFilter filter,
PhysicalPlan plan) |
void |
PigGenericMapBase.setMapPlan(PhysicalPlan plan)
for local map/reduce simulation
|
void |
PigGenericMapReduce.Reduce.setReducePlan(PhysicalPlan plan)
Set the reduce plan: to be used by local runner for illustrator
|
Constructor and Description |
---|
MRCompiler(PhysicalPlan plan) |
MRCompiler(PhysicalPlan plan,
PigContext pigContext) |
PhyPlanSetter(PhysicalPlan plan) |
UDFEndOfAllInputNeededVisitor(PhysicalPlan plan) |
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Constructor and Description |
---|
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
DotMRPrinter.InnerOperator.getPlan() |
Modifier and Type | Method and Description |
---|---|
void |
UDFFinder.setPlan(PhysicalPlan plan) |
Constructor and Description |
---|
EndOfAllInputChecker(PhysicalPlan plan) |
InnerOperator(PhysicalPlan plan,
String name) |
InnerPlan(PhysicalPlan mapPlan,
PhysicalPlan combinePlan,
PhysicalPlan reducePlan) |
ScalarPhyFinder(PhysicalPlan plan) |
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Constructor and Description |
---|
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Field and Description |
---|---|
protected PhysicalPlan |
PhysicalOperator.parentPlan |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
PhysicalOperator.getParentPlan() |
Modifier and Type | Method and Description |
---|---|
protected static List<PhysicalPlan> |
PhysicalOperator.clonePlans(List<PhysicalPlan> origPlans) |
Modifier and Type | Method and Description |
---|---|
void |
PhysicalOperator.setParentPlan(PhysicalPlan physicalPlan) |
Modifier and Type | Method and Description |
---|---|
protected static List<PhysicalPlan> |
PhysicalOperator.clonePlans(List<PhysicalPlan> origPlans) |
Constructor and Description |
---|
POPrinter(PrintStream ps,
PhysicalPlan pp) |
Modifier and Type | Method and Description |
---|---|
void |
UnaryExpressionOperator.setInputAsExpr(PhysicalPlan plan)
Set the contained expression to the be the input value.
|
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
PhysicalPlan.clone() |
Modifier and Type | Method and Description |
---|---|
protected Collection<PhysicalPlan> |
DotPOPrinter.getMultiOutputNestedPlans(PhysicalOperator op) |
protected Collection<PhysicalPlan> |
DotPOPrinter.getNestedPlans(PhysicalOperator op) |
Modifier and Type | Method and Description |
---|---|
protected DotPlanDumper |
DotPOPrinter.makeDumper(PhysicalPlan plan,
PrintStream ps) |
Constructor and Description |
---|
DotPOPrinter(PhysicalPlan plan,
PrintStream ps) |
DotPOPrinter(PhysicalPlan plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> subgraphs,
Set<Operator> multiInputSubgraphs,
Set<Operator> multiOutputSubgraphs) |
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
UdfCacheShipFilesVisitor(PhysicalPlan plan) |
XMLPhysicalPlanPrinter(PhysicalPlan plan,
Document doc,
Element parent) |
Constructor and Description |
---|
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Field and Description |
---|---|
protected List<PhysicalPlan> |
POForEach.inputPlans |
protected List<List<PhysicalPlan>> |
POFRJoin.phyPlanLists |
protected List<PhysicalPlan> |
POPreCombinerLocalRearrange.plans |
protected List<PhysicalPlan> |
POLocalRearrange.plans |
protected List<PhysicalPlan> |
POCollectedGroup.plans |
protected List<PhysicalPlan> |
POLocalRearrange.secondaryPlans |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
POPartialAgg.getKeyPlan() |
PhysicalPlan |
POLimit.getLimitPlan() |
PhysicalPlan |
POFilter.getPlan() |
Modifier and Type | Method and Description |
---|---|
List<PhysicalPlan> |
POCounter.getCounterPlans() |
List<PhysicalPlan> |
POMergeJoin.getInnerPlansOf(int index) |
List<PhysicalPlan> |
POForEach.getInputPlans() |
List<PhysicalPlan> |
JoinPackager.getInputPlans() |
MultiMap<PhysicalOperator,PhysicalPlan> |
POSkewedJoin.getJoinPlans() |
List<List<PhysicalPlan>> |
POFRJoin.getJoinPlans() |
List<PhysicalPlan> |
POMergeCogroup.getLRInnerPlansOf(int i) |
List<PhysicalPlan> |
POSplit.getPlans()
Returns the list of nested plans.
|
List<PhysicalPlan> |
POPreCombinerLocalRearrange.getPlans() |
List<PhysicalPlan> |
POLocalRearrange.getPlans() |
List<PhysicalPlan> |
PODemux.getPlans()
Returns the list of inner plans.
|
List<PhysicalPlan> |
POCollectedGroup.getPlans() |
List<PhysicalPlan> |
PORank.getRankPlans() |
List<PhysicalPlan> |
POSort.getSortPlans() |
List<PhysicalPlan> |
POPartialAgg.getValuePlans() |
Modifier and Type | Method and Description |
---|---|
void |
POForEach.addInputPlan(PhysicalPlan plan,
boolean flatten) |
void |
POSplit.addPlan(PhysicalPlan inPlan)
Appends the specified plan to the end of
the nested input plan list
|
void |
PODemux.addPlan(PhysicalPlan inPlan)
Appends the specified plan at the end of the list.
|
void |
POSplit.removePlan(PhysicalPlan plan)
Removes plan from
the nested input plan list
|
void |
POPartialAgg.setKeyPlan(PhysicalPlan keyPlan) |
void |
POLimit.setLimitPlan(PhysicalPlan expressionPlan) |
void |
POFilter.setPlan(PhysicalPlan plan) |
void |
POMergeJoin.setupRightPipeline(PhysicalPlan rightPipeline) |
Modifier and Type | Method and Description |
---|---|
void |
POCounter.setCounterPlans(List<PhysicalPlan> counterPlans) |
void |
POForEach.setInputPlans(List<PhysicalPlan> plans) |
void |
JoinPackager.setInputPlans(List<PhysicalPlan> plans) |
void |
POSkewedJoin.setJoinPlans(MultiMap<PhysicalOperator,PhysicalPlan> joinPlans) |
void |
POPreCombinerLocalRearrange.setPlans(List<PhysicalPlan> plans) |
void |
POLocalRearrange.setPlans(List<PhysicalPlan> plans) |
void |
POCollectedGroup.setPlans(List<PhysicalPlan> plans) |
void |
POLocalRearrange.setPlansFromCombiner(List<PhysicalPlan> plans) |
void |
PORank.setRankPlans(List<PhysicalPlan> rankPlans) |
void |
POLocalRearrange.setSecondaryPlans(List<PhysicalPlan> plans) |
void |
POSort.setSortPlans(List<PhysicalPlan> sortPlans) |
void |
POPartialAgg.setValuePlans(List<PhysicalPlan> valuePlans) |
Constructor and Description |
---|
POCounter(OperatorKey operatorKey,
int requestedParallelism,
List inp,
List<PhysicalPlan> counterPlans,
List<Boolean> ascendingCol) |
POForEach(OperatorKey k,
int rp,
List<PhysicalPlan> inp,
List<Boolean> isToBeFlattened) |
POForEach(OperatorKey operatorKey,
int requestedParallelism,
List<PhysicalPlan> innerPlans,
List<Boolean> flattenList,
Schema schema) |
POFRJoin(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
List<List<PhysicalPlan>> ppLists,
List<List<Byte>> keyTypes,
FileSpec[] replFiles,
int fragment,
boolean isLeftOuter,
Tuple nullTuple) |
POFRJoin(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
List<List<PhysicalPlan>> ppLists,
List<List<Byte>> keyTypes,
FileSpec[] replFiles,
int fragment,
boolean isLeftOuter,
Tuple nullTuple,
Schema[] inputSchemas,
Schema[] keySchemas) |
POMergeJoin(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
MultiMap<PhysicalOperator,PhysicalPlan> inpPlans,
List<List<Byte>> keyTypes,
LOJoin.JOINTYPE joinType,
Schema leftInputSchema,
Schema rightInputSchema,
Schema mergedInputSchema) |
POOptimizedForEach(OperatorKey k,
int rp,
List<PhysicalPlan> inp,
List<Boolean> isToBeFlattened) |
PORank(OperatorKey operatorKey,
int requestedParallelism,
List inp,
List<PhysicalPlan> rankPlans,
List<Boolean> ascendingCol) |
POSort(OperatorKey k,
int rp,
List inp,
List<PhysicalPlan> sortPlans,
List<Boolean> mAscCols,
POUserComparisonFunc mSortFunc) |
Modifier and Type | Method and Description |
---|---|
static PhysicalPlan |
PlanHelper.getLocalRearrangePlanFromSplit(PhysicalPlan plan,
OperatorKey rearrangeKey)
Finds POLocalRearrange from POSplit sub-plan
|
Modifier and Type | Method and Description |
---|---|
static <C extends PhysicalOperator> |
PlanHelper.containsPhysicalOperator(PhysicalPlan plan,
Class<C> opClass) |
static PhysicalPlan |
PlanHelper.getLocalRearrangePlanFromSplit(PhysicalPlan plan,
OperatorKey rearrangeKey)
Finds POLocalRearrange from POSplit sub-plan
|
static <C> LinkedList<C> |
PlanHelper.getPhysicalOperators(PhysicalPlan plan,
Class<C> opClass)
Returns a LinkedList of operators contained within the physical plan which implement the supplied class, in dependency order.
|
Modifier and Type | Method and Description |
---|---|
SparkOperPlan |
SparkLauncher.compile(PhysicalPlan physicalPlan,
PigContext pigContext) |
void |
SparkLauncher.explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose) |
PigStats |
SparkLauncher.launchPig(PhysicalPlan physicalPlan,
String grpName,
PigContext pigContext) |
static org.apache.hadoop.mapred.JobConf |
SparkUtil.newJobConf(PigContext pigContext,
PhysicalPlan physicalPlan,
SparkEngineConf sparkEngineConf) |
Constructor and Description |
---|
LoadConverter(PigContext pigContext,
PhysicalPlan physicalPlan,
SparkContext sparkContext,
org.apache.hadoop.mapred.JobConf jobConf,
SparkEngineConf sparkEngineConf) |
Constructor and Description |
---|
POReduceBySpark(OperatorKey k,
int rp,
List<PhysicalPlan> inp,
List<Boolean> isToBeFlattened,
POPackage pkg,
POLocalRearrange lr) |
Modifier and Type | Method and Description |
---|---|
protected PhysicalOperator |
SparkSecondaryKeyOptimizerUtil.getCurrentNode(PhysicalOperator root,
PhysicalPlan reducePlan) |
Modifier and Type | Field and Description |
---|---|
PhysicalPlan |
SparkOperator.physicalPlan |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
DotSparkPrinter.InnerOperator.getPlan() |
Constructor and Description |
---|
InnerOperator(PhysicalPlan plan,
String name) |
InnerPlan(PhysicalPlan plan) |
SparkCompiler(PhysicalPlan physicalPlan,
PigContext pigContext) |
Modifier and Type | Method and Description |
---|---|
TezPlanContainer |
TezLauncher.compile(PhysicalPlan php,
PigContext pc) |
void |
TezLauncher.explain(PhysicalPlan php,
PigContext pc,
PrintStream ps,
String format,
boolean verbose) |
PigStats |
TezLauncher.launchPig(PhysicalPlan php,
String grpName,
PigContext pc) |
Modifier and Type | Field and Description |
---|---|
PhysicalPlan |
TezEdgeDescriptor.combinePlan |
PhysicalPlan |
TezOperator.plan |
Modifier and Type | Method and Description |
---|---|
static Pair<POProject,Byte>[] |
TezCompiler.getSortCols(List<PhysicalPlan> plans) |
Constructor and Description |
---|
LoRearrangeDiscoverer(PhysicalPlan predPlan,
TezOperator predTezOpVertexGrp,
TezOperator pkgTezOp,
POPackage pkg) |
TezCompiler(PhysicalPlan plan,
PigContext pigContext) |
Modifier and Type | Method and Description |
---|---|
POLocalRearrangeTez |
POLocalRearrangeTezFactory.create(int index,
POLocalRearrangeTezFactory.LocalRearrangeType type,
List<PhysicalPlan> plans,
byte keyType) |
Modifier and Type | Method and Description |
---|---|
static PhysicalPlan |
UnionOptimizer.getUnionPredPlanFromSplit(PhysicalPlan plan,
String unionOpKey) |
Modifier and Type | Method and Description |
---|---|
void |
UnionOptimizer.connectPredecessorsToVertexGroups(TezOperator unionOp,
TezOperator pred,
PhysicalPlan predClonedUnionPlan,
TezOperator[] storeVertexGroupOps,
TezOperator[] outputVertexGroupOps)
Connects the unionOp predecessor to the store vertex groups and the output vertex groups
and disconnects it from the unionOp.
|
static PhysicalPlan |
UnionOptimizer.getUnionPredPlanFromSplit(PhysicalPlan plan,
String unionOpKey) |
Modifier and Type | Method and Description |
---|---|
static boolean |
TezCompilerUtil.bagDataTypeInCombinePlan(PhysicalPlan combinePlan) |
Modifier and Type | Method and Description |
---|---|
static PhysicalPlan |
CombinerOptimizerUtil.createPlanWithPredecessors(PhysicalOperator algeOp,
PhysicalPlan pplan)
Create new plan and add to it the clones of operator algeOp and its
predecessors from the physical plan pplan .
|
Modifier and Type | Method and Description |
---|---|
static List<Pair<PhysicalOperator,PhysicalPlan>> |
CombinerOptimizerUtil.findAlgebraicOps(List<PhysicalPlan> feInners)
find algebraic operators and also check if the foreach statement is
suitable for combiner use
|
Modifier and Type | Method and Description |
---|---|
static void |
AccumulatorOptimizerUtil.addAccumulator(PhysicalPlan plan,
List<PhysicalOperator> pos) |
static void |
CombinerOptimizerUtil.addCombiner(PhysicalPlan mapPlan,
PhysicalPlan reducePlan,
PhysicalPlan combinePlan,
CompilationMessageCollector messageCollector,
boolean doMapAgg)
Algebraic functions and distinct in nested plan of a foreach are
partially computed in the map and combine phase.
|
SecondaryKeyOptimizerUtil.SecondaryKeyOptimizerInfo |
SecondaryKeyOptimizerUtil.applySecondaryKeySort(PhysicalPlan mapPlan,
PhysicalPlan reducePlan) |
static FileSpec |
MapRedUtil.checkLeafIsStore(PhysicalPlan plan,
PigContext pigContext) |
static PhysicalPlan |
CombinerOptimizerUtil.createPlanWithPredecessors(PhysicalOperator algeOp,
PhysicalPlan pplan)
Create new plan and add to it the clones of operator algeOp and its
predecessors from the physical plan pplan .
|
protected PhysicalOperator |
SecondaryKeyOptimizerUtil.getCurrentNode(PhysicalOperator root,
PhysicalPlan reducePlan) |
Modifier and Type | Method and Description |
---|---|
static List<Pair<PhysicalOperator,PhysicalPlan>> |
CombinerOptimizerUtil.findAlgebraicOps(List<PhysicalPlan> feInners)
find algebraic operators and also check if the foreach statement is
suitable for combiner use
|
Constructor and Description |
---|
DistinctPatcher(PhysicalPlan physicalPlan) |
DistinctPatcher(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
fixMapProjects(PhysicalPlan plan) |
fixMapProjects(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
ParallelConstantVisitor(PhysicalPlan plan,
int rp) |
Constructor and Description |
---|
DistinctPatcher(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
fixMapProjects(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Method and Description |
---|---|
static void |
CompilerUtils.addEmptyBagOuterJoin(PhysicalPlan fePlan,
Schema inputSchema,
boolean skewedRightOuterJoin,
String isFirstReduceOfKeyClassName) |
void |
UDFContextSeparator.setPlan(PhysicalPlan plan,
String planOpKey) |
Modifier and Type | Field and Description |
---|---|
protected PhysicalPlan |
ExpToPhyTranslationVisitor.currentPlan |
Modifier and Type | Field and Description |
---|---|
protected Deque<PhysicalPlan> |
ExpToPhyTranslationVisitor.currentPlans |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
ExpToPhyTranslationVisitor.getPhysicalPlan() |
Constructor and Description |
---|
ExpToPhyTranslationVisitor(OperatorPlan plan,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
ExpToPhyTranslationVisitor(OperatorPlan plan,
PlanWalker walker,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
Modifier and Type | Field and Description |
---|---|
protected PhysicalPlan |
LogToPhyTranslationVisitor.currentPlan |
Modifier and Type | Field and Description |
---|---|
protected Deque<PhysicalPlan> |
LogToPhyTranslationVisitor.currentPlans |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
LogToPhyTranslationVisitor.getPhysicalPlan() |
Modifier and Type | Method and Description |
---|---|
static void |
LogToPhyTranslationVisitor.updateWithEmptyBagCheck(PhysicalPlan fePlan,
Operator joinInput)
updates plan with check for empty bag and if bag is empty to flatten a bag
with as many null's as dictated by the schema
|
Modifier and Type | Method and Description |
---|---|
static Map<LogicalRelationalOperator,Collection<IdentityHashSet<Tuple>>> |
EquivalenceClasses.getLoToEqClassMap(PhysicalPlan plan,
LogicalPlan lp,
Map<Operator,PhysicalOperator> logToPhyMap,
Map<Operator,DataBag> logToDataMap,
Map<LOForEach,Map<LogicalRelationalOperator,PhysicalOperator>> forEachInnerLogToPhyMap,
HashMap<PhysicalOperator,Collection<IdentityHashSet<Tuple>>> poToEqclassesMap) |
void |
LocalMapReduceSimulator.launchPig(PhysicalPlan php,
Map<LOLoad,DataBag> baseData,
LineageTracer lineage,
IllustratorAttacher attacher,
ExampleGenerator eg,
PigContext pc) |
void |
IllustratorAttacher.revisit(PhysicalPlan plan)
revisit an enhanced physical plan from MR compilation
|
Constructor and Description |
---|
IllustratorAttacher(PhysicalPlan plan,
LineageTracer lineage,
int maxRecords,
Map<POLoad,LogicalSchema> poLoadToSchemaMap,
PigContext hadoopPigContext) |
LineageTrimmingVisitor(LogicalPlan plan,
Map<LOLoad,DataBag> baseData,
ExampleGenerator eg,
Map<Operator,PhysicalOperator> LogToPhyMap,
PhysicalPlan physPlan,
PigContext pc) |
PhysicalPlanResetter(PhysicalPlan plan) |
Constructor and Description |
---|
AliasVisitor(PhysicalPlan plan,
List<String> alias,
List<String> aliasLocation) |
FeatureVisitor(PhysicalPlan plan,
BitSet feature) |
Copyright © 2007-2017 The Apache Software Foundation