Modifier and Type | Field and Description |
---|---|
protected Map<Operator,PhysicalOperator> |
HExecutionEngine.newLogToPhyMap |
Modifier and Type | Method and Description |
---|---|
Map<LOForEach,Map<LogicalRelationalOperator,PhysicalOperator>> |
HExecutionEngine.getForEachInnerLogToPhyMap(LogicalPlan plan) |
Map<Operator,PhysicalOperator> |
HExecutionEngine.getLogToPhyMap() |
Modifier and Type | Field and Description |
---|---|
static PhysicalOperator |
PigMapReduceCounter.PigReduceCounter.leaf |
Modifier and Type | Field and Description |
---|---|
static List<PhysicalOperator> |
PigMapReduceCounter.PigReduceCounter.leaves |
MultiMap<PhysicalOperator,PhysicalOperator> |
MapReduceOper.phyToMRMap |
MultiMap<PhysicalOperator,PhysicalOperator> |
MapReduceOper.phyToMRMap |
Set<PhysicalOperator> |
MapReduceOper.scalars |
Modifier and Type | Method and Description |
---|---|
protected void |
PigGenericMapReduce.MapWithPartitionIndex.runPipeline(PhysicalOperator leaf) |
protected void |
PigGenericMapReduce.Reduce.runPipeline(PhysicalOperator leaf) |
protected void |
PigGenericMapBase.runPipeline(PhysicalOperator leaf) |
void |
PhyPlanSetter.visit(PhysicalOperator op) |
Constructor and Description |
---|
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Method and Description |
---|---|
List<PhysicalOperator> |
ScalarPhyFinder.getScalars() |
Constructor and Description |
---|
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Field and Description |
---|---|
protected List<PhysicalOperator> |
PhysicalOperator.inputs |
protected List<PhysicalOperator> |
PhysicalOperator.outputs |
Modifier and Type | Method and Description |
---|---|
PhysicalOperator |
PhysicalOperator.clone()
Make a copy of this operator.
|
Modifier and Type | Method and Description |
---|---|
List<PhysicalOperator> |
PhysicalOperator.getInputs() |
Modifier and Type | Method and Description |
---|---|
protected void |
PhysicalOperator.cloneHelper(PhysicalOperator op) |
void |
PhysicalOperator.copyAliasFrom(PhysicalOperator op) |
Modifier and Type | Method and Description |
---|---|
void |
PhysicalOperator.setInputs(List<PhysicalOperator> inputs) |
Constructor and Description |
---|
PhysicalOperator(PhysicalOperator copy) |
Constructor and Description |
---|
PhysicalOperator(OperatorKey k,
int rp,
List<PhysicalOperator> inp) |
PhysicalOperator(OperatorKey k,
List<PhysicalOperator> inp) |
Modifier and Type | Class and Description |
---|---|
class |
Add |
class |
BinaryComparisonOperator
This is a base class for all binary comparison operators.
|
class |
BinaryExpressionOperator
A base class for all Binary expression operators.
|
class |
ConstantExpression
This class implements a Constant of any type.
|
class |
Divide |
class |
EqualToExpr |
class |
ExpressionOperator
A base class for all types of expressions.
|
class |
GreaterThanExpr |
class |
GTOrEqualToExpr |
class |
LessThanExpr |
class |
LTOrEqualToExpr |
class |
Mod |
class |
Multiply |
class |
NotEqualToExpr |
class |
POAnd
Boolean and operator.
|
class |
POBinCond |
class |
POCast
This is just a cast that converts DataByteArray into either String or
Integer.
|
class |
POIsNull |
class |
POMapLookUp |
class |
PONegative |
class |
PONot
Boolean not operator.
|
class |
POOr
Boolean or operator.
|
class |
POProject
Implements the overloaded form of the project operator.
|
class |
PORegexp |
class |
PORelationToExprProject
Implements a specialized form of POProject which is
used *ONLY* in the following case:
This project is Project(*) introduced after a relational operator
to supply a bag as output (as an expression).
|
class |
POUserComparisonFunc |
class |
POUserFunc |
class |
Subtract |
class |
UnaryComparisonOperator
This is a base class for all unary comparison operators.
|
class |
UnaryExpressionOperator |
Modifier and Type | Method and Description |
---|---|
PhysicalOperator |
POUserFunc.getReferencedOperator() |
Modifier and Type | Method and Description |
---|---|
void |
POUserFunc.setReferencedOperator(PhysicalOperator referencedOperator) |
Constructor and Description |
---|
POUserFunc(OperatorKey k,
int rp,
List<PhysicalOperator> inp) |
POUserFunc(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
FuncSpec funcSpec) |
POUserFunc(OperatorKey k,
int rp,
List<PhysicalOperator> inp,
FuncSpec funcSpec,
EvalFunc func) |
Modifier and Type | Class and Description |
---|---|
class |
XMLPhysicalPlanPrinter<P extends OperatorPlan<PhysicalOperator>> |
Modifier and Type | Method and Description |
---|---|
void |
PhysicalPlan.connect(PhysicalOperator from,
PhysicalOperator to) |
protected String[] |
DotPOPrinter.getAttributes(PhysicalOperator op) |
protected Collection<PhysicalPlan> |
DotPOPrinter.getMultiOutputNestedPlans(PhysicalOperator op) |
protected String |
DotPOPrinter.getName(PhysicalOperator op) |
protected Collection<PhysicalPlan> |
DotPOPrinter.getNestedPlans(PhysicalOperator op) |
void |
PhysicalPlan.remove(PhysicalOperator op) |
void |
PhysicalPlan.replace(PhysicalOperator oldNode,
PhysicalOperator newNode) |
void |
PhyPlanVisitor.visit(PhysicalOperator op) |
Modifier and Type | Method and Description |
---|---|
void |
PhysicalPlan.setOpMap(MultiMap<PhysicalOperator,PhysicalOperator> opmap) |
void |
PhysicalPlan.setOpMap(MultiMap<PhysicalOperator,PhysicalOperator> opmap) |
Constructor and Description |
---|
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Class and Description |
---|---|
class |
POBroadcastSpark |
class |
POCollectedGroup
The collected group operator is a special operator used when users give
the hint 'using "collected"' in a group by clause.
|
class |
POCounter
This operator is part of the RANK operator implementation.
|
class |
POCross
Recover this class for nested cross operation.
|
class |
PODemux
The MapReduce Demultiplexer operator.
|
class |
PODistinct
Find the distinct set of tuples in a bag.
|
class |
POFilter
This is an implementation of the Filter operator.
|
class |
POForEach |
class |
POFRJoin
The operator models the join keys using the Local Rearrange operators which
are configured with the plan specified by the user.
|
class |
POFRJoinSpark |
class |
POGlobalRearrange
Dummy operator to test MRCompiler.
|
class |
POLimit |
class |
POLoad
The load operator which is used in two ways:
1) As a local operator it can be used to load files
2) In the Map Reduce setting, it is used to create jobs
from MapReduce operators which keep the loads and
stores in the Map and Reduce Plans till the job is created
|
class |
POLocalRearrange
The local rearrange operator is a part of the co-group
implementation.
|
class |
POMergeCogroup |
class |
POMergeJoin
This operator implements merge join algorithm to do map side joins.
|
class |
PONative |
class |
POOptimizedForEach
A specialized version of POForeach with the difference
that in getNext(), it knows that "input" has already been
attached by its input operator which SHOULD be POJoinPackage
|
class |
POPackage
The package operator that packages
the globally rearranged tuples into
output format as required by co-group.
|
class |
POPartialAgg
Do partial aggregation in map plan.
|
class |
POPartitionRearrange
The partition rearrange operator is a part of the skewed join
implementation.
|
class |
POPoissonSample |
class |
POPreCombinerLocalRearrange
A specialized local rearrange operator which behaves
like the regular local rearrange in the getNext()
as far as getting its input and constructing the
"key" out of the input.
|
class |
PORank
This operator is part of the RANK operator implementation.
|
class |
POReservoirSample |
class |
POSkewedJoin
The PhysicalOperator that represents a skewed join.
|
class |
POSort
This implementation is applicable for both the physical plan and for the
local backend, as the conversion of physical to mapreduce would see the SORT
operator and take necessary steps to convert it to a quantile and a sort job.
|
class |
POSortedDistinct
This operator is a variation of PODistinct, the input to this operator
must be sorted already.
|
class |
POSplit
The MapReduce Split operator.
|
class |
POStore
The store operator which is used in two ways:
1) As a local operator it can be used to store files
2) In the Map Reduce setting, it is used to create jobs
from MapReduce operators which keep the loads and
stores in the Map and Reduce Plans till the job is created
|
class |
POStream |
class |
POUnion
The union operator that combines the two inputs into a single
stream.
|
Modifier and Type | Field and Description |
---|---|
protected PhysicalOperator[] |
POForEach.planLeafOps |
Modifier and Type | Field and Description |
---|---|
protected List<PhysicalOperator> |
POForEach.opsToBeReset |
Modifier and Type | Method and Description |
---|---|
PhysicalOperator |
POStream.clone() |
PhysicalOperator |
POPartialAgg.clone() |
PhysicalOperator |
POFilter.clone() |
PhysicalOperator[] |
POForEach.getPlanLeafOps() |
Modifier and Type | Method and Description |
---|---|
MultiMap<PhysicalOperator,PhysicalPlan> |
POSkewedJoin.getJoinPlans() |
List<PhysicalOperator> |
POForEach.getOpsToBeReset() |
Modifier and Type | Method and Description |
---|---|
void |
POUnion.setInputs(List<PhysicalOperator> inputs) |
void |
POSkewedJoin.setJoinPlans(MultiMap<PhysicalOperator,PhysicalPlan> joinPlans) |
void |
POForEach.setOpsToBeReset(List<PhysicalOperator> opsToBeReset) |
Modifier and Type | Method and Description |
---|---|
static <C extends PhysicalOperator> |
PlanHelper.containsPhysicalOperator(PhysicalPlan plan,
Class<C> opClass) |
Modifier and Type | Method and Description |
---|---|
static void |
SparkUtil.assertPredecessorSize(List<<any>> predecessors,
PhysicalOperator physicalOperator,
int size) |
static void |
SparkUtil.assertPredecessorSizeGreaterThan(List<<any>> predecessors,
PhysicalOperator physicalOperator,
int size) |
static int |
SparkPigContext.getParallelism(List<<any>> predecessors,
PhysicalOperator physicalOperator) |
Constructor and Description |
---|
JobGraphBuilder(SparkOperPlan plan,
Map<Class<? extends PhysicalOperator>,RDDConverter> convertMap,
SparkPigStats sparkStats,
JavaSparkContext sparkContext,
JobMetricsListener jobMetricsListener,
String jobGroupID,
org.apache.hadoop.mapred.JobConf jobConf,
PigContext pc) |
Modifier and Type | Interface and Description |
---|---|
interface |
RDDConverter<IN,OUT,T extends PhysicalOperator>
Given an RDD and a PhysicalOperater, and implementation of this class can
convert the RDD to another RDD.
|
Modifier and Type | Method and Description |
---|---|
<any> |
LocalRearrangeConverter.convert(List<<any>> predecessors,
PhysicalOperator physicalOperator) |
Modifier and Type | Class and Description |
---|---|
class |
POGlobalRearrangeSpark
POGlobalRearrange for spark mode
|
class |
POJoinGroupSpark
Collapse POLocalRearrange,POGlobalRearrange and POPackage to POJoinGroupSpark to reduce unnecessary map operations in the join/group
|
class |
POPoissonSampleSpark |
class |
POReduceBySpark
ReduceBy operator that maps to Sparks ReduceBy.
|
class |
POSampleSortSpark |
Modifier and Type | Method and Description |
---|---|
List<PhysicalOperator> |
POJoinGroupSpark.getPredecessors() |
Modifier and Type | Method and Description |
---|---|
void |
POJoinGroupSpark.setPredecessors(List<PhysicalOperator> predecessors) |
Modifier and Type | Method and Description |
---|---|
protected PhysicalOperator |
SparkSecondaryKeyOptimizerUtil.getCurrentNode(PhysicalOperator root,
PhysicalPlan reducePlan) |
Modifier and Type | Method and Description |
---|---|
protected PhysicalOperator |
SparkSecondaryKeyOptimizerUtil.getCurrentNode(PhysicalOperator root,
PhysicalPlan reducePlan) |
Modifier and Type | Field and Description |
---|---|
Set<PhysicalOperator> |
SparkOperator.scalars |
Modifier and Type | Method and Description |
---|---|
void |
SparkOperator.add(PhysicalOperator physicalOper) |
Modifier and Type | Field and Description |
---|---|
Set<PhysicalOperator> |
TezOperator.scalars |
Modifier and Type | Class and Description |
---|---|
class |
POBloomFilterRearrangeTez |
class |
POBuildBloomRearrangeTez
This operator writes out the key value for the hash join reduce operation similar to POLocalRearrangeTez.
|
class |
POCounterStatsTez
POCounterStatsTez is used to group counters from previous vertex POCounterTez tasks
|
class |
POCounterTez |
class |
POFRJoinTez
POFRJoinTez is used on the backend to load replicated table from Tez
ShuffleUnorderedKVInput and load fragmented table from data pipeline.
|
class |
POIdentityInOutTez
POIdentityInOutTez is used to pass through tuples as is to next vertex from
previous vertex's POLocalRearrangeTez.
|
class |
POLocalRearrangeTez
POLocalRearrangeTez is used to write to a Tez OrderedPartitionedKVOutput
(shuffle) or UnorderedKVOutput (broadcast)
|
class |
POPartitionRearrangeTez
The partition rearrange operator is a part of the skewed join implementation.
|
class |
PORankTez |
class |
POShuffledValueInputTez
POShuffledValueInputTez is used read tuples from a Tez Intermediate output from a shuffle edge
To be used with POValueOutputTez.
|
class |
POShuffleTezLoad |
class |
POSimpleTezLoad
POSimpleTezLoad is used on the backend to read tuples from a Tez MRInput
|
class |
POStoreTez
POStoreTez is used to write to a Tez MROutput
|
class |
POValueInputTez
POValueInputTez is used read tuples from a Tez Intermediate output from a 1-1
edge
|
class |
POValueOutputTez |
Modifier and Type | Method and Description |
---|---|
protected void |
PigProcessor.runPipeline(PhysicalOperator leaf) |
Modifier and Type | Method and Description |
---|---|
protected PhysicalOperator |
SecondaryKeyOptimizerUtil.getCurrentNode(PhysicalOperator root,
PhysicalPlan reducePlan) |
Modifier and Type | Method and Description |
---|---|
static List<Pair<PhysicalOperator,PhysicalPlan>> |
CombinerOptimizerUtil.findAlgebraicOps(List<PhysicalPlan> feInners)
find algebraic operators and also check if the foreach statement is
suitable for combiner use
|
Modifier and Type | Method and Description |
---|---|
static PhysicalPlan |
CombinerOptimizerUtil.createPlanWithPredecessors(PhysicalOperator algeOp,
PhysicalPlan pplan)
Create new plan and add to it the clones of operator algeOp and its
predecessors from the physical plan pplan .
|
protected PhysicalOperator |
SecondaryKeyOptimizerUtil.getCurrentNode(PhysicalOperator root,
PhysicalPlan reducePlan) |
Modifier and Type | Method and Description |
---|---|
static void |
AccumulatorOptimizerUtil.addAccumulator(PhysicalPlan plan,
List<PhysicalOperator> pos) |
static void |
CombinerOptimizerUtil.addAlgebraicFuncToCombineFE(POForEach cfe,
Map<PhysicalOperator,Integer> op2newpos)
add algebraic functions with appropriate projection to new foreach in combiner
|
Constructor and Description |
---|
DistinctPatcher(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
fixMapProjects(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Field and Description |
---|---|
protected Map<Operator,PhysicalOperator> |
ExpToPhyTranslationVisitor.logToPhyMap |
Constructor and Description |
---|
ExpToPhyTranslationVisitor(OperatorPlan plan,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
ExpToPhyTranslationVisitor(OperatorPlan plan,
PlanWalker walker,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
Modifier and Type | Field and Description |
---|---|
protected Map<Operator,PhysicalOperator> |
LogToPhyTranslationVisitor.logToPhyMap |
Modifier and Type | Method and Description |
---|---|
Map<Operator,PhysicalOperator> |
LogToPhyTranslationVisitor.getLogToPhyMap() |
Modifier and Type | Method and Description |
---|---|
Map<PhysicalOperator,DataBag> |
IllustratorAttacher.getDataMap() |
Map<Operator,PhysicalOperator> |
ExampleGenerator.getLogToPhyMap() |
Map<PhysicalOperator,PhysicalOperator> |
LocalMapReduceSimulator.getPhyToMRMap() |
Map<PhysicalOperator,PhysicalOperator> |
LocalMapReduceSimulator.getPhyToMRMap() |
Modifier and Type | Method and Description |
---|---|
void |
Illustrator.setEquivalenceClasses(LinkedList<IdentityHashSet<Tuple>> eqClasses,
PhysicalOperator po) |
Constructor and Description |
---|
AugmentBaseDataVisitor(OperatorPlan plan,
Map<Operator,PhysicalOperator> logToPhysMap,
Map<LOLoad,DataBag> baseData,
Map<Operator,DataBag> derivedData) |
LineageTrimmingVisitor(LogicalPlan plan,
Map<LOLoad,DataBag> baseData,
ExampleGenerator eg,
Map<Operator,PhysicalOperator> LogToPhyMap,
PhysicalPlan physPlan,
PigContext pc) |
Copyright © 2007-2017 The Apache Software Foundation