Package | Description |
---|---|
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer | |
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer |
Implementation of physical operators that use hadoop as the execution engine
and data storage.
|
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators | |
org.apache.pig.backend.hadoop.executionengine.tez.plan | |
org.apache.pig.backend.hadoop.executionengine.tez.plan.operator | |
org.apache.pig.impl.plan |
Modifier and Type | Class and Description |
---|---|
class |
MapReduceOper
An operator model for a Map Reduce job.
|
class |
NativeMapReduceOper |
Modifier and Type | Class and Description |
---|---|
static class |
DotMRPrinter.InnerOperator
Helper class to represent the relationship of map, reduce and
combine phases in an MR operator.
|
Modifier and Type | Class and Description |
---|---|
class |
PhysicalOperator
This is the base class for all operators.
|
Modifier and Type | Class and Description |
---|---|
class |
Add |
class |
BinaryComparisonOperator
This is a base class for all binary comparison operators.
|
class |
BinaryExpressionOperator
A base class for all Binary expression operators.
|
class |
ConstantExpression
This class implements a Constant of any type.
|
class |
Divide |
class |
EqualToExpr |
class |
ExpressionOperator
A base class for all types of expressions.
|
class |
GreaterThanExpr |
class |
GTOrEqualToExpr |
class |
LessThanExpr |
class |
LTOrEqualToExpr |
class |
Mod |
class |
Multiply |
class |
NotEqualToExpr |
class |
POAnd
Boolean and operator.
|
class |
POBinCond |
class |
POCast
This is just a cast that converts DataByteArray into either String or
Integer.
|
class |
POIsNull |
class |
POMapLookUp |
class |
PONegative |
class |
PONot
Boolean not operator.
|
class |
POOr
Boolean or operator.
|
class |
POProject
Implements the overloaded form of the project operator.
|
class |
PORegexp |
class |
PORelationToExprProject
Implements a specialized form of POProject which is
used *ONLY* in the following case:
This project is Project(*) introduced after a relational operator
to supply a bag as output (as an expression).
|
class |
POUserComparisonFunc |
class |
POUserFunc |
class |
Subtract |
class |
UnaryComparisonOperator
This is a base class for all unary comparison operators.
|
class |
UnaryExpressionOperator |
Modifier and Type | Class and Description |
---|---|
class |
PlanPrinter<O extends Operator,P extends OperatorPlan<O>> |
Modifier and Type | Method and Description |
---|---|
boolean |
PatternPlan.match(OperatorPlan<? extends Operator<?>> inpPlan)
Return true if the given plan has nodes that match the pattern
represented by this class
If a match is found, the PatterNodes in the plan will return non
null node for getMatch().
|
Constructor and Description |
---|
DotPOPrinter(PhysicalPlan plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> subgraphs,
Set<Operator> multiInputSubgraphs,
Set<Operator> multiOutputSubgraphs) |
DotPOPrinter(PhysicalPlan plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> subgraphs,
Set<Operator> multiInputSubgraphs,
Set<Operator> multiOutputSubgraphs) |
DotPOPrinter(PhysicalPlan plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> subgraphs,
Set<Operator> multiInputSubgraphs,
Set<Operator> multiOutputSubgraphs) |
Modifier and Type | Class and Description |
---|---|
class |
POCollectedGroup
The collected group operator is a special operator used when users give
the hint 'using "collected"' in a group by clause.
|
class |
POCounter
This operator is part of the RANK operator implementation.
|
class |
POCross
Recover this class for nested cross operation.
|
class |
PODemux
The MapReduce Demultiplexer operator.
|
class |
PODistinct
Find the distinct set of tuples in a bag.
|
class |
POFilter
This is an implementation of the Filter operator.
|
class |
POForEach |
class |
POFRJoin
The operator models the join keys using the Local Rearrange operators which
are configured with the plan specified by the user.
|
class |
POGlobalRearrange
Dummy operator to test MRCompiler.
|
class |
POLimit |
class |
POLoad
The load operator which is used in two ways:
1) As a local operator it can be used to load files
2) In the Map Reduce setting, it is used to create jobs
from MapReduce operators which keep the loads and
stores in the Map and Reduce Plans till the job is created
|
class |
POLocalRearrange
The local rearrange operator is a part of the co-group
implementation.
|
class |
POMergeCogroup |
class |
POMergeJoin
This operator implements merge join algorithm to do map side joins.
|
class |
PONative |
class |
POOptimizedForEach
A specialized version of POForeach with the difference
that in getNext(), it knows that "input" has already been
attached by its input operator which SHOULD be POJoinPackage
|
class |
POPackage
The package operator that packages
the globally rearranged tuples into
output format as required by co-group.
|
class |
POPartialAgg
Do partial aggregation in map plan.
|
class |
POPartitionRearrange
The partition rearrange operator is a part of the skewed join
implementation.
|
class |
POPoissonSample |
class |
POPreCombinerLocalRearrange
A specialized local rearrange operator which behaves
like the regular local rearrange in the getNext()
as far as getting its input and constructing the
"key" out of the input.
|
class |
PORank
This operator is part of the RANK operator implementation.
|
class |
POReservoirSample |
class |
POSkewedJoin
The PhysicalOperator that represents a skewed join.
|
class |
POSort
This implementation is applicable for both the physical plan and for the
local backend, as the conversion of physical to mapreduce would see the SORT
operator and take necessary steps to convert it to a quantile and a sort job.
|
class |
POSortedDistinct
This operator is a variation of PODistinct, the input to this operator
must be sorted already.
|
class |
POSplit
The MapReduce Split operator.
|
class |
POStore
The store operator which is used in two ways:
1) As a local operator it can be used to store files
2) In the Map Reduce setting, it is used to create jobs
from MapReduce operators which keep the loads and
stores in the Map and Reduce Plans till the job is created
|
class |
POStream |
class |
POUnion
The union operator that combines the two inputs into a single
stream.
|
Modifier and Type | Class and Description |
---|---|
class |
TezOperator
An operator model for a Tez job.
|
class |
TezPlanContainerNode |
Modifier and Type | Class and Description |
---|---|
class |
NativeTezOper |
class |
POCounterStatsTez
POCounterStatsTez is used to group counters from previous vertex POCounterTez tasks
|
class |
POCounterTez |
class |
POFRJoinTez
POFRJoinTez is used on the backend to load replicated table from Tez
ShuffleUnorderedKVInput and load fragmented table from data pipeline.
|
class |
POIdentityInOutTez
POIdentityInOutTez is used to pass through tuples as is to next vertex from
previous vertex's POLocalRearrangeTez.
|
class |
POLocalRearrangeTez
POLocalRearrangeTez is used to write to a Tez OrderedPartitionedKVOutput
(shuffle) or UnorderedKVOutput (broadcast)
|
class |
POPartitionRearrangeTez
The partition rearrange operator is a part of the skewed join implementation.
|
class |
PORankTez |
class |
POShuffledValueInputTez
POShuffledValueInputTez is used read tuples from a Tez Intermediate output from a shuffle edge
To be used with POValueOutputTez.
|
class |
POShuffleTezLoad |
class |
POSimpleTezLoad
POSimpleTezLoad is used on the backend to read tuples from a Tez MRInput
|
class |
POStoreTez
POStoreTez is used to write to a Tez MROutput
|
class |
POValueInputTez
POValueInputTez is used read tuples from a Tez Intermediate output from a 1-1
edge
|
class |
POValueOutputTez |
Modifier and Type | Class and Description |
---|---|
class |
DependencyOrderWalker<O extends Operator,P extends OperatorPlan<O>>
DependencyOrderWalker traverses the graph in such a way that no node is visited
before all the nodes it depends on have been visited.
|
class |
DepthFirstWalker<O extends Operator,P extends OperatorPlan<O>>
DepthFirstWalker traverses a plan in a depth first manner.
|
class |
DotPlanDumper<E extends Operator,P extends OperatorPlan<E>,N extends Operator,S extends OperatorPlan<N>>
This class puts everything that is needed to dump a plan in a
format readable by graphviz's dot algorithm.
|
class |
DotPlanDumper<E extends Operator,P extends OperatorPlan<E>,N extends Operator,S extends OperatorPlan<N>>
This class puts everything that is needed to dump a plan in a
format readable by graphviz's dot algorithm.
|
class |
OperatorPlan<E extends Operator>
A generic graphing class for use by LogicalPlan, PhysicalPlan, etc.
|
class |
PlanDumper<E extends Operator,P extends OperatorPlan<E>,S extends OperatorPlan<? extends Operator>>
This class dumps a nested plan to a print stream.
|
class |
PlanDumper<E extends Operator,P extends OperatorPlan<E>,S extends OperatorPlan<? extends Operator>>
This class dumps a nested plan to a print stream.
|
class |
PlanVisitor<O extends Operator,P extends OperatorPlan<O>>
A visitor mechanism for navigating and operating on a plan of
Operators.
|
class |
PlanWalker<O extends Operator,P extends OperatorPlan<O>>
PlanWalker encapsulates the logic to traverse a plan.
|
class |
ReverseDependencyOrderWalker<O extends Operator,P extends OperatorPlan<O>>
ReverseDependencyOrderWalker traverses the graph in such a way that no node is visited
before all the nodes that are its successors on have been visited.
|
Modifier and Type | Field and Description |
---|---|
protected Set<Operator> |
DotPlanDumper.mMultiInputSubgraphs |
protected Set<Operator> |
DotPlanDumper.mMultiOutputSubgraphs |
protected Set<Operator> |
DotPlanDumper.mSubgraphs |
Modifier and Type | Method and Description |
---|---|
int |
Operator.compareTo(Operator o) |
protected void |
PlanDumper.dumpEdge(Operator op,
Operator suc)
Will be called to dump the edges of the plan.
|
protected void |
PlanDumper.dumpEdge(Operator op,
Operator suc)
Will be called to dump the edges of the plan.
|
protected void |
DotPlanDumper.dumpEdge(Operator op,
Operator suc) |
protected void |
DotPlanDumper.dumpEdge(Operator op,
Operator suc) |
void |
Operator.rewire(Operator<V> oldPred,
int oldPredIndex,
Operator<V> newPred,
boolean useOldPred)
Make any necessary changes to a node based on a change of position in the
plan.
|
void |
Operator.rewire(Operator<V> oldPred,
int oldPredIndex,
Operator<V> newPred,
boolean useOldPred)
Make any necessary changes to a node based on a change of position in the
plan.
|
Constructor and Description |
---|
DotPlanDumper(P plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> mSubgraphs,
Set<Operator> mMultiInputSubgraphs,
Set<Operator> mMultiOutputSubgraphs) |
DotPlanDumper(P plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> mSubgraphs,
Set<Operator> mMultiInputSubgraphs,
Set<Operator> mMultiOutputSubgraphs) |
DotPlanDumper(P plan,
PrintStream ps,
boolean isSubGraph,
Set<Operator> mSubgraphs,
Set<Operator> mMultiInputSubgraphs,
Set<Operator> mMultiOutputSubgraphs) |
Copyright © 2007-2012 The Apache Software Foundation