org.apache.pig.newplan.logical.relational

Class LogToPhyTranslationVisitor

java.lang.Object

org.apache.pig.newplan.PlanVisitor

org.apache.pig.newplan.logical.relational.LogicalRelationalNodesVisitor

org.apache.pig.newplan.logical.relational.LogToPhyTranslationVisitor

public class LogToPhyTranslationVisitor
extends LogicalRelationalNodesVisitor

Field Summary

Fields

Modifier and Type	Field and Description
protected PhysicalPlan	currentPlan
protected Deque<PhysicalPlan>	currentPlans
protected Map<Operator,PhysicalOperator>	logToPhyMap
protected NodeIdGenerator	nodeGen
protected PigContext	pc

Fields inherited from class org.apache.pig.newplan.PlanVisitor

currentWalker, DEFAULT_SCOPE, plan

Constructor Summary

Constructors

Constructor and Description
LogToPhyTranslationVisitor(OperatorPlan plan)

Method Summary

Modifier and Type	Method and Description
Map<Operator,PhysicalOperator>	getLogToPhyMap()
PhysicalPlan	getPhysicalPlan()
void	setPigContext(PigContext pc)
static void	updateWithEmptyBagCheck(PhysicalPlan fePlan, Operator joinInput) updates plan with check for empty bag and if bag is empty to flatten a bag with as many null's as dictated by the schema
void	visit(LOCogroup cg)
void	visit(LOCross cross)
void	visit(LODistinct loDistinct)
void	visit(LOFilter filter)
void	visit(LOForEach foreach)
void	visit(LOInnerLoad load)
void	visit(LOJoin loj)
void	visit(LOLimit loLimit)
void	visit(LOLoad loLoad)
void	visit(LONative loNative)
void	visit(LORank loRank) Transformation from Logical to Physical Plan involves the following steps: First, it is generated a random number which will link a POCounter within a PORank.
void	visit(LOSort sort)
void	visit(LOSplit loSplit)
void	visit(LOSplitOutput loSplitOutput)
void	visit(LOStore loStore)
void	visit(LOStream stream)
void	visit(LOUnion loUnion)

Methods inherited from class org.apache.pig.newplan.logical.relational.LogicalRelationalNodesVisitor

visit, visit

Methods inherited from class org.apache.pig.newplan.PlanVisitor

getPlan, popWalker, pushWalker, visit

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logToPhyMap

protected Map<Operator,PhysicalOperator> logToPhyMap

currentPlans

protected Deque<PhysicalPlan> currentPlans

currentPlan

protected PhysicalPlan currentPlan

nodeGen

protected NodeIdGenerator nodeGen

pc

protected PigContext pc

Constructor Detail

LogToPhyTranslationVisitor

public LogToPhyTranslationVisitor(OperatorPlan plan)
                           throws FrontendException

Throws:

FrontendException

Method Detail

setPigContext

public void setPigContext(PigContext pc)

getLogToPhyMap

public Map<Operator,PhysicalOperator> getLogToPhyMap()

getPhysicalPlan

public PhysicalPlan getPhysicalPlan()

visit

public void visit(LOLoad loLoad)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LONative loNative)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOFilter filter)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOSort sort)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LORank loRank)
           throws FrontendException

Transformation from Logical to Physical Plan involves the following steps: First, it is generated a random number which will link a POCounter within a PORank. On this way, avoiding possible collisions on parallel rank operations. Then, if it is row number mode:

 In case of a RANK operation (row number mode), are used two steps:
   1.- Each tuple is counted sequentially on each mapper, and are produced global counters
   2.- Global counters are gathered and summed, each tuple calls to the respective counter value
       in order to calculate the corresponding rank value.
 

or not:

 In case of a RANK BY operation, then are necessary five steps:
   1.- Group by the fields involved on the rank operation: POPackage
   2.- In case of multi-fields, the key (group field) is flatten: POForEach
   3.- Sort operation by the fields available after flattening: POSort
   4.- Each group is sequentially counted on each mapper through a global counter: POCounter
   5.- Global counters are summed and passed to the rank operation: PORank
 

Overrides:

visit in class LogicalRelationalNodesVisitor

Parameters:

loRank - describe if the rank operation is on a row number mode or is rank by (dense or not)

Throws:

FrontendException

visit

public void visit(LOCross cross)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOStream stream)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOInnerLoad load)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOForEach foreach)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOStore loStore)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOCogroup cg)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOJoin loj)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOUnion loUnion)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LODistinct loDistinct)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOLimit loLimit)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOSplit loSplit)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

visit

public void visit(LOSplitOutput loSplitOutput)
           throws FrontendException

Overrides:

visit in class LogicalRelationalNodesVisitor

Throws:

FrontendException

updateWithEmptyBagCheck

public static void updateWithEmptyBagCheck(PhysicalPlan fePlan,
                                           Operator joinInput)
                                    throws FrontendException

updates plan with check for empty bag and if bag is empty to flatten a bag with as many null's as dictated by the schema

Parameters:

fePlan - the plan to update

joinInput - the relation for which the corresponding bag is being checked

Throws:

FrontendException