org.apache.pig.newplan.logical.relational
Class LogToPhyTranslationVisitor

java.lang.Object
  extended by org.apache.pig.newplan.PlanVisitor
      extended by org.apache.pig.newplan.logical.relational.LogicalRelationalNodesVisitor
          extended by org.apache.pig.newplan.logical.relational.LogToPhyTranslationVisitor

public class LogToPhyTranslationVisitor
extends LogicalRelationalNodesVisitor


Field Summary
protected  PhysicalPlan currentPlan
           
protected  Deque<PhysicalPlan> currentPlans
           
protected  Map<Operator,PhysicalOperator> logToPhyMap
           
protected  NodeIdGenerator nodeGen
           
protected  PigContext pc
           
 
Fields inherited from class org.apache.pig.newplan.PlanVisitor
currentWalker, DEFAULT_SCOPE, plan
 
Constructor Summary
LogToPhyTranslationVisitor(OperatorPlan plan)
           
 
Method Summary
 Map<Operator,PhysicalOperator> getLogToPhyMap()
           
 PhysicalPlan getPhysicalPlan()
           
 void setPigContext(PigContext pc)
           
static void updateWithEmptyBagCheck(PhysicalPlan fePlan, Operator joinInput)
          updates plan with check for empty bag and if bag is empty to flatten a bag with as many null's as dictated by the schema
 void visit(LOCogroup cg)
           
 void visit(LOCross cross)
           
 void visit(LODistinct loDistinct)
           
 void visit(LOFilter filter)
           
 void visit(LOForEach foreach)
           
 void visit(LOInnerLoad load)
           
 void visit(LOJoin loj)
           
 void visit(LOLimit loLimit)
           
 void visit(LOLoad loLoad)
           
 void visit(LONative loNative)
           
 void visit(LORank loRank)
          Transformation from Logical to Physical Plan involves the following steps: First, it is generated a random number which will link a POCounter within a PORank.
 void visit(LOSort sort)
           
 void visit(LOSplit loSplit)
           
 void visit(LOSplitOutput loSplitOutput)
           
 void visit(LOStore loStore)
           
 void visit(LOStream stream)
           
 void visit(LOUnion loUnion)
           
 
Methods inherited from class org.apache.pig.newplan.logical.relational.LogicalRelationalNodesVisitor
visit, visit
 
Methods inherited from class org.apache.pig.newplan.PlanVisitor
getPlan, popWalker, pushWalker, visit
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

logToPhyMap

protected Map<Operator,PhysicalOperator> logToPhyMap

currentPlans

protected Deque<PhysicalPlan> currentPlans

currentPlan

protected PhysicalPlan currentPlan

nodeGen

protected NodeIdGenerator nodeGen

pc

protected PigContext pc
Constructor Detail

LogToPhyTranslationVisitor

public LogToPhyTranslationVisitor(OperatorPlan plan)
                           throws FrontendException
Throws:
FrontendException
Method Detail

setPigContext

public void setPigContext(PigContext pc)

getLogToPhyMap

public Map<Operator,PhysicalOperator> getLogToPhyMap()

getPhysicalPlan

public PhysicalPlan getPhysicalPlan()

visit

public void visit(LOLoad loLoad)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LONative loNative)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOFilter filter)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOSort sort)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LORank loRank)
           throws FrontendException
Transformation from Logical to Physical Plan involves the following steps: First, it is generated a random number which will link a POCounter within a PORank. On this way, avoiding possible collisions on parallel rank operations. Then, if it is row number mode:
 In case of a RANK operation (row number mode), are used two steps:
   1.- Each tuple is counted sequentially on each mapper, and are produced global counters
   2.- Global counters are gathered and summed, each tuple calls to the respective counter value
       in order to calculate the corresponding rank value.
 
or not:
 In case of a RANK BY operation, then are necessary five steps:
   1.- Group by the fields involved on the rank operation: POPackage
   2.- In case of multi-fields, the key (group field) is flatten: POForEach
   3.- Sort operation by the fields available after flattening: POSort
   4.- Each group is sequentially counted on each mapper through a global counter: POCounter
   5.- Global counters are summed and passed to the rank operation: PORank
 

Overrides:
visit in class LogicalRelationalNodesVisitor
Parameters:
loRank - describe if the rank operation is on a row number mode or is rank by (dense or not)
Throws:
FrontendException

visit

public void visit(LOCross cross)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOStream stream)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOInnerLoad load)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOForEach foreach)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOStore loStore)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOCogroup cg)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOJoin loj)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOUnion loUnion)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LODistinct loDistinct)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOLimit loLimit)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOSplit loSplit)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

visit

public void visit(LOSplitOutput loSplitOutput)
           throws FrontendException
Overrides:
visit in class LogicalRelationalNodesVisitor
Throws:
FrontendException

updateWithEmptyBagCheck

public static void updateWithEmptyBagCheck(PhysicalPlan fePlan,
                                           Operator joinInput)
                                    throws FrontendException
updates plan with check for empty bag and if bag is empty to flatten a bag with as many null's as dictated by the schema

Parameters:
fePlan - the plan to update
joinInput - the relation for which the corresponding bag is being checked
Throws:
FrontendException


Copyright © 2007-2012 The Apache Software Foundation