org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators
Class POMultiQueryPackage

java.lang.Object
  extended by org.apache.pig.impl.plan.Operator<PhyPlanVisitor>
      extended by org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
          extended by org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage
              extended by org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMultiQueryPackage
All Implemented Interfaces:
Serializable, Cloneable, Comparable<Operator>

public class POMultiQueryPackage
extends POPackage

The package operator that packages the globally rearranged tuples into output format as required by multi-query de-multiplexer.

This operator is used when merging multiple Map-Reduce splittees into a Map-only splitter during multi-query optimization. The package operators of the reduce plans of the splittees form an indexed package list inside this operator. When this operator receives an input, it extracts the index from the key and calls the corresponding package to get the output data.

Due to the recursive nature of multi-query optimization, this operator may be contained in another multi-query packager.

The successor of this operator must be a PODemux operator which knows how to consume the output of this operator.

See Also:
Serialized Form

Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage
POPackage.PackageType
 
Field Summary
 
Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage
distinct, isKeyTuple, keyAsTuple, keyInfo, mBagFactory, mTupleFactory
 
Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
alias, dummyBag, dummyBool, dummyDBA, dummyDouble, dummyFloat, dummyInt, dummyLong, dummyMap, dummyString, dummyTuple, input, inputAttached, inputs, lineageTracer, outputs, parentPlan, pigLogger, reporter, requestedParallelism, res, resultType
 
Fields inherited from class org.apache.pig.impl.plan.Operator
mKey
 
Constructor Summary
POMultiQueryPackage(OperatorKey k)
          Constructs an operator with the specified key.
POMultiQueryPackage(OperatorKey k, int rp)
          Constructs an operator with the specified key and degree of parallelism.
POMultiQueryPackage(OperatorKey k, int rp, List<PhysicalOperator> inp)
          Constructs an operator with the specified key, degree of parallelism and inputs.
POMultiQueryPackage(OperatorKey k, List<PhysicalOperator> inp)
          Constructs an operator with the specified key and inputs.
 
Method Summary
 void addIsKeyWrappedList(List<Boolean> lst)
          Adds a list of IsKeyWrapped boolean values
 void addPackage(POPackage pack)
          Appends the specified package object to the end of the package list.
 void addPackage(POPackage pack, byte mapKeyType)
          Appends the specified package object to the end of the package list.
 void attachInput(PigNullableWritable k, Iterator<NullableTuple> inp)
          Attaches the required inputs
 void detachInput()
          attachInput's better half!
 List<Boolean> getIsKeyWrappedList()
          Returns the list of booleans that indicates if the key needs to unwrapped for the corresponding plan.
 Result getNext(Tuple t)
          Constructs the output tuple from the inputs.
 List<POPackage> getPackages()
          Returns the list of packages.
 boolean isInCombiner()
           
 boolean isSameMapKeyType()
           
 String name()
           
 void setInCombiner(boolean inCombiner)
           
 void setSameMapKeyType(boolean sameMapKeyType)
           
 boolean supportsMultipleInputs()
          Indicates whether this operator supports multiple inputs.
 boolean supportsMultipleOutputs()
          Indicates whether this operator supports multiple outputs.
 void visit(PhyPlanVisitor v)
          Visit this node with the provided visitor.
 
Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage
clone, getInner, getKeyInfo, getKeyPositionsInTuple, getKeyType, getNumInps, getPackageType, getValueTuple, isDistinct, setDistinct, setInner, setKeyInfo, setKeyTuple, setKeyType, setNumInps, setPackageType, setUseSecondaryKey
 
Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
attachInput, cloneHelper, getAlias, getAliasString, getInputs, getLogger, getNext, getNext, getNext, getNext, getNext, getNext, getNext, getNext, getNext, getPigLogger, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setAlias, setInputs, setLineageTracer, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType
 
Methods inherited from class org.apache.pig.impl.plan.Operator
compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap
 
Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k)
Constructs an operator with the specified key.

Parameters:
k - the operator key

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k,
                           int rp)
Constructs an operator with the specified key and degree of parallelism.

Parameters:
k - the operator key
rp - the degree of parallelism requested

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k,
                           List<PhysicalOperator> inp)
Constructs an operator with the specified key and inputs.

Parameters:
k - the operator key
inp - the inputs that this operator will read data from

POMultiQueryPackage

public POMultiQueryPackage(OperatorKey k,
                           int rp,
                           List<PhysicalOperator> inp)
Constructs an operator with the specified key, degree of parallelism and inputs.

Parameters:
k - the operator key
rp - the degree of parallelism requested
inp - the inputs that this operator will read data from
Method Detail

name

public String name()
Overrides:
name in class POPackage

supportsMultipleInputs

public boolean supportsMultipleInputs()
Description copied from class: Operator
Indicates whether this operator supports multiple inputs.

Overrides:
supportsMultipleInputs in class POPackage
Returns:
true if it does, otherwise false.

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException
Description copied from class: Operator
Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Overrides:
visit in class POPackage
Parameters:
v - Visitor to visit with.
Throws:
VisitorException - if the visitor has a problem.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()
Description copied from class: Operator
Indicates whether this operator supports multiple outputs.

Overrides:
supportsMultipleOutputs in class POPackage
Returns:
true if it does, otherwise false.

attachInput

public void attachInput(PigNullableWritable k,
                        Iterator<NullableTuple> inp)
Description copied from class: POPackage
Attaches the required inputs

Overrides:
attachInput in class POPackage
Parameters:
k - - the key being worked on
inp - - iterator of indexed tuples typically obtained from Hadoop

detachInput

public void detachInput()
Description copied from class: POPackage
attachInput's better half!

Overrides:
detachInput in class POPackage

addPackage

public void addPackage(POPackage pack)
Appends the specified package object to the end of the package list.

Parameters:
pack - package to be appended to the list

addPackage

public void addPackage(POPackage pack,
                       byte mapKeyType)
Appends the specified package object to the end of the package list.

Parameters:
pack - package to be appended to the list
mapKeyType - the map key type associated with the package

getPackages

public List<POPackage> getPackages()
Returns the list of packages.

Returns:
the list of the packages

getNext

public Result getNext(Tuple t)
               throws ExecException
Constructs the output tuple from the inputs.

The output is consumed by for the demultiplexer operator (PODemux) in the format (key, {bag of tuples}) where key is an indexed WritableComparable, not the wrapped value as a pig type.

Overrides:
getNext in class POPackage
Throws:
ExecException

getIsKeyWrappedList

public List<Boolean> getIsKeyWrappedList()
Returns the list of booleans that indicates if the key needs to unwrapped for the corresponding plan.

Returns:
the list of isKeyWrapped boolean values

addIsKeyWrappedList

public void addIsKeyWrappedList(List<Boolean> lst)
Adds a list of IsKeyWrapped boolean values

Parameters:
lst - the list of boolean values to add

setInCombiner

public void setInCombiner(boolean inCombiner)

isInCombiner

public boolean isInCombiner()

setSameMapKeyType

public void setSameMapKeyType(boolean sameMapKeyType)

isSameMapKeyType

public boolean isSameMapKeyType()


Copyright © ${year} The Apache Software Foundation