org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Class POPackage

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POPackage

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.lang.Comparable<Operator>, Illustrable

Direct Known Subclasses:

POShuffleTezLoad

public class POPackage
extends PhysicalOperator

The package operator that packages the globally rearranged tuples into output format as required by co-group. This is last stage of processing co-group. This operator has a slightly different format than other operators in that, it takes two things as input. The key being worked on and the iterator of bags that contain indexed tuples that just need to be packaged into their appropriate output bags based on the index.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	POPackage.POPackageTupleBuffer

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.Object	key
protected PigNullableWritable	keyWritable
protected int	numInputs
protected Packager	pkgr

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, mBagFactory, mTupleFactory, outputs, parentPlan, pigLogger, reporter, requestedParallelism, res, RESULT_EMPTY, RESULT_EOP, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POPackage(OperatorKey k)
POPackage(OperatorKey k, int rp)
POPackage(OperatorKey k, int rp, java.util.List<PhysicalOperator> inp)
POPackage(OperatorKey k, int rp, java.util.List<PhysicalOperator> inp, Packager pkgr)
POPackage(OperatorKey k, java.util.List<PhysicalOperator> inp)
POPackage(POPackage copy)

Method Summary

Modifier and Type	Method and Description
void	attachInput(PigNullableWritable k, java.util.Iterator<NullableTuple> inp) Attaches the required inputs
POPackage	clone() Make a deep copy of this operator.
void	detachInput() attachInput's better half!
Result	getNextTuple() From the inputs, constructs the output tuple for this co-group in the required format which is (key, {bag of tuples from input 1}, {bag of tuples from input 2}, ...)
int	getNumInps()
Packager	getPkgr()
Tuple	illustratorMarkup(java.lang.Object in, java.lang.Object out, int eqClassIndex) input tuple mark up to be illustrate-able
java.lang.String	name()
int	numberOfEquivalenceClasses()
void	setIllustrator(Illustrator illustrator)
void	setNumInps(int numInps)
void	setPkgr(Packager pkgr)
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
boolean	supportsMultipleOutputs() Indicates whether this operator supports multiple outputs.
void	visit(PhyPlanVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, attachInput, cloneHelper, clonePlans, copyAliasFrom, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getParentPlan, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isEndOfAllInput, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType, staticDataCleanup

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

key

protected java.lang.Object key

numInputs

protected int numInputs

pkgr

protected Packager pkgr

keyWritable

protected PigNullableWritable keyWritable

Constructor Detail

POPackage

public POPackage(OperatorKey k)

POPackage

public POPackage(OperatorKey k,
                 int rp)

POPackage

public POPackage(OperatorKey k,
                 java.util.List<PhysicalOperator> inp)

POPackage

public POPackage(OperatorKey k,
                 int rp,
                 java.util.List<PhysicalOperator> inp)

POPackage

public POPackage(OperatorKey k,
                 int rp,
                 java.util.List<PhysicalOperator> inp,
                 Packager pkgr)

POPackage

public POPackage(POPackage copy)

Method Detail

setIllustrator

public void setIllustrator(Illustrator illustrator)

Specified by:

setIllustrator in interface Illustrable

Overrides:

setIllustrator in class PhysicalOperator

name

public java.lang.String name()

Specified by:

name in class Operator<PhyPlanVisitor>

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException

Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class PhysicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()

Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Specified by:

supportsMultipleOutputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

attachInput

public void attachInput(PigNullableWritable k,
                        java.util.Iterator<NullableTuple> inp)

Attaches the required inputs

Parameters:

k - - the key being worked on

inp - - iterator of indexed tuples typically obtained from Hadoop

detachInput

public void detachInput()

attachInput's better half!

Overrides:

detachInput in class PhysicalOperator

getNumInps

public int getNumInps()

setNumInps

public void setNumInps(int numInps)

getNextTuple

public Result getNextTuple()
                    throws ExecException

From the inputs, constructs the output tuple for this co-group in the required format which is (key, {bag of tuples from input 1}, {bag of tuples from input 2}, ...)

Overrides:

getNextTuple in class PhysicalOperator

Throws:

ExecException

getPkgr

public Packager getPkgr()

setPkgr

public void setPkgr(Packager pkgr)

clone

public POPackage clone()
                throws java.lang.CloneNotSupportedException

Make a deep copy of this operator.

Overrides:

clone in class PhysicalOperator

Throws:

java.lang.CloneNotSupportedException

See Also:

Do not use the clone method directly. Operators are cloned when logical plans are cloned using {@link LogicalPlanCloner}

illustratorMarkup

public Tuple illustratorMarkup(java.lang.Object in,
                               java.lang.Object out,
                               int eqClassIndex)

Description copied from interface: Illustrable

input tuple mark up to be illustrate-able

Parameters:

in - input tuple

out - output tuple before wrapped in ExampleTuple

eqClassIndex - index into equivalence classes in illustrator

Returns:

tuple

numberOfEquivalenceClasses

public int numberOfEquivalenceClasses()