org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Class POLocalRearrange

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.lang.Comparable<Operator>, Illustrable

Direct Known Subclasses:

POLocalRearrangeTez, POPartitionRearrange

public class POLocalRearrange
extends PhysicalOperator

The local rearrange operator is a part of the co-group implementation. It has an embedded physical plan that generates tuples of the form (grpKey,(indxed inp Tuple)).

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields

Modifier and Type	Field and Description
protected byte	index
protected Result	inp
protected boolean	isCross
protected byte	keyType
protected java.util.List<ExpressionOperator>	leafOps
protected byte	mainKeyType
protected Tuple	mFakeTuple
protected boolean	mIsDistinct
protected static TupleFactory	mTupleFactory
protected java.util.List<PhysicalPlan>	plans
protected byte	secondaryKeyType
protected java.util.List<ExpressionOperator>	secondaryLeafOps
protected java.util.List<PhysicalPlan>	secondaryPlans
protected static long	serialVersionUID

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, outputs, parentPlan, pigLogger, reporter, requestedParallelism, res, RESULT_EMPTY, RESULT_EOP, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POLocalRearrange(OperatorKey k)
POLocalRearrange(OperatorKey k, int rp)
POLocalRearrange(OperatorKey k, int rp, java.util.List<PhysicalOperator> inp)
POLocalRearrange(OperatorKey k, java.util.List<PhysicalOperator> inp)
POLocalRearrange(POLocalRearrange copy)

Method Summary

Methods

Modifier and Type	Method and Description
void	attachInput(Tuple t) Overridden since the attachment of the new input should cause the old processing to end.
POLocalRearrange	clone() Make a deep copy of this operator.
protected Tuple	constructLROutput(java.util.List<Result> resLst, java.util.List<Result> secondaryResLst, Tuple value)
protected void	deepCopyTo(POLocalRearrange clone)
byte	getIndex()
protected java.lang.Object	getKeyFromResult(java.util.List<Result> resLst, byte type)
byte	getKeyType()
byte	getMainKeyType()
Result	getNextTuple() Calls getNext on the generate operator inside the nested physical plan.
java.util.List<PhysicalPlan>	getPlans()
java.util.Map<java.lang.Integer,java.lang.Integer>	getProjectedColsMap()
java.util.Map<java.lang.Integer,java.lang.Integer>	getSecondaryProjectedColsMap()
Tuple	illustratorMarkup(java.lang.Object in, java.lang.Object out, int eqClassIndex) input tuple mark up to be illustrate-able
boolean	isCross()
boolean	isDistinct()
boolean	isKeyCompound()
boolean	isKeyTuple()
boolean	isProjectStar()
boolean	isSecondaryKeyTuple()
boolean	isSecondaryProjectStar()
java.lang.String	name()
void	setCross(boolean isCross)
void	setDistinct(boolean isDistinct)
void	setIndex(int index) Sets the co-group index of this operator
void	setKeyType(byte keyType)
void	setMultiQueryIndex(int index) Sets the multi-query index of this operator
void	setPlans(java.util.List<PhysicalPlan> plans)
void	setPlansFromCombiner(java.util.List<PhysicalPlan> plans)
void	setSecondaryPlans(java.util.List<PhysicalPlan> plans)
protected void	setStripKeyFromValue(boolean stripKeyFromValue)
void	setUseSecondaryKey(boolean useSecondaryKey)
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
boolean	supportsMultipleOutputs() Indicates whether this operator supports multiple outputs.
void	visit(PhyPlanVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, cloneHelper, copyAliasFrom, detachInput, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setIllustrator, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType, staticDataCleanup

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

serialVersionUID

protected static final long serialVersionUID

See Also:

Constant Field Values

mTupleFactory

protected static final TupleFactory mTupleFactory

plans

protected java.util.List<PhysicalPlan> plans

secondaryPlans

protected java.util.List<PhysicalPlan> secondaryPlans

leafOps

protected java.util.List<ExpressionOperator> leafOps

secondaryLeafOps

protected java.util.List<ExpressionOperator> secondaryLeafOps

index

protected byte index

keyType

protected byte keyType

mainKeyType

protected byte mainKeyType

secondaryKeyType

protected byte secondaryKeyType

mIsDistinct

protected boolean mIsDistinct

isCross

protected boolean isCross

inp

protected Result inp

mFakeTuple

protected Tuple mFakeTuple

Constructor Detail

POLocalRearrange

public POLocalRearrange(OperatorKey k)

POLocalRearrange

public POLocalRearrange(OperatorKey k,
                int rp)

POLocalRearrange

public POLocalRearrange(OperatorKey k,
                java.util.List<PhysicalOperator> inp)

POLocalRearrange

public POLocalRearrange(OperatorKey k,
                int rp,
                java.util.List<PhysicalOperator> inp)

POLocalRearrange

public POLocalRearrange(POLocalRearrange copy)

Method Detail

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException

Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class PhysicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

name

public java.lang.String name()

Specified by:

name in class Operator<PhyPlanVisitor>

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()

Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Specified by:

supportsMultipleOutputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

getIndex

public byte getIndex()

setIndex

public void setIndex(int index)
              throws ExecException

Sets the co-group index of this operator

Parameters:

index - the position of this operator in a co-group operation

Throws:

ExecException - if the index value is bigger then 0x7F

setMultiQueryIndex

public void setMultiQueryIndex(int index)
                        throws ExecException

Sets the multi-query index of this operator

Parameters:

index - the position of the parent plan of this operator in the enclosed split operator

Throws:

ExecException - if the index value is bigger then 0x7F

isDistinct

public boolean isDistinct()

setDistinct

public void setDistinct(boolean isDistinct)

attachInput

public void attachInput(Tuple t)

Overridden since the attachment of the new input should cause the old processing to end.

Overrides:

attachInput in class PhysicalOperator

Parameters:

t - - The tuple that should be used as input

getNextTuple

public Result getNextTuple()
                    throws ExecException

Calls getNext on the generate operator inside the nested physical plan. Converts the generated tuple into the proper format, i.e, (key,indexedTuple(value))

Overrides:

getNextTuple in class PhysicalOperator

Throws:

ExecException

getKeyFromResult

protected java.lang.Object getKeyFromResult(java.util.List<Result> resLst,
                                byte type)
                                     throws ExecException

Throws:

ExecException

constructLROutput

protected Tuple constructLROutput(java.util.List<Result> resLst,
                      java.util.List<Result> secondaryResLst,
                      Tuple value)
                           throws ExecException

Throws:

ExecException

getKeyType

public byte getKeyType()

getMainKeyType

public byte getMainKeyType()

setKeyType

public void setKeyType(byte keyType)

getPlans

public java.util.List<PhysicalPlan> getPlans()

setUseSecondaryKey

public void setUseSecondaryKey(boolean useSecondaryKey)

setPlans

public void setPlans(java.util.List<PhysicalPlan> plans)
              throws PlanException

Throws:

PlanException

setSecondaryPlans

public void setSecondaryPlans(java.util.List<PhysicalPlan> plans)
                       throws PlanException

Throws:

PlanException

clone

public POLocalRearrange clone()
                       throws java.lang.CloneNotSupportedException

Make a deep copy of this operator.

Overrides:

clone in class PhysicalOperator

Throws:

java.lang.CloneNotSupportedException

See Also:

Do not use the clone method directly. Operators are cloned when logical plans are cloned using {@link LogicalPlanCloner}

deepCopyTo

protected void deepCopyTo(POLocalRearrange clone)
                   throws java.lang.CloneNotSupportedException

Throws:

java.lang.CloneNotSupportedException

isCross

public boolean isCross()

setCross

public void setCross(boolean isCross)

getProjectedColsMap

public java.util.Map<java.lang.Integer,java.lang.Integer> getProjectedColsMap()

Returns:

the mProjectedColsMap

getSecondaryProjectedColsMap

public java.util.Map<java.lang.Integer,java.lang.Integer> getSecondaryProjectedColsMap()

Returns:

the mProjectedColsMap

isProjectStar

public boolean isProjectStar()

Returns:

the mProjectStar

isSecondaryProjectStar

public boolean isSecondaryProjectStar()

Returns:

the mProjectStar

isKeyTuple

public boolean isKeyTuple()

Returns:

the keyTuple

isKeyCompound

public boolean isKeyCompound()

Returns:

the isKeyCompound

isSecondaryKeyTuple

public boolean isSecondaryKeyTuple()

Returns:

the keyTuple

setPlansFromCombiner

public void setPlansFromCombiner(java.util.List<PhysicalPlan> plans)
                          throws PlanException

Parameters:

plans -

Throws:

ExecException

PlanException

setStripKeyFromValue

protected void setStripKeyFromValue(boolean stripKeyFromValue)

illustratorMarkup

public Tuple illustratorMarkup(java.lang.Object in,
                      java.lang.Object out,
                      int eqClassIndex)

Description copied from interface: Illustrable

input tuple mark up to be illustrate-able

Parameters:

in - input tuple

out - output tuple before wrapped in ExampleTuple

eqClassIndex - index into equivalence classes in illustrator

Returns:

tuple