org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Class POMergeJoin

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.lang.Comparable<Operator>, Illustrable

Direct Known Subclasses:

POMergeJoinTez

public class POMergeJoin
extends PhysicalOperator

This operator implements merge join algorithm to do map side joins. Currently, only two-way joins are supported. One input of join is identified as left and other is identified as right. Left input tuples are the input records in map. Right tuples are read from HDFS by opening right stream. This join doesn't support outer join. Data is assumed to be sorted in ascending order. It will fail if data is sorted in descending order.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	POMergeJoin.TuplesToSchemaTupleList This is a class that extends ArrayList, making it easy to provide on the fly conversion from Tuple to SchemaTuple.

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields

Modifier and Type	Field and Description
protected LoadFunc	rightLoader
protected FuncSpec	rightLoaderFuncSpec

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, mBagFactory, mTupleFactory, outputs, parentPlan, pigLogger, reporter, requestedParallelism, res, RESULT_EMPTY, RESULT_EOP, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POMergeJoin(OperatorKey k, int rp, java.util.List<PhysicalOperator> inp, MultiMap<PhysicalOperator,PhysicalPlan> inpPlans, java.util.List<java.util.List<java.lang.Byte>> keyTypes, LOJoin.JOINTYPE joinType, Schema leftInputSchema, Schema rightInputSchema, Schema mergedInputSchema)
POMergeJoin(POMergeJoin copy)

Method Summary

Modifier and Type	Method and Description
POMergeJoin	clone() Make a copy of this operator.
java.lang.String	getIndexFile()
java.util.List<PhysicalPlan>	getInnerPlansOf(int index)
LOJoin.JOINTYPE	getJoinType()
POLocalRearrange[]	getLRs()
Result	getNextTuple()
protected LoadFunc	getRightLoader() Instantiate right loader
java.lang.String	getSignature()
Tuple	illustratorMarkup(java.lang.Object in, java.lang.Object out, int eqClassIndex) input tuple mark up to be illustrate-able
boolean	isEndOfInput()
java.lang.String	name()
void	setEndOfInput(boolean isEndOfInput)
void	setIndexFile(java.lang.String indexFile)
void	setRightInputFileName(java.lang.String rightInputFileName)
void	setRightLoaderFuncSpec(FuncSpec rightLoaderFuncSpec)
void	setSignature(java.lang.String signature)
void	setupRightPipeline(PhysicalPlan rightPipeline)
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
boolean	supportsMultipleOutputs() Indicates whether this operator supports multiple outputs.
void	throwProcessingException(boolean withCauseException, java.lang.Exception e)
void	visit(PhyPlanVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, attachInput, cloneHelper, clonePlans, copyAliasFrom, detachInput, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getParentPlan, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isEndOfAllInput, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setIllustrator, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType, staticDataCleanup

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

rightLoader

protected transient LoadFunc rightLoader

rightLoaderFuncSpec

protected FuncSpec rightLoaderFuncSpec

Constructor Detail

POMergeJoin

public POMergeJoin(OperatorKey k,
                   int rp,
                   java.util.List<PhysicalOperator> inp,
                   MultiMap<PhysicalOperator,PhysicalPlan> inpPlans,
                   java.util.List<java.util.List<java.lang.Byte>> keyTypes,
                   LOJoin.JOINTYPE joinType,
                   Schema leftInputSchema,
                   Schema rightInputSchema,
                   Schema mergedInputSchema)
            throws PlanException

Parameters:

k -

rp -

inp -

inpPlans - there can only be 2 inputs each being a List Ex. join A by ($0,$1), B by ($1,$2);

Throws:

PlanException

POMergeJoin

public POMergeJoin(POMergeJoin copy)

Method Detail

isEndOfInput

public boolean isEndOfInput()

setEndOfInput

public void setEndOfInput(boolean isEndOfInput)

getNextTuple

public Result getNextTuple()
                    throws ExecException

Overrides:

getNextTuple in class PhysicalOperator

Throws:

ExecException

getRightLoader

protected LoadFunc getRightLoader()
                           throws ExecException,
                                  java.io.IOException

Instantiate right loader

Returns:

Throws:

java.io.IOException

ExecException

throwProcessingException

public void throwProcessingException(boolean withCauseException,
                                     java.lang.Exception e)
                              throws ExecException

Throws:

ExecException

setupRightPipeline

public void setupRightPipeline(PhysicalPlan rightPipeline)
                        throws FrontendException

Throws:

FrontendException

setRightLoaderFuncSpec

public void setRightLoaderFuncSpec(FuncSpec rightLoaderFuncSpec)

getInnerPlansOf

public java.util.List<PhysicalPlan> getInnerPlansOf(int index)

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException

Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class PhysicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

name

public java.lang.String name()

Specified by:

name in class Operator<PhyPlanVisitor>

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()

Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Specified by:

supportsMultipleOutputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

setRightInputFileName

public void setRightInputFileName(java.lang.String rightInputFileName)

Parameters:

rightInputFileName - the rightInputFileName to set

getSignature

public java.lang.String getSignature()

setSignature

public void setSignature(java.lang.String signature)

setIndexFile

public void setIndexFile(java.lang.String indexFile)

getIndexFile

public java.lang.String getIndexFile()

illustratorMarkup

public Tuple illustratorMarkup(java.lang.Object in,
                               java.lang.Object out,
                               int eqClassIndex)

Description copied from interface: Illustrable

input tuple mark up to be illustrate-able

Parameters:

in - input tuple

out - output tuple before wrapped in ExampleTuple

eqClassIndex - index into equivalence classes in illustrator

Returns:

tuple

getJoinType

public LOJoin.JOINTYPE getJoinType()

getLRs

public POLocalRearrange[] getLRs()

clone

public POMergeJoin clone()
                  throws java.lang.CloneNotSupportedException

Description copied from class: PhysicalOperator

Make a copy of this operator. This function is blank, however, we should leave a place holder so that the subclasses can clone to make deep copy as this one creates a shallow copy of non-primitive types (objects, arrays and lists)

Overrides:

clone in class PhysicalOperator

Throws:

java.lang.CloneNotSupportedException

See Also:

Do not use the clone method directly. Operators are cloned when logical plans are cloned using {@link LogicalPlanCloner}