org.apache.pig.backend.hadoop.executionengine.spark.operator

Class POReduceBySpark

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach

org.apache.pig.backend.hadoop.executionengine.spark.operator.POReduceBySpark

All Implemented Interfaces:

Serializable, Cloneable, Comparable<Operator>, Illustrable

public class POReduceBySpark
extends POForEach

ReduceBy operator that maps to Sparks ReduceBy. Extends ForEach and adds packager, secondary sort and partitioner support.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields

Modifier and Type	Field and Description
protected POLocalRearrange	lr
protected POPackage	pkg

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach

bags, buffer, data, earlyTermination, endOfAllInputProcessed, endOfAllInputProcessing, inpTuple, inputPlans, isToBeFlattenedArray, its, mapSideOnly, noItems, opsToBeReset, planLeafOps, processingPlan, resultTypes, tIn

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, mBagFactory, mTupleFactory, outputs, parentPlan, pigLogger, reporter, requestedParallelism, res, RESULT_EMPTY, RESULT_EOP, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POReduceBySpark(OperatorKey k, int rp, List<PhysicalPlan> inp, List<Boolean> isToBeFlattened, POPackage pkg, POLocalRearrange lr)

Method Summary

Modifier and Type	Method and Description
String	getCustomPartitioner()
protected String	getFlatStr()
POLocalRearrange	getLROp()
POPackage	getPKGOp()
boolean[]	getSecondarySortOrder()
boolean	isUseSecondaryKey()
String	name()
void	setCustomPartitioner(String customPartitioner)
void	setSecondarySortOrder(boolean[] secondarySortOrder)
void	setUseSecondaryKey(boolean useSecondaryKey)

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach

addInputPlan, attachInputToPlans, clone, createTuple, getInputPlans, getLeaves, getNextTuple, getOpsToBeReset, getPlanLeafOps, getToBeFlattened, illustratorMarkup, inProcessing, isMapSideOnly, needEndOfAllInputProcessing, processPlan, setAccumEnd, setAccumStart, setAccumulative, setInputPlans, setMapSideOnly, setOpsToBeReset, setToBeFlattened, setUpFlattens, supportsMultipleInputs, supportsMultipleOutputs, visit

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, attachInput, cloneHelper, clonePlans, copyAliasFrom, detachInput, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getParentPlan, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isEndOfAllInput, isInputAttached, processInput, reset, setIllustrator, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType, staticDataCleanup

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

lr

protected POLocalRearrange lr

pkg

protected POPackage pkg

Constructor Detail

POReduceBySpark

public POReduceBySpark(OperatorKey k,
                       int rp,
                       List<PhysicalPlan> inp,
                       List<Boolean> isToBeFlattened,
                       POPackage pkg,
                       POLocalRearrange lr)

Method Detail

getPKGOp

public POPackage getPKGOp()

name

public String name()

Overrides:

name in class POForEach

getFlatStr

protected String getFlatStr()

isUseSecondaryKey

public boolean isUseSecondaryKey()

setUseSecondaryKey

public void setUseSecondaryKey(boolean useSecondaryKey)

getSecondarySortOrder

public boolean[] getSecondarySortOrder()

setSecondarySortOrder

public void setSecondarySortOrder(boolean[] secondarySortOrder)

getCustomPartitioner

public String getCustomPartitioner()

setCustomPartitioner

public void setCustomPartitioner(String customPartitioner)

getLROp

public POLocalRearrange getLROp()