TezOperator (Pig 0.17.0 API)

java.lang.Object
- org.apache.pig.impl.plan.Operator<TezOpPlanVisitor>
- - org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperator

All Implemented Interfaces:

Serializable, Cloneable, Comparable<Operator>

Direct Known Subclasses:

NativeTezOper
```
public class TezOperator
extends Operator<TezOpPlanVisitor>
```
An operator model for a Tez job. Acts as a host to the plans that will execute in Tez vertices.

See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class TezOperator.LoaderInfo

static class TezOperator.OPER_FEATURE

static class TezOperator.VertexGroupInfo

Nested Classes
Modifier and Type	Class and Description
`static class`	`TezOperator.LoaderInfo`
`static class`	`TezOperator.OPER_FEATURE`
`static class`	`TezOperator.VertexGroupInfo`

Field Summary

Fields
Modifier and Type	Field and Description
`Map<OperatorKey,TezEdgeDescriptor>`	`inEdges`
`Map<OperatorKey,TezEdgeDescriptor>`	`outEdges`
`PhysicalPlan`	`plan`
`Set<PhysicalOperator>`	`scalars`
`Set<String>`	`UDFs`

Fields inherited from class org.apache.pig.impl.plan.Operator
mKey

Constructor Summary

Constructors
Constructor and Description

TezOperator(OperatorKey k)

Constructors
Constructor and Description
`TezOperator(OperatorKey k)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addCrossKey(String key)`
`void`	`addUnionPredecessor(OperatorKey unionPredecessor)`
`void`	`addVertexGroupStore(OperatorKey storeKey, OperatorKey vertexGroupKey)`
`boolean`	`combineSmallSplits()`
`void`	`copyFeatures(TezOperator copyFrom, List<TezOperator.OPER_FEATURE> excludeFeatures)`
`List<String>`	`getCrossKeys()`
`int`	`getEffectiveParallelism(int defaultParallelism)`
`int`	`getEstimatedParallelism()`
`TezOperator.LoaderInfo`	`getLoaderInfo()`
`double`	`getParallelismFactor(TezOperator successor)`
`String`	`getProcessorName()`
`int`	`getRequestedParallelism()`
`TezOperator`	`getSampleOperator()`
`TezOperator`	`getSortOperator()`
`boolean[]`	`getSortOrder()`
`OperatorKey`	`getSplitParent()`
`long`	`getTotalInputFilesSize()`
`List<OperatorKey>`	`getUnionMembers()`
`TezOperator.VertexGroupInfo`	`getVertexGroupInfo()`
`List<OperatorKey>`	`getVertexGroupMembers()`
`Map<OperatorKey,OperatorKey>`	`getVertexGroupStores()`
`int`	`getVertexParallelism()`
`boolean`	`isBuildBloom()`
`boolean`	`isClosed()`
`boolean`	`isCogroup()`
`boolean`	`isDistinct()`
`boolean`	`isDontEstimateParallelism()`
`boolean`	`isFilterBloom()`
`boolean`	`isGlobalSort()`
`boolean`	`isGroupBy()`
`boolean`	`isIndexer()`
`Boolean`	`isIntermediateReducer()`
`boolean`	`isLimit()`
`boolean`	`isLimitAfterSort()`
`boolean`	`isNative()`
`boolean`	`isNeedEstimateParallelism()`
`boolean`	`isOverrideIntermediateParallelism()`
`boolean`	`isRankCounter()`
`boolean`	`isRegularJoin()`
`boolean`	`isSampleAggregation()`
`boolean`	`isSampleBasedPartitioner()`
`boolean`	`isSampler()`
`boolean`	`isSkewedJoin()`
`boolean`	`isSplitter()`
`boolean`	`isUnion()`
`boolean`	`isUseGraceParallelism()`
`boolean`	`isUseMRMapSettings()`
`boolean`	`isUseSecondaryKey()`
`boolean`	`isVertexGroup()`
`void`	`markBuildBloom()`
`void`	`markCogroup()`
`void`	`markDistinct()`
`void`	`markFilterBloom()`
`void`	`markGlobalSort()`
`void`	`markGroupBy()`
`void`	`markIndexer()`
`void`	`markLimit()`
`void`	`markLimitAfterSort()`
`void`	`markNative()`
`void`	`markRankCounter()`
`void`	`markRegularJoin()`
`void`	`markSampleAggregation()`
`void`	`markSampleBasedPartitioner()`
`void`	`markSampler()`
`void`	`markSkewedJoin()`
`void`	`markUnion()`
`String`	`name()`
`boolean`	`needSegmentBelow()`
`protected void`	`noCombineSmallSplits()`
`void`	`removeVertexGroupStore(OperatorKey vertexGroupKey)`
`void`	`setClosed(boolean closed)`
`void`	`setDontEstimateParallelism(boolean dontEstimateParallelism)`
`void`	`setEstimatedParallelism(int estimatedParallelism)`
`void`	`setIntermediateReducer(Boolean intermediateReducer)`
`void`	`setNeedEstimatedQuantile(boolean needEstimateParallelism)`
`void`	`setOverrideIntermediateParallelism(boolean overrideIntermediateParallelism)`
`void`	`setRequestedParallelism(int requestedParallelism)`
`void`	`setRequestedParallelismByReference(TezOperator oper)`
`void`	`setSampleOperator(TezOperator sampleOperator)`
`void`	`setSortOperator(TezOperator sortOperator)`
`void`	`setSortOrder(boolean[] sortOrder)`
`void`	`setSplitParent(OperatorKey splitParent)`
`void`	`setSplitter(boolean spl)`
`void`	`setTotalInputFilesSize(long totalInputFilesSize)`
`void`	`setUseGraceParallelism(boolean useGraceParallelism)`
`void`	`setUseMRMapSettings(boolean useMRMapSettings)`
`void`	`setUseSecondaryKey(boolean useSecondaryKey)`
`void`	`setVertexGroupInfo(TezOperator.VertexGroupInfo vertexGroup)`
`void`	`setVertexGroupMembers(List<OperatorKey> vertexGroupMembers)`
`void`	`setVertexParallelism(int vertexParallelism)`
`boolean`	`supportsMultipleInputs()` Indicates whether this operator supports multiple inputs.
`boolean`	`supportsMultipleOutputs()` Indicates whether this operator supports multiple outputs.
`String`	`toString()`
`void`	`visit(TezOpPlanVisitor v)` Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.impl.plan.Operator
clone, compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, unsetProjectionMap

Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

plan
```
public transient PhysicalPlan plan
```

outEdges

public Map<OperatorKey,TezEdgeDescriptor> outEdges

inEdges

public Map<OperatorKey,TezEdgeDescriptor> inEdges

UDFs
```
public Set<String> UDFs
```

scalars
```
public Set<PhysicalOperator> scalars
```

Constructor Detail
- TezOperator
```
public TezOperator(OperatorKey k)
```

Method Detail

getProcessorName
```
public String getProcessorName()
```

visit
```
public void visit(TezOpPlanVisitor v)
           throws VisitorException
```
Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class Operator<TezOpPlanVisitor>

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

supportsMultipleInputs
```
public boolean supportsMultipleInputs()
```
Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<TezOpPlanVisitor>

Returns:

true if it does, otherwise false.

supportsMultipleOutputs
```
public boolean supportsMultipleOutputs()
```
Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Specified by:

supportsMultipleOutputs in class Operator<TezOpPlanVisitor>

Returns:

true if it does, otherwise false.

getRequestedParallelism
```
public int getRequestedParallelism()
```

setRequestedParallelism

public void setRequestedParallelism(int requestedParallelism)

setRequestedParallelismByReference

public void setRequestedParallelismByReference(TezOperator oper)

getEstimatedParallelism
```
public int getEstimatedParallelism()
```

setEstimatedParallelism

public void setEstimatedParallelism(int estimatedParallelism)

getEffectiveParallelism

public int getEffectiveParallelism(int defaultParallelism)

isDontEstimateParallelism

public boolean isDontEstimateParallelism()

setDontEstimateParallelism

public void setDontEstimateParallelism(boolean dontEstimateParallelism)

isOverrideIntermediateParallelism

public boolean isOverrideIntermediateParallelism()

setOverrideIntermediateParallelism

public void setOverrideIntermediateParallelism(boolean overrideIntermediateParallelism)

getSplitParent
```
public OperatorKey getSplitParent()
```

setSplitParent

public void setSplitParent(OperatorKey splitParent)

setSplitter
```
public void setSplitter(boolean spl)
```

isSplitter
```
public boolean isSplitter()
```

isClosed
```
public boolean isClosed()
```

setClosed

public void setClosed(boolean closed)

isIndexer
```
public boolean isIndexer()
```

markIndexer
```
public void markIndexer()
```

isSampler
```
public boolean isSampler()
```

markSampler
```
public void markSampler()
```

isSampleAggregation
```
public boolean isSampleAggregation()
```

markSampleAggregation
```
public void markSampleAggregation()
```

isSampleBasedPartitioner

public boolean isSampleBasedPartitioner()

markSampleBasedPartitioner

public void markSampleBasedPartitioner()

isGlobalSort
```
public boolean isGlobalSort()
```

markGlobalSort
```
public void markGlobalSort()
```

isGroupBy
```
public boolean isGroupBy()
```

markGroupBy
```
public void markGroupBy()
```

isCogroup
```
public boolean isCogroup()
```

markCogroup
```
public void markCogroup()
```

isRegularJoin
```
public boolean isRegularJoin()
```

markRegularJoin
```
public void markRegularJoin()
```

isSkewedJoin
```
public boolean isSkewedJoin()
```

markSkewedJoin
```
public void markSkewedJoin()
```

isLimit
```
public boolean isLimit()
```

markLimit
```
public void markLimit()
```

isLimitAfterSort
```
public boolean isLimitAfterSort()
```

markLimitAfterSort
```
public void markLimitAfterSort()
```

isUnion
```
public boolean isUnion()
```

markUnion
```
public void markUnion()
```

isDistinct
```
public boolean isDistinct()
```

markDistinct
```
public void markDistinct()
```

isNative
```
public boolean isNative()
```

markNative
```
public void markNative()
```

isRankCounter
```
public boolean isRankCounter()
```

markRankCounter
```
public void markRankCounter()
```

isBuildBloom
```
public boolean isBuildBloom()
```

markBuildBloom
```
public void markBuildBloom()
```

isFilterBloom
```
public boolean isFilterBloom()
```

markFilterBloom
```
public void markFilterBloom()
```

copyFeatures

public void copyFeatures(TezOperator copyFrom,
                         List<TezOperator.OPER_FEATURE> excludeFeatures)

setNeedEstimatedQuantile

public void setNeedEstimatedQuantile(boolean needEstimateParallelism)

isNeedEstimateParallelism

public boolean isNeedEstimateParallelism()

isUseSecondaryKey
```
public boolean isUseSecondaryKey()
```

setUseSecondaryKey

public void setUseSecondaryKey(boolean useSecondaryKey)

getUnionMembers

public List<OperatorKey> getUnionMembers()

getVertexGroupMembers

public List<OperatorKey> getVertexGroupMembers()

addUnionPredecessor

public void addUnionPredecessor(OperatorKey unionPredecessor)

setVertexGroupMembers

public void setVertexGroupMembers(List<OperatorKey> vertexGroupMembers)

isVertexGroup
```
public boolean isVertexGroup()
```

getVertexGroupInfo

public TezOperator.VertexGroupInfo getVertexGroupInfo()

setVertexGroupInfo

public void setVertexGroupInfo(TezOperator.VertexGroupInfo vertexGroup)

addVertexGroupStore

public void addVertexGroupStore(OperatorKey storeKey,
                                OperatorKey vertexGroupKey)

removeVertexGroupStore

public void removeVertexGroupStore(OperatorKey vertexGroupKey)

getVertexGroupStores

public Map<OperatorKey,OperatorKey> getVertexGroupStores()

name
```
public String name()
```
Specified by:

name in class Operator<TezOpPlanVisitor>

toString
```
public String toString()
```
Overrides:

toString in class Operator<TezOpPlanVisitor>

needSegmentBelow
```
public boolean needSegmentBelow()
```

setSortOrder

public void setSortOrder(boolean[] sortOrder)

getSortOrder
```
public boolean[] getSortOrder()
```

getSampleOperator

public TezOperator getSampleOperator()

setSampleOperator

public void setSampleOperator(TezOperator sampleOperator)

getSortOperator
```
public TezOperator getSortOperator()
```

setSortOperator

public void setSortOperator(TezOperator sortOperator)

noCombineSmallSplits
```
protected void noCombineSmallSplits()
```

combineSmallSplits
```
public boolean combineSmallSplits()
```

addCrossKey
```
public void addCrossKey(String key)
```

getCrossKeys
```
public List<String> getCrossKeys()
```

isUseMRMapSettings
```
public boolean isUseMRMapSettings()
```

setUseMRMapSettings

public void setUseMRMapSettings(boolean useMRMapSettings)

getVertexParallelism
```
public int getVertexParallelism()
```

setVertexParallelism

public void setVertexParallelism(int vertexParallelism)

getLoaderInfo

public TezOperator.LoaderInfo getLoaderInfo()

getTotalInputFilesSize
```
public long getTotalInputFilesSize()
```

setTotalInputFilesSize

public void setTotalInputFilesSize(long totalInputFilesSize)

setUseGraceParallelism

public void setUseGraceParallelism(boolean useGraceParallelism)

isUseGraceParallelism

public boolean isUseGraceParallelism()

getParallelismFactor

public double getParallelismFactor(TezOperator successor)
                            throws VisitorException

Throws:: VisitorException

isIntermediateReducer

public Boolean isIntermediateReducer()
                              throws IOException

Throws:: IOException

setIntermediateReducer

public void setIntermediateReducer(Boolean intermediateReducer)

Class TezOperator

Nested Class Summary

Field Summary

Fields inherited from class org.apache.pig.impl.plan.Operator

Constructor Summary

Method Summary

Methods inherited from class org.apache.pig.impl.plan.Operator

Methods inherited from class java.lang.Object

Field Detail

plan

outEdges

inEdges

UDFs

scalars

Constructor Detail

TezOperator

Method Detail

getProcessorName

visit

supportsMultipleInputs

supportsMultipleOutputs

getRequestedParallelism

setRequestedParallelism

setRequestedParallelismByReference

getEstimatedParallelism

setEstimatedParallelism

getEffectiveParallelism

isDontEstimateParallelism

setDontEstimateParallelism

isOverrideIntermediateParallelism

setOverrideIntermediateParallelism

getSplitParent

setSplitParent

setSplitter

isSplitter

isClosed

setClosed

isIndexer

markIndexer

isSampler

markSampler

isSampleAggregation

markSampleAggregation

isSampleBasedPartitioner

markSampleBasedPartitioner

isGlobalSort

markGlobalSort

isGroupBy

markGroupBy

isCogroup

markCogroup

isRegularJoin

markRegularJoin

isSkewedJoin

markSkewedJoin

isLimit

markLimit

isLimitAfterSort

markLimitAfterSort

isUnion

markUnion

isDistinct

markDistinct

isNative

markNative

isRankCounter

markRankCounter

isBuildBloom

markBuildBloom

isFilterBloom

markFilterBloom

copyFeatures

setNeedEstimatedQuantile

isNeedEstimateParallelism

isUseSecondaryKey

setUseSecondaryKey

getUnionMembers

getVertexGroupMembers

addUnionPredecessor

setVertexGroupMembers