org.apache.pig.backend.hadoop.executionengine.tez.plan

Class TezOperPlan

java.lang.Object

org.apache.pig.impl.plan.OperatorPlan<TezOperator>

org.apache.pig.backend.hadoop.executionengine.tez.plan.TezOperPlan

All Implemented Interfaces:

Serializable, Cloneable, Iterable<TezOperator>

public class TezOperPlan
extends OperatorPlan<TezOperator>

A Plan used to create the plan of Tez operators which can be converted into the Job Control object. This is necessary to capture the dependencies among jobs.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.impl.plan.OperatorPlan

OperatorPlan.IndexHelper<E>

Field Summary

Fields inherited from class org.apache.pig.impl.plan.OperatorPlan

log, mFromEdges, mKeys, mOps, mSoftFromEdges, mSoftToEdges, mToEdges

Constructor Summary

Constructors

Constructor and Description
TezOperPlan()

Method Summary

Methods

Modifier and Type	Method and Description
void	addExtraResource(String resourceName, org.apache.hadoop.fs.Path remoteFsPath)
void	addExtraResource(URL url)
boolean	disconnect(TezOperator from, TezOperator to) Remove an edge from between two nodes.
org.apache.hadoop.security.Credentials	getCredentials()
int	getEstimatedTotalParallelism()
Map<String,org.apache.hadoop.yarn.api.records.LocalResource>	getExtraResources()
static List<TezOperator>	getGrandParentsForGraceParallelism(TezOperPlan tezPlan, TezOperator op)
void	moveTree(TezOperator root, TezOperPlan newPlan) Move everything below a given operator to the new operator plan.
void	remove(TezOperator op) Remove an operator from the plan.
void	setEstimatedParallelism(int estimatedTotalParallelism)
String	toString()

Methods inherited from class org.apache.pig.impl.plan.OperatorPlan

add, addAsLeaf, connect, createSoftLink, doInsertBetween, dump, getKeys, getLeaves, getOperator, getOperatorKey, getPredecessors, getRoots, getSoftLinkPredecessors, getSoftLinkSuccessors, getSuccessors, insertBetween, isSingleLeafPlan, iterator, merge, mergeSharedPlan, pathExists, pushAfter, pushBefore, removeAndReconnect, removeAndReconnectMultiSucc, removeSoftLink, replace, size, swap, trimAbove, trimBelow

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TezOperPlan

public TezOperPlan()

Method Detail

getCredentials

public org.apache.hadoop.security.Credentials getCredentials()

getEstimatedTotalParallelism

public int getEstimatedTotalParallelism()

setEstimatedParallelism

public void setEstimatedParallelism(int estimatedTotalParallelism)

toString

public String toString()

Overrides:

toString in class Object

addExtraResource

public void addExtraResource(URL url)
                      throws IOException

Throws:

IOException

addExtraResource

public void addExtraResource(String resourceName,
                    org.apache.hadoop.fs.Path remoteFsPath)
                      throws IOException

Throws:

IOException

getExtraResources

public Map<String,org.apache.hadoop.yarn.api.records.LocalResource> getExtraResources()
                                                                               throws Exception

Throws:

Exception

remove

public void remove(TezOperator op)

Description copied from class: OperatorPlan

Remove an operator from the plan. Any edges that the node has will be removed as well.

Overrides:

remove in class OperatorPlan<TezOperator>

Parameters:

op - Operator to remove.

disconnect

public boolean disconnect(TezOperator from,
                 TezOperator to)

Description copied from class: OperatorPlan

Remove an edge from between two nodes. Use OperatorPlan.insertBetween(Operator, Operator, Operator) if disconnect is used in the process of inserting a new node between two nodes by calling disconnect followed by a connect.

Overrides:

disconnect in class OperatorPlan<TezOperator>

Parameters:

from - Operator data would flow from.

to - Operator data would flow to.

Returns:

true if the nodes were connected according to the specified data flow, false otherwise.

moveTree

public void moveTree(TezOperator root,
            TezOperPlan newPlan)
              throws PlanException

Move everything below a given operator to the new operator plan. The specified operator will be moved and will be the root of the new operator plan

Parameters:

root - Operator to move everything under including the root operator

newPlan - new operator plan to move things into

Throws:

PlanException

getGrandParentsForGraceParallelism

public static List<TezOperator> getGrandParentsForGraceParallelism(TezOperPlan tezPlan,
                                                   TezOperator op)