Modifier and Type | Method and Description |
---|---|
void |
PigServer.discardBatch()
Discards a batch of Pig commands.
|
static String |
LoadFunc.getAbsolutePath(String location,
org.apache.hadoop.fs.Path curDir)
Construct the absolute path from the file location and the current
directory.
|
List<FuncSpec> |
EvalFunc.getArgToFuncMapping()
Allow a UDF to specify type specific implementations of itself.
|
boolean |
PigServer.isBatchEmpty()
Returns whether there is anything to process in the current batch.
|
protected PigStats |
PigServer.launchPlan(LogicalPlan lp,
String jobName)
A common method for launching the jobs according to the logical plan
|
void |
PigServer.printAliases()
Intended to be used by unit tests only.
|
LoadPushDown.RequiredFieldResponse |
LoadPushDown.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList)
Indicate to the loader fields that will be needed.
|
static void |
ResourceSchema.ResourceFieldSchema.throwInvalidSchemaException() |
Modifier and Type | Method and Description |
---|---|
PigStats |
ExecutionEngine.launchPig(LogicalPlan lp,
String grpName,
PigContext pc)
This method is responsible for the actual execution of a LogicalPlan.
|
Modifier and Type | Class and Description |
---|---|
class |
JobCreationException |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
HExecutionEngine.compile(LogicalPlan plan,
Properties properties) |
void |
HExecutionEngine.explain(LogicalPlan lp,
PigContext pc,
PrintStream ps,
String format,
boolean verbose,
File file,
String suffix) |
PigStats |
HExecutionEngine.launchPig(LogicalPlan lp,
String grpName,
PigContext pc) |
Modifier and Type | Class and Description |
---|---|
class |
MRCompilerException |
Modifier and Type | Class and Description |
---|---|
class |
LogicalToPhysicalTranslatorException |
Modifier and Type | Method and Description |
---|---|
void |
PatternNode.accept(PlanVisitor v) |
boolean |
PatternNode.isEqual(Operator operator) |
Modifier and Type | Method and Description |
---|---|
void |
POMergeJoin.setupRightPipeline(PhysicalPlan rightPipeline) |
Modifier and Type | Class and Description |
---|---|
class |
SparkCompilerException
Create a new SparkCompilerException with null as the error message.
|
Modifier and Type | Class and Description |
---|---|
class |
TezCompilerException |
Modifier and Type | Method and Description |
---|---|
LoadPushDown.RequiredFieldResponse |
HBaseStorage.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) |
protected void |
HBaseStorage.storeProjectedFieldNames(LoadPushDown.RequiredFieldList requiredFieldList)
Stores the requiredFieldsList as a serialized object so it can be fetched on the cluster.
|
Constructor and Description |
---|
GenericInvoker(String fullName) |
GenericInvoker(String fullName,
String paramSpecsStr) |
GenericInvoker(String fullName,
String paramSpecsStr,
String isStatic) |
InvokeForDouble(String fullName) |
InvokeForDouble(String fullName,
String paramSpecsStr) |
InvokeForDouble(String fullName,
String paramSpecsStr,
String isStatic) |
InvokeForFloat(String fullName) |
InvokeForFloat(String fullName,
String paramSpecsStr) |
InvokeForFloat(String fullName,
String paramSpecsStr,
String isStatic) |
InvokeForInt(String fullName) |
InvokeForInt(String fullName,
String paramSpecsStr) |
InvokeForInt(String fullName,
String paramSpecsStr,
String isStatic) |
InvokeForLong(String fullName) |
InvokeForLong(String fullName,
String paramSpecsStr) |
InvokeForLong(String fullName,
String paramSpecsStr,
String isStatic) |
InvokeForString(String fullName) |
InvokeForString(String fullName,
String paramSpecsStr) |
InvokeForString(String fullName,
String paramSpecsStr,
String isStatic) |
Invoker(String fullName,
String paramSpecsStr) |
Invoker(String fullName,
String paramSpecsStr,
String isStatic) |
ParquetLoader() |
ParquetLoader(String requestedSchemaStr) |
ParquetStorer() |
Modifier and Type | Method and Description |
---|---|
static Schema.FieldSchema |
DataType.determineFieldSchema(Object o)
Determine the field schema of an object
|
static Schema.FieldSchema |
DataType.determineFieldSchema(ResourceSchema.ResourceFieldSchema rcFieldSchema)
Determine the field schema of an ResourceFieldSchema
|
Modifier and Type | Class and Description |
---|---|
class |
SchemaMergeException |
Modifier and Type | Method and Description |
---|---|
static Schema |
Schema.generateNestedSchema(byte topLevelType,
byte... innerTypes) |
Schema.FieldSchema |
Schema.getField(int fieldNum)
Given a field number, find the associated FieldSchema.
|
Schema.FieldSchema |
Schema.getField(String alias)
Given an alias name, find the associated FieldSchema.
|
Schema.FieldSchema |
Schema.getFieldSubNameMatch(String alias)
Given an alias name, find the associated FieldSchema.
|
static Schema |
Schema.getPigSchema(ResourceSchema rSchema) |
int |
Schema.getPosition(String alias)
Given an alias, find the associated position of the field schema.
|
int |
Schema.getPositionSubName(String alias)
Given an alias, find the associated position of the field schema.
|
static Schema |
SchemaUtil.newBagSchema(Byte[] dataTypes)
Create a new tuple schema according one array: the type of fields, the
tuple name is t, and the bag name is b.
|
static Schema |
SchemaUtil.newBagSchema(List<Byte> dataTypes)
Create a new tuple schema according one list: types of fields, the
default names of fields are f0,f1,f2..., and the tuple is t, the bag name
is b.
|
static Schema |
SchemaUtil.newBagSchema(List<String> names,
List<Byte> dataTypes)
Create a bag schema according two list: name of fields, type of fields,
and the default bag name is b, the default tuple name is t.
|
static Schema |
SchemaUtil.newBagSchema(String[] names,
Byte[] dataTypes)
Create a new tuple schema according two arrays: names of field,types of
fields.
|
static Schema |
SchemaUtil.newBagSchema(String bagName,
String tupleName,
List<String> fieldNames,
List<Byte> dataTypes)
Create a bag schema according the bag name,tuple name and two list: name
of fields, type of fields
|
static Schema |
SchemaUtil.newBagSchema(String bagName,
String tupleName,
String[] fieldNames,
Byte[] dataTypes) |
static Schema |
SchemaUtil.newTupleSchema(Byte[] dataTypes)
Create a new tuple schema according one array: types of fields, the
default names of fields are f0,f1,f2..., and the tuple name is t.
|
static Schema |
SchemaUtil.newTupleSchema(List<Byte> dataTypes)
Create a new tuple schema according one list: types of fields, the
default names of fields are f0,f1,f2..., and the tuple name is t.
|
static Schema |
SchemaUtil.newTupleSchema(List<String> fieldNames,
List<Byte> dataTypes)
Create a new tuple schema according the two list: names of fields, types
of fields, the default tuple name is t.
|
static Schema |
SchemaUtil.newTupleSchema(String[] names,
Byte[] dataTypes)
Create a new tuple schema according the two arrays: names of fields,
types of fields, the default tuple name is t.
|
static Schema |
SchemaUtil.newTupleSchema(String tupleName,
List<String> fieldNames,
List<Byte> dataTypes)
Create a new tuple schema according the tuple name and two list: names of
fields, types of fields
|
static Schema |
SchemaUtil.newTupleSchema(String tupleName,
String[] fieldNames,
Byte[] dataTypes)
Create a new tuple schema according the tuple name and two arrays: names
of fields, types of fields
|
void |
Schema.reconcile(Schema other)
Reconcile this schema with another schema.
|
static void |
Schema.stringifySchema(StringBuilder sb,
Schema schema,
byte type) |
static void |
Schema.stringifySchema(StringBuilder sb,
Schema schema,
byte type,
int indentLevel) |
Constructor and Description |
---|
FieldSchema(String a,
Schema s,
byte t)
Constructor for tuple fields.
|
Modifier and Type | Class and Description |
---|---|
class |
TypeCheckerException |
Modifier and Type | Class and Description |
---|---|
class |
PlanException |
class |
PlanValidationException |
class |
VisitorException |
Modifier and Type | Class and Description |
---|---|
class |
OptimizerException |
Modifier and Type | Method and Description |
---|---|
abstract void |
Operator.accept(PlanVisitor v)
Accept a visitor at this node in the graph.
|
protected LogicalExpression |
FilterExtractor.addToFilterPlan(LogicalExpression op) |
protected FilterExtractor.KeyState |
FilterExtractor.checkPushDown(BinaryExpression binExpr) |
protected FilterExtractor.KeyState |
FilterExtractor.checkPushDown(LogicalExpression op) |
protected FilterExtractor.KeyState |
PredicatePushDownFilterExtractor.checkPushDown(ProjectExpression project) |
protected FilterExtractor.KeyState |
PartitionFilterExtractor.checkPushDown(ProjectExpression project) |
protected abstract FilterExtractor.KeyState |
FilterExtractor.checkPushDown(ProjectExpression project) |
protected FilterExtractor.KeyState |
FilterExtractor.checkPushDown(UnaryExpression unaryExpr) |
Pair<Integer,Integer> |
OperatorSubPlan.disconnect(Operator from,
Operator to) |
Pair<Integer,Integer> |
OperatorPlan.disconnect(Operator from,
Operator to)
Disconnect two operators in the plan.
|
Pair<Integer,Integer> |
BaseOperatorPlan.disconnect(Operator from,
Operator to)
Disconnect two operators in the plan.
|
protected void |
DependencyOrderWalker.doAllPredecessors(Operator node,
Set<Operator> seen,
Collection<Operator> fifo) |
protected void |
ReverseDependencyOrderWalkerWOSeenChk.doAllSuccessors(Operator node,
Collection<Operator> fifo) |
protected void |
ReverseDependencyOrderWalker.doAllSuccessors(Operator node,
Set<Operator> seen,
Collection<Operator> fifo) |
void |
BaseOperatorPlan.explain(PrintStream ps,
String format,
boolean verbose) |
protected Expression |
FilterExtractor.getExpression(BinaryExpression binOp,
Expression.OpType opType) |
Expression |
FilterExtractor.getExpression(LogicalExpression op) |
protected Expression |
FilterExtractor.getExpression(UnaryExpression unaryOp,
Expression.OpType opType) |
void |
OperatorSubPlan.insertBetween(Operator pred,
Operator operatorToInsert,
Operator succ) |
void |
OperatorPlan.insertBetween(Operator pred,
Operator operatorToInsert,
Operator succ)
This method insert node operatorToInsert between pred and succ.
|
void |
BaseOperatorPlan.insertBetween(Operator pred,
Operator operatorToInsert,
Operator succ) |
abstract boolean |
Operator.isEqual(Operator operator)
This is like a shallow equals comparison.
|
boolean |
OperatorSubPlan.isEqual(OperatorPlan other) |
boolean |
OperatorPlan.isEqual(OperatorPlan other)
This is like a shallow comparison.
|
boolean |
BaseOperatorPlan.isEqual(OperatorPlan other) |
protected static boolean |
BaseOperatorPlan.isEqual(OperatorPlan p1,
OperatorPlan p2) |
void |
BaseOperatorPlan.moveTree(Operator root,
BaseOperatorPlan newPlan)
Move everything below a given operator to the new operator plan.
|
protected void |
PlanVisitor.popWalker()
Pop the next to previous walker off of the stack and set it as the current
walker.
|
void |
OperatorSubPlan.remove(Operator op) |
void |
OperatorPlan.remove(Operator op)
Remove an operator from the plan.
|
void |
BaseOperatorPlan.remove(Operator op)
Remove an operator from the plan.
|
void |
OperatorSubPlan.removeAndReconnect(Operator operatorToRemove) |
void |
OperatorPlan.removeAndReconnect(Operator operatorToRemove)
This method remove a node operatorToRemove.
|
void |
BaseOperatorPlan.removeAndReconnect(Operator operatorToRemove) |
void |
OperatorSubPlan.replace(Operator oldOperator,
Operator newOperator) |
void |
OperatorPlan.replace(Operator oldOperator,
Operator newOperator)
This method replace the oldOperator with the newOperator, make all connection
to the new operator in the place of old operator
|
void |
BaseOperatorPlan.replace(Operator oldOperator,
Operator newOperator) |
void |
BaseOperatorPlan.trimBelow(Operator op)
Trim everything below a given operator.
|
void |
PredicatePushDownFilterExtractor.visit() |
void |
PlanVisitor.visit()
Entry point for visiting the plan.
|
void |
FilterExtractor.visit() |
void |
SubtreeDependencyOrderWalker.walk(PlanVisitor visitor) |
void |
ReverseDependencyOrderWalkerWOSeenChk.walk(PlanVisitor visitor)
Begin traversing the graph.
|
void |
ReverseDependencyOrderWalker.walk(PlanVisitor visitor)
Begin traversing the graph.
|
abstract void |
PlanWalker.walk(PlanVisitor visitor)
Begin traversing the graph.
|
void |
DepthFirstWalker.walk(PlanVisitor visitor)
Begin traversing the graph.
|
void |
DependencyOrderWalker.walk(PlanVisitor visitor)
Begin traversing the graph.
|
Modifier and Type | Method and Description |
---|---|
static LOForEach |
Util.addForEachAfter(LogicalPlan plan,
LogicalRelationalOperator op,
int branch,
Set<Integer> columnsToDrop) |
static Schema |
Util.fixSchemaAddTupleInBag(Schema sch)
If schema argument has fields where a bag does not contain a tuple schema,
it inserts a tuple schema.
|
static <C extends LogicalRelationalOperator> |
Util.getLogicalRelationalOperators(LogicalPlan plan,
Class<C> opClass)
Returns a LinkedList of operators contained within the physical plan
which implement the supplied class, in dependency order.
|
Modifier and Type | Method and Description |
---|---|
void |
UserFuncExpression.accept(PlanVisitor v) |
void |
SubtractExpression.accept(PlanVisitor v) |
void |
ScalarExpression.accept(PlanVisitor v) |
void |
RegexExpression.accept(PlanVisitor v) |
void |
ProjectExpression.accept(PlanVisitor v) |
void |
OrExpression.accept(PlanVisitor v) |
void |
NotExpression.accept(PlanVisitor v) |
void |
NotEqualExpression.accept(PlanVisitor v) |
void |
NegativeExpression.accept(PlanVisitor v) |
void |
MultiplyExpression.accept(PlanVisitor v) |
void |
ModExpression.accept(PlanVisitor v) |
void |
MapLookupExpression.accept(PlanVisitor v) |
void |
LessThanExpression.accept(PlanVisitor v) |
void |
LessThanEqualExpression.accept(PlanVisitor v) |
void |
IsNullExpression.accept(PlanVisitor v) |
void |
GreaterThanExpression.accept(PlanVisitor v) |
void |
GreaterThanEqualExpression.accept(PlanVisitor v) |
void |
EqualExpression.accept(PlanVisitor v) |
void |
DivideExpression.accept(PlanVisitor v) |
void |
DereferenceExpression.accept(PlanVisitor v) |
void |
ConstantExpression.accept(PlanVisitor v) |
void |
CastExpression.accept(PlanVisitor v) |
void |
BinCondExpression.accept(PlanVisitor v) |
void |
AndExpression.accept(PlanVisitor v) |
void |
AddExpression.accept(PlanVisitor v) |
LogicalExpressionPlan |
LogicalExpressionPlan.deepCopy() |
LogicalExpression |
UserFuncExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
SubtractExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
ScalarExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
RegexExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
ProjectExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
OrExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
NotExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
NotEqualExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
NegativeExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
MultiplyExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
ModExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
MapLookupExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
abstract LogicalExpression |
LogicalExpression.deepCopy(LogicalExpressionPlan lgExpPlan)
Create the deep copy of this expression and add that into the passed
LogicalExpressionPlan Return the copy of this expression with updated
logical expression plan.
|
LogicalExpression |
LessThanExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
LessThanEqualExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
IsNullExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
GreaterThanExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
GreaterThanEqualExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
EqualExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
DivideExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
DereferenceExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
ConstantExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
CastExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
BinCondExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
AndExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
LogicalExpression |
AddExpression.deepCopy(LogicalExpressionPlan lgExpPlan) |
protected abstract void |
AllSameExpressionVisitor.execute(LogicalExpression op)
Method to call on every node in the logical expression plan.
|
void |
LogicalExpressionPlan.explain(PrintStream ps,
String format,
boolean verbose) |
LogicalRelationalOperator |
ProjectExpression.findReferent()
Find the LogicalRelationalOperator that this projection refers to.
|
List<LogicalExpression> |
UserFuncExpression.getArguments() |
LogicalExpression |
BinCondExpression.getCondition()
Returns the operator which handles this condition
|
LogicalExpression |
UnaryExpression.getExpression()
Get the expression that this unary expression operators on.
|
LogicalSchema.LogicalFieldSchema |
UserFuncExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
SubtractExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
ScalarExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
RegexExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
ProjectExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
OrExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
NotExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
NotEqualExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
NegativeExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
MultiplyExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
ModExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
MapLookupExpression.getFieldSchema() |
abstract LogicalSchema.LogicalFieldSchema |
LogicalExpression.getFieldSchema()
Get the field schema for the output of this expression operator.
|
LogicalSchema.LogicalFieldSchema |
LessThanExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
LessThanEqualExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
IsNullExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
GreaterThanExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
GreaterThanEqualExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
EqualExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
DivideExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
DereferenceExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
ConstantExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
CastExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
BinCondExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
AndExpression.getFieldSchema() |
LogicalSchema.LogicalFieldSchema |
AddExpression.getFieldSchema() |
LogicalExpression |
BinaryExpression.getLhs()
Get the left hand side of this binary expression.
|
LogicalExpression |
BinCondExpression.getLhs()
Get the left hand side of this expression.
|
LogicalExpression |
MapLookupExpression.getMap() |
LogicalExpression |
DereferenceExpression.getReferredExpression() |
LogicalExpression |
BinaryExpression.getRhs()
Get the right hand side of this binary expression.
|
LogicalExpression |
BinCondExpression.getRhs()
Get the right hand side of this expression.
|
byte |
ProjectExpression.getType() |
byte |
LogicalExpression.getType()
Get the data type for this expression.
|
void |
ConstantExpression.inheritSchema(LogicalExpression expr) |
boolean |
UserFuncExpression.isDeterministic() |
boolean |
UserFuncExpression.isEqual(Operator other) |
boolean |
SubtractExpression.isEqual(Operator other) |
boolean |
RegexExpression.isEqual(Operator other) |
boolean |
ProjectExpression.isEqual(Operator other) |
boolean |
OrExpression.isEqual(Operator other) |
boolean |
NotExpression.isEqual(Operator other) |
boolean |
NotEqualExpression.isEqual(Operator other) |
boolean |
NegativeExpression.isEqual(Operator other) |
boolean |
MultiplyExpression.isEqual(Operator other) |
boolean |
ModExpression.isEqual(Operator other) |
boolean |
MapLookupExpression.isEqual(Operator other) |
boolean |
LessThanExpression.isEqual(Operator other) |
boolean |
LessThanEqualExpression.isEqual(Operator other) |
boolean |
IsNullExpression.isEqual(Operator other) |
boolean |
GreaterThanExpression.isEqual(Operator other) |
boolean |
GreaterThanEqualExpression.isEqual(Operator other) |
boolean |
EqualExpression.isEqual(Operator other) |
boolean |
DivideExpression.isEqual(Operator other) |
boolean |
DereferenceExpression.isEqual(Operator other) |
boolean |
ConstantExpression.isEqual(Operator other) |
boolean |
CastExpression.isEqual(Operator other) |
boolean |
BinCondExpression.isEqual(Operator other) |
boolean |
AndExpression.isEqual(Operator other) |
boolean |
AddExpression.isEqual(Operator other) |
boolean |
LogicalExpressionPlan.isEqual(OperatorPlan other) |
void |
LogicalExpression.neverUseForRealSetFieldSchema(LogicalSchema.LogicalFieldSchema fs) |
void |
ProjectExpression.setColumnNumberFromAlias()
If there is an alias, finds the column number from it.
|
void |
ProjectExpression.setEndAlias(String endAlias) |
void |
ProjectExpression.setStartAlias(String startAlias) |
void |
LogicalExpressionVisitor.visit(AddExpression op) |
void |
ExpToPhyTranslationVisitor.visit(AddExpression op) |
void |
AllSameExpressionVisitor.visit(AddExpression op) |
void |
LogicalExpressionVisitor.visit(AndExpression op) |
void |
ExpToPhyTranslationVisitor.visit(AndExpression op) |
void |
AllSameExpressionVisitor.visit(AndExpression andExpr) |
void |
LogicalExpressionVisitor.visit(BinCondExpression op) |
void |
ExpToPhyTranslationVisitor.visit(BinCondExpression op) |
void |
AllSameExpressionVisitor.visit(BinCondExpression op) |
void |
LogicalExpressionVisitor.visit(CastExpression op) |
void |
ExpToPhyTranslationVisitor.visit(CastExpression op) |
void |
AllSameExpressionVisitor.visit(CastExpression cast) |
void |
LogicalExpressionVisitor.visit(ConstantExpression op) |
void |
ExpToPhyTranslationVisitor.visit(ConstantExpression op) |
void |
AllSameExpressionVisitor.visit(ConstantExpression constant) |
void |
LogicalExpressionVisitor.visit(DereferenceExpression op) |
void |
ExpToPhyTranslationVisitor.visit(DereferenceExpression op) |
void |
AllSameExpressionVisitor.visit(DereferenceExpression derefenceExpression) |
void |
LogicalExpressionVisitor.visit(DivideExpression op) |
void |
ExpToPhyTranslationVisitor.visit(DivideExpression op) |
void |
AllSameExpressionVisitor.visit(DivideExpression op) |
void |
LogicalExpressionVisitor.visit(EqualExpression op) |
void |
ExpToPhyTranslationVisitor.visit(EqualExpression op) |
void |
AllSameExpressionVisitor.visit(EqualExpression equal) |
void |
LogicalExpressionVisitor.visit(GreaterThanEqualExpression op) |
void |
ExpToPhyTranslationVisitor.visit(GreaterThanEqualExpression op) |
void |
AllSameExpressionVisitor.visit(GreaterThanEqualExpression op) |
void |
LogicalExpressionVisitor.visit(GreaterThanExpression op) |
void |
ExpToPhyTranslationVisitor.visit(GreaterThanExpression op) |
void |
AllSameExpressionVisitor.visit(GreaterThanExpression greaterThanExpression) |
void |
LogicalExpressionVisitor.visit(IsNullExpression op) |
void |
ExpToPhyTranslationVisitor.visit(IsNullExpression op) |
void |
AllSameExpressionVisitor.visit(IsNullExpression op) |
void |
LogicalExpressionVisitor.visit(LessThanEqualExpression op) |
void |
ExpToPhyTranslationVisitor.visit(LessThanEqualExpression op) |
void |
AllSameExpressionVisitor.visit(LessThanEqualExpression op) |
void |
LogicalExpressionVisitor.visit(LessThanExpression op) |
void |
ExpToPhyTranslationVisitor.visit(LessThanExpression op) |
void |
AllSameExpressionVisitor.visit(LessThanExpression lessThanExpression) |
void |
LogicalExpressionVisitor.visit(MapLookupExpression op) |
void |
ExpToPhyTranslationVisitor.visit(MapLookupExpression op) |
void |
AllSameExpressionVisitor.visit(MapLookupExpression op) |
void |
LogicalExpressionVisitor.visit(ModExpression op) |
void |
ExpToPhyTranslationVisitor.visit(ModExpression op) |
void |
AllSameExpressionVisitor.visit(ModExpression op) |
void |
LogicalExpressionVisitor.visit(MultiplyExpression op) |
void |
ExpToPhyTranslationVisitor.visit(MultiplyExpression op) |
void |
AllSameExpressionVisitor.visit(MultiplyExpression op) |
void |
LogicalExpressionVisitor.visit(NegativeExpression op) |
void |
ExpToPhyTranslationVisitor.visit(NegativeExpression op) |
void |
AllSameExpressionVisitor.visit(NegativeExpression op) |
void |
LogicalExpressionVisitor.visit(NotEqualExpression op) |
void |
ExpToPhyTranslationVisitor.visit(NotEqualExpression op) |
void |
AllSameExpressionVisitor.visit(NotEqualExpression op) |
void |
LogicalExpressionVisitor.visit(NotExpression op) |
void |
ExpToPhyTranslationVisitor.visit(NotExpression op) |
void |
AllSameExpressionVisitor.visit(NotExpression op) |
void |
LogicalExpressionVisitor.visit(OrExpression op) |
void |
ExpToPhyTranslationVisitor.visit(OrExpression op) |
void |
AllSameExpressionVisitor.visit(OrExpression exp) |
void |
LogicalExpressionVisitor.visit(ProjectExpression op) |
void |
ExpToPhyTranslationVisitor.visit(ProjectExpression op) |
void |
AllSameExpressionVisitor.visit(ProjectExpression project) |
void |
LogicalExpressionVisitor.visit(RegexExpression op) |
void |
ExpToPhyTranslationVisitor.visit(RegexExpression op) |
void |
AllSameExpressionVisitor.visit(RegexExpression op) |
void |
LogicalExpressionVisitor.visit(ScalarExpression op) |
void |
LogicalExpressionVisitor.visit(SubtractExpression op) |
void |
ExpToPhyTranslationVisitor.visit(SubtractExpression op) |
void |
AllSameExpressionVisitor.visit(SubtractExpression op) |
void |
LogicalExpressionVisitor.visit(UserFuncExpression op) |
void |
ExpToPhyTranslationVisitor.visit(UserFuncExpression op) |
void |
AllSameExpressionVisitor.visit(UserFuncExpression op) |
Constructor and Description |
---|
AllSameExpressionVisitor(OperatorPlan plan,
PlanWalker walker) |
ExpToPhyTranslationVisitor(OperatorPlan plan,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
ExpToPhyTranslationVisitor(OperatorPlan plan,
PlanWalker walker,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
Map<Operator,PhysicalOperator> map) |
LogicalExpressionVisitor(OperatorPlan p,
PlanWalker walker) |
Modifier and Type | Method and Description |
---|---|
protected void |
LogicalPlanPrinter.depthFirstLP() |
protected abstract void |
AllSameRalationalNodesVisitor.execute(LogicalRelationalOperator op)
Method to call on every node in the logical plan.
|
protected LogicalExpressionVisitor |
ProjectionPatcher.ProjectionFinder.getVisitor(LogicalExpressionPlan expr) |
protected abstract LogicalExpressionVisitor |
AllExpressionVisitor.getVisitor(LogicalExpressionPlan expr)
Get a new instance of the expression visitor to apply to
a given expression.
|
void |
SchemaPatcher.transformed(OperatorPlan fp,
OperatorPlan tp) |
void |
ProjectionPatcher.transformed(OperatorPlan fp,
OperatorPlan tp) |
void |
LogicalPlanPrinter.visit() |
void |
UidResetter.visit(LOCogroup loCogroup) |
void |
SchemaResetter.visit(LOCogroup loCogroup) |
void |
AllSameRalationalNodesVisitor.visit(LOCogroup cg) |
void |
AllExpressionVisitor.visit(LOCogroup cg) |
void |
UidResetter.visit(LOCross loCross) |
void |
SchemaResetter.visit(LOCross loCross) |
void |
AllSameRalationalNodesVisitor.visit(LOCross cross) |
void |
SchemaResetter.visit(LOCube loCube) |
void |
AllSameRalationalNodesVisitor.visit(LOCube cube) |
void |
AllExpressionVisitor.visit(LOCube cu) |
void |
UidResetter.visit(LODistinct loDistinct) |
void |
SchemaResetter.visit(LODistinct loDistinct) |
void |
AllSameRalationalNodesVisitor.visit(LODistinct distinct) |
void |
UidResetter.visit(LOFilter filter) |
void |
SchemaResetter.visit(LOFilter filter) |
void |
AllSameRalationalNodesVisitor.visit(LOFilter filter) |
void |
AllExpressionVisitor.visit(LOFilter filter) |
void |
UidResetter.visit(LOForEach foreach) |
void |
SchemaResetter.visit(LOForEach foreach) |
void |
AllSameRalationalNodesVisitor.visit(LOForEach foreach) |
void |
AllExpressionVisitor.visit(LOForEach foreach) |
void |
UidResetter.visit(LOGenerate gen) |
void |
SchemaResetter.visit(LOGenerate gen) |
void |
AllExpressionVisitor.visit(LOGenerate gen) |
void |
UidResetter.visit(LOInnerLoad load) |
void |
SchemaResetter.visit(LOInnerLoad load) |
void |
AllExpressionVisitor.visit(LOInnerLoad load) |
void |
UidResetter.visit(LOJoin join) |
void |
SchemaResetter.visit(LOJoin join) |
void |
AllSameRalationalNodesVisitor.visit(LOJoin join) |
void |
AllExpressionVisitor.visit(LOJoin join) |
void |
UidResetter.visit(LOLimit loLimit) |
void |
SchemaResetter.visit(LOLimit loLimit) |
void |
AllSameRalationalNodesVisitor.visit(LOLimit limit) |
void |
AllExpressionVisitor.visit(LOLimit limit) |
void |
UidResetter.visit(LOLoad load) |
void |
SchemaResetter.visit(LOLoad load) |
void |
AllSameRalationalNodesVisitor.visit(LOLoad load) |
void |
AllSameRalationalNodesVisitor.visit(LONative loNative) |
void |
UidResetter.visit(LORank loRank) |
void |
SchemaResetter.visit(LORank loRank) |
void |
AllSameRalationalNodesVisitor.visit(LORank rank) |
void |
AllExpressionVisitor.visit(LORank rank) |
void |
UidResetter.visit(LOSort loSort) |
void |
SchemaResetter.visit(LOSort loSort) |
void |
AllSameRalationalNodesVisitor.visit(LOSort sort) |
void |
AllExpressionVisitor.visit(LOSort sort) |
void |
UidResetter.visit(LOSplit loSplit) |
void |
SchemaResetter.visit(LOSplit loSplit) |
void |
AllSameRalationalNodesVisitor.visit(LOSplit split) |
void |
UidResetter.visit(LOSplitOutput loSplitOutput) |
void |
SchemaResetter.visit(LOSplitOutput loSplitOutput) |
void |
AllSameRalationalNodesVisitor.visit(LOSplitOutput splitOutput) |
void |
AllExpressionVisitor.visit(LOSplitOutput splitOutput) |
void |
UidResetter.visit(LOStore store) |
void |
SchemaResetter.visit(LOStore store) |
void |
AllSameRalationalNodesVisitor.visit(LOStore store) |
void |
UidResetter.visit(LOStream loStream) |
void |
SchemaResetter.visit(LOStream loStream) |
void |
AllSameRalationalNodesVisitor.visit(LOStream stream) |
void |
UidResetter.visit(LOUnion loUnion) |
void |
SchemaResetter.visit(LOUnion loUnion) |
void |
AllSameRalationalNodesVisitor.visit(LOUnion union) |
Constructor and Description |
---|
AllExpressionVisitor(OperatorPlan plan,
PlanWalker walker) |
AllSameRalationalNodesVisitor(OperatorPlan plan,
PlanWalker walker) |
LogicalPlanPrinter(OperatorPlan plan,
PrintStream ps) |
ProjectionFinder(OperatorPlan plan) |
SchemaResetter(OperatorPlan plan) |
SchemaResetter(OperatorPlan plan,
boolean skipDuplicateUidCheck) |
UidResetter(OperatorPlan plan) |
Modifier and Type | Method and Description |
---|---|
void |
LOUnion.accept(PlanVisitor v) |
void |
LOStream.accept(PlanVisitor v) |
void |
LOStore.accept(PlanVisitor v) |
void |
LOSplitOutput.accept(PlanVisitor v) |
void |
LOSplit.accept(PlanVisitor v) |
void |
LOSort.accept(PlanVisitor v) |
void |
LORank.accept(PlanVisitor v) |
void |
LONative.accept(PlanVisitor v) |
void |
LOLoad.accept(PlanVisitor v) |
void |
LOLimit.accept(PlanVisitor v) |
void |
LOJoin.accept(PlanVisitor v) |
void |
LOInnerLoad.accept(PlanVisitor v) |
void |
LOGenerate.accept(PlanVisitor v) |
void |
LOForEach.accept(PlanVisitor v) |
void |
LOFilter.accept(PlanVisitor v) |
void |
LODistinct.accept(PlanVisitor v) |
void |
LOCube.accept(PlanVisitor v) |
void |
LOCross.accept(PlanVisitor v) |
void |
LOCogroup.accept(PlanVisitor v) |
protected boolean |
LogicalRelationalOperator.checkEquality(LogicalRelationalOperator other)
Do some basic equality checks on two relational operators.
|
static LOSort |
LOSort.createCopy(LOSort sort) |
LogicalSchema |
LOForEach.dumpNestedSchema(String alias,
String nestedAlias) |
void |
LogicalPlan.explain(PrintStream ps,
String format,
boolean verbose) |
static Pair<List<LOInnerLoad>,Boolean> |
LOForEach.findReacheableInnerLoadFromBoundaryProject(ProjectExpression project) |
LogicalSchema.LogicalFieldSchema |
LogicalSchema.getField(String alias)
Fetch a field by alias
|
LogicalSchema.LogicalFieldSchema |
LogicalSchema.getFieldSubNameMatch(String alias)
Given an alias name, find the associated LogicalFieldSchema.
|
LoadFunc |
LOLoad.getLoadFunc() |
abstract LogicalSchema |
LogicalRelationalOperator.getSchema()
Get the schema for the output of this relational operator.
|
LogicalSchema |
LOUnion.getSchema() |
LogicalSchema |
LOStream.getSchema() |
LogicalSchema |
LOStore.getSchema() |
LogicalSchema |
LOSplitOutput.getSchema() |
LogicalSchema |
LOSplit.getSchema() |
LogicalSchema |
LOSort.getSchema() |
LogicalSchema |
LORank.getSchema()
Get the schema for the output of LORank.
|
LogicalSchema |
LONative.getSchema() |
LogicalSchema |
LOLoad.getSchema()
Get the schema for this load.
|
LogicalSchema |
LOLimit.getSchema() |
LogicalSchema |
LOJoin.getSchema() |
LogicalSchema |
LOInnerLoad.getSchema() |
LogicalSchema |
LOGenerate.getSchema() |
LogicalSchema |
LOForEach.getSchema() |
LogicalSchema |
LOFilter.getSchema() |
LogicalSchema |
LODistinct.getSchema() |
LogicalSchema |
LOCube.getSchema() |
LogicalSchema |
LOCross.getSchema() |
LogicalSchema |
LOCogroup.getSchema() |
String |
LogicalPlan.getSignature()
Returns the signature of the LogicalPlan.
|
SortInfo |
LOSort.getSortInfo() |
boolean |
LOUnion.isEqual(Operator other) |
boolean |
LOStream.isEqual(Operator other) |
boolean |
LOStore.isEqual(Operator other) |
boolean |
LOSplitOutput.isEqual(Operator other) |
boolean |
LOSplit.isEqual(Operator other) |
boolean |
LOSort.isEqual(Operator other) |
boolean |
LORank.isEqual(Operator other) |
boolean |
LONative.isEqual(Operator obj) |
boolean |
LOLoad.isEqual(Operator other) |
boolean |
LOLimit.isEqual(Operator other) |
boolean |
LOJoin.isEqual(Operator other) |
boolean |
LOInnerLoad.isEqual(Operator other) |
boolean |
LOGenerate.isEqual(Operator other) |
boolean |
LOForEach.isEqual(Operator other) |
boolean |
LOFilter.isEqual(Operator other) |
boolean |
LODistinct.isEqual(Operator other) |
boolean |
LOCube.isEqual(Operator other) |
boolean |
LOCross.isEqual(Operator other) |
boolean |
LOCogroup.isEqual(Operator other) |
boolean |
LogicalPlan.isEqual(OperatorPlan other)
Equality is checked by calling equals on every leaf in the plan.
|
static boolean |
LogicalSchema.LogicalFieldSchema.isEqualUnlessUnknown(LogicalSchema.LogicalFieldSchema fs1,
LogicalSchema.LogicalFieldSchema fs2) |
static boolean |
LogicalSchema.isEqualUnlessUnknown(LogicalSchema s1,
LogicalSchema s2) |
static LogicalSchema.LogicalFieldSchema |
LogicalSchema.LogicalFieldSchema.merge(LogicalSchema.LogicalFieldSchema fs1,
LogicalSchema.LogicalFieldSchema fs2,
LogicalSchema.MergeMode mode)
Merge two LogicalFieldSchema, the behavior of merge depends on mode.
|
static LogicalSchema |
LogicalSchema.merge(LogicalSchema s1,
LogicalSchema s2,
LogicalSchema.MergeMode mode)
Merge two schemas.
|
static LogicalSchema |
LogicalSchema.mergeSchemaByAlias(LogicalSchema schema1,
LogicalSchema schema2)
Merges two schemas using their column aliases
(unlike mergeSchema(..) functions which merge using positions)
Schema will not be merged if types are incompatible,
as per DataType.mergeType(..)
For Tuples and Bags, SubSchemas have to be equal be considered compatible
|
static LogicalSchema |
LogicalSchema.mergeSchemasByAlias(List<LogicalSchema> schemas)
Merges collection of schemas using their column aliases
(unlike mergeSchema(..) functions which merge using positions)
Schema will not be merged if types are incompatible,
as per DataType.mergeType(..)
For Tuples and Bags, SubSchemas have to be equal be considered compatible
|
LogicalSchema.LogicalFieldSchema |
LogicalSchema.LogicalFieldSchema.mergeUid(LogicalSchema.LogicalFieldSchema uidOnlyFieldSchema)
Adds the uid from FieldSchema argument to this FieldSchema
If the argument is null, it stamps this FieldSchema with uid
|
LogicalSchema |
LogicalSchema.mergeUid(LogicalSchema uidOnlySchema) |
void |
LogicalPlan.optimize(PigContext pigContext) |
static void |
LogToPhyTranslationVisitor.updateWithEmptyBagCheck(PhysicalPlan fePlan,
Operator joinInput)
updates plan with check for empty bag and if bag is empty to flatten a bag
with as many null's as dictated by the schema
|
void |
LogicalPlan.validate(PigContext pigContext,
String scope,
boolean skipInputOutputValidation) |
void |
LogicalRelationalNodesVisitor.visit(LOCogroup loCogroup) |
void |
LogToPhyTranslationVisitor.visit(LOCogroup cg) |
void |
LogicalRelationalNodesVisitor.visit(LOCross loCross) |
void |
LogToPhyTranslationVisitor.visit(LOCross cross) |
void |
LogicalRelationalNodesVisitor.visit(LOCube cube) |
void |
LogicalRelationalNodesVisitor.visit(LODistinct loDistinct) |
void |
LogToPhyTranslationVisitor.visit(LODistinct loDistinct) |
void |
LogicalRelationalNodesVisitor.visit(LOFilter filter) |
void |
LogToPhyTranslationVisitor.visit(LOFilter filter) |
void |
LogicalRelationalNodesVisitor.visit(LOForEach foreach) |
void |
LogToPhyTranslationVisitor.visit(LOForEach foreach) |
void |
LogicalRelationalNodesVisitor.visit(LOGenerate gen) |
void |
LogicalRelationalNodesVisitor.visit(LOInnerLoad load) |
void |
LogToPhyTranslationVisitor.visit(LOInnerLoad load) |
void |
LogicalRelationalNodesVisitor.visit(LOJoin join) |
void |
LogToPhyTranslationVisitor.visit(LOJoin loj) |
void |
LogicalRelationalNodesVisitor.visit(LOLimit loLimit) |
void |
LogToPhyTranslationVisitor.visit(LOLimit loLimit) |
void |
LogicalRelationalNodesVisitor.visit(LOLoad load) |
void |
LogToPhyTranslationVisitor.visit(LOLoad loLoad) |
void |
LogicalRelationalNodesVisitor.visit(LONative nativeMR) |
void |
LogToPhyTranslationVisitor.visit(LONative loNative) |
void |
LogicalRelationalNodesVisitor.visit(LORank loRank) |
void |
LogToPhyTranslationVisitor.visit(LORank loRank)
Transformation from Logical to Physical Plan involves the following steps:
First, it is generated a random number which will link a POCounter within a PORank.
|
void |
LogicalRelationalNodesVisitor.visit(LOSort loSort) |
void |
LogToPhyTranslationVisitor.visit(LOSort sort) |
void |
LogicalRelationalNodesVisitor.visit(LOSplit loSplit) |
void |
LogToPhyTranslationVisitor.visit(LOSplit loSplit) |
void |
LogicalRelationalNodesVisitor.visit(LOSplitOutput loSplitOutput) |
void |
LogToPhyTranslationVisitor.visit(LOSplitOutput loSplitOutput) |
void |
LogicalRelationalNodesVisitor.visit(LOStore store) |
void |
LogToPhyTranslationVisitor.visit(LOStore loStore) |
void |
LogicalRelationalNodesVisitor.visit(LOStream loStream) |
void |
LogToPhyTranslationVisitor.visit(LOStream stream) |
void |
LogicalRelationalNodesVisitor.visit(LOUnion loUnion) |
void |
LogToPhyTranslationVisitor.visit(LOUnion loUnion) |
Constructor and Description |
---|
LogicalRelationalNodesVisitor(OperatorPlan plan,
PlanWalker walker) |
LogToPhyTranslationVisitor(OperatorPlan plan) |
LOInnerLoad(OperatorPlan plan,
LOForEach foreach,
String colAlias) |
Modifier and Type | Method and Description |
---|---|
boolean |
MapKeysPruneHelper.check() |
boolean |
ColumnPruneHelper.check() |
boolean |
TypeCastInserter.TypeCastInserterTransformer.check(OperatorPlan matched) |
boolean |
SplitFilter.SplitFilterTransformer.check(OperatorPlan matched) |
boolean |
PushUpFilter.PushUpFilterTransformer.check(OperatorPlan matched) |
boolean |
PartitionFilterOptimizer.PartitionFilterPushDownTransformer.check(OperatorPlan matched) |
boolean |
MergeForEach.MergeForEachTransformer.check(OperatorPlan matched) |
boolean |
MergeFilter.MergeFilterTransformer.check(OperatorPlan matched) |
boolean |
GroupByConstParallelSetter.GroupAllParallelSetterTransformer.check(OperatorPlan matched) |
boolean |
FilterAboveForeach.FilterAboveForEachTransformer.check(OperatorPlan matched) |
boolean |
ConstantCalculator.ConstantCalculatorTransformer.check(OperatorPlan matched) |
boolean |
ColumnMapKeyPrune.ColumnMapKeyPruneTransformer.check(OperatorPlan matched) |
boolean |
AddForEach.AddForEachTransformer.check(OperatorPlan matched) |
protected abstract LogicalSchema |
TypeCastInserter.determineSchema(LogicalRelationalOperator op) |
protected LogicalSchema |
StreamTypeCastInserter.determineSchema(LogicalRelationalOperator op) |
protected LogicalSchema |
LoadTypeCastInserter.determineSchema(LogicalRelationalOperator op)
if we are inserting casts in a load and if the loader implements
determineSchema(), insert casts only where necessary Note that in this
case, the data coming out of the loader is not a BYTEARRAY but is
whatever determineSchema() says it is.
|
protected void |
ConstantCalculator.ConstantCalculatorTransformer.ConstantCalculatorExpressionVisitor.execute(LogicalExpression op) |
protected Set<Integer> |
ColumnPruneHelper.getColumns(LogicalSchema schema,
Set<Long> uids) |
protected LogicalExpressionVisitor |
MapKeysPruneHelper.MapMarker.getVisitor(LogicalExpressionPlan expr) |
protected LogicalExpressionVisitor |
MapKeysPruneHelper.FullMapCollector.getVisitor(LogicalExpressionPlan expr) |
protected LogicalExpressionVisitor |
ConstantCalculator.ConstantCalculatorTransformer.ConstantCalculatorLogicalPlanVisitor.getVisitor(LogicalExpressionPlan expr) |
static boolean |
OptimizerUtils.planHasNonDeterministicUdf(LogicalExpressionPlan filterPlan)
Helper method to determine if the logical expression plan for a Filter contains
non-deterministic operations and should therefore be treated extra carefully
during optimization.
|
protected void |
PartitionFilterOptimizer.PartitionFilterPushDownTransformer.setupColNameMaps() |
void |
TypeCastInserter.TypeCastInserterTransformer.transform(OperatorPlan matched) |
void |
SplitFilter.SplitFilterTransformer.transform(OperatorPlan matched) |
void |
PushUpFilter.PushUpFilterTransformer.transform(OperatorPlan matched) |
void |
PartitionFilterOptimizer.PartitionFilterPushDownTransformer.transform(OperatorPlan matched) |
void |
NestedLimitOptimizer.OptimizeNestedLimitTransformer.transform(OperatorPlan matched) |
void |
MergeForEach.MergeForEachTransformer.transform(OperatorPlan matched) |
void |
MergeFilter.MergeFilterTransformer.transform(OperatorPlan matched) |
void |
LimitOptimizer.OptimizeLimitTransformer.transform(OperatorPlan matched) |
void |
GroupByConstParallelSetter.GroupAllParallelSetterTransformer.transform(OperatorPlan plan) |
void |
FilterAboveForeach.FilterAboveForEachTransformer.transform(OperatorPlan matched) |
void |
ConstantCalculator.ConstantCalculatorTransformer.transform(OperatorPlan matched) |
void |
ColumnMapKeyPrune.ColumnMapKeyPruneTransformer.transform(OperatorPlan matched) |
void |
AddForEach.AddForEachTransformer.transform(OperatorPlan matched) |
void |
LogicalRelationalNodeValidator.validate()
Validates logical operators as defined in the logical plan of a pig
script.
|
void |
MapKeysPruneHelper.FullMapCollector.visit(LOCogroup cogroup) |
void |
ColumnPruneVisitor.visit(LOCogroup cg) |
void |
ColumnPruneVisitor.visit(LOCross cross) |
void |
MapKeysPruneHelper.MapMarker.visit(LOFilter filter) |
void |
ColumnPruneVisitor.visit(LOFilter filter) |
void |
ColumnPruneVisitor.visit(LOForEach foreach) |
void |
MapKeysPruneHelper.MapMarker.visit(LOGenerate gen) |
void |
MapKeysPruneHelper.MapMarker.visit(LOJoin join) |
void |
ColumnPruneVisitor.visit(LOJoin join) |
void |
ColumnPruneVisitor.visit(LOLimit limit) |
void |
MapKeysPruneHelper.MapMarker.visit(LOLoad load) |
void |
ColumnPruneVisitor.visit(LOLoad load) |
void |
ColumnPruneVisitor.visit(LORank rank) |
void |
MapKeysPruneHelper.MapMarker.visit(LOSort sort) |
void |
ColumnPruneVisitor.visit(LOSort sort) |
void |
ColumnPruneVisitor.visit(LOSplit split) |
void |
MapKeysPruneHelper.MapMarker.visit(LOSplitOutput splitOutput) |
void |
MapKeysPruneHelper.FullMapCollector.visit(LOSplitOutput splitOutput) |
void |
ColumnPruneVisitor.visit(LOSplitOutput splitOutput) |
void |
MapKeysPruneHelper.FullMapCollector.visit(LOStore store) |
void |
ColumnPruneVisitor.visit(LOStore store) |
void |
MapKeysPruneHelper.FullMapCollector.visit(LOUnion union) |
void |
ColumnPruneVisitor.visit(LOUnion union) |
Constructor and Description |
---|
ColumnPruneVisitor(OperatorPlan plan,
Map<LOLoad,Pair<Map<Integer,Set<String>>,Set<Integer>>> requiredItems,
boolean columnPrune) |
ConstantCalculatorExpressionVisitor(OperatorPlan expPlan,
LogicalRelationalOperator currentOp,
PigContext pc) |
ConstantCalculatorLogicalPlanVisitor(OperatorPlan plan,
PigContext pc) |
FullMapCollector(OperatorPlan plan,
Set<Long> fullMapUids) |
MapMarker(OperatorPlan plan) |
Modifier and Type | Method and Description |
---|---|
void |
ImplicitSplitInsertVisitor.execute(LogicalRelationalOperator op) |
protected LogicalExpressionVisitor |
ScalarVisitor.getVisitor(LogicalExpressionPlan exprPlan) |
protected LogicalExpressionVisitor |
ProjStarInUdfExpander.getVisitor(LogicalExpressionPlan exprPlan) |
protected LogicalExpressionVisitor |
ColumnAliasConversionVisitor.getVisitor(LogicalExpressionPlan exprPlan) |
protected LogicalExpressionVisitor |
CastLineageSetter.getVisitor(LogicalExpressionPlan exprPlan) |
static boolean |
TypeCheckingExpVisitor.schemaEqualsForMatching(Schema inputSchema,
Schema udfSchema,
EvalFunc.SchemaType udfSchemaType,
boolean ignoreByteArrays)
Compare two schemas for equality for argument matching purposes.
|
protected void |
SchemaAliasVisitor.validate(LogicalRelationalOperator op)
The logic here is to check if we have duplicate alias in each schema
|
void |
TypeCheckingExpVisitor.visit(AddExpression binOp) |
void |
TypeCheckingExpVisitor.visit(AndExpression andExp) |
void |
TypeCheckingExpVisitor.visit(BinCondExpression binCond) |
void |
TypeCheckingExpVisitor.visit(CastExpression cast)
For Basic Types:
0) Casting to itself is always ok
1) Casting from number to number is always ok
2) ByteArray to anything is ok
3) number to chararray is ok
For Composite Types:
Recursively traverse the schemas till you get a basic type
|
void |
TypeCheckingExpVisitor.visit(DereferenceExpression deref) |
void |
TypeCheckingExpVisitor.visit(DivideExpression binOp) |
void |
TypeCheckingExpVisitor.visit(EqualExpression binOp) |
void |
TypeCheckingExpVisitor.visit(GreaterThanEqualExpression binOp) |
void |
TypeCheckingExpVisitor.visit(GreaterThanExpression binOp) |
void |
TypeCheckingExpVisitor.visit(LessThanEqualExpression binOp) |
void |
TypeCheckingExpVisitor.visit(LessThanExpression binOp) |
void |
TypeCheckingRelVisitor.visit(LOCogroup cg)
COGroup
All group by cols from all inputs have to be of the
same type
|
void |
SchemaAliasVisitor.visit(LOCogroup group) |
void |
ProjectStarExpander.visit(LOCogroup cg) |
void |
LineageFindRelVisitor.visit(LOCogroup group) |
void |
SchemaAliasVisitor.visit(LOCross cross) |
void |
LineageFindRelVisitor.visit(LOCross relOp) |
void |
ProjectStarExpander.visit(LOCube cu) |
void |
SchemaAliasVisitor.visit(LODistinct distinct) |
void |
LineageFindRelVisitor.visit(LODistinct relOp) |
void |
TypeCheckingRelVisitor.visit(LOFilter filter)
The schema of filter output will be the same as filter input
|
void |
SchemaAliasVisitor.visit(LOFilter filter) |
void |
ScalarVariableValidator.visit(LOFilter filter) |
void |
LineageFindRelVisitor.visit(LOFilter filter) |
void |
UDFFinder.visit(LOForEach foreach) |
void |
TypeCheckingRelVisitor.visit(LOForEach forEach) |
void |
SchemaAliasVisitor.visit(LOForEach foreach) |
void |
ProjectStarExpander.visit(LOForEach foreach) |
void |
ProjStarInUdfExpander.visit(LOForEach foreach) |
void |
LineageFindRelVisitor.visit(LOForEach fe) |
void |
ForEachUserSchemaVisitor.visit(LOForEach foreach) |
void |
DuplicateForEachColumnRewriteVisitor.visit(LOForEach foreach) |
void |
DanglingNestedNodeRemover.visit(LOForEach foreach) |
void |
UDFFinder.visit(LOGenerate generate) |
void |
TypeCheckingRelVisitor.visit(LOGenerate gen) |
void |
SchemaAliasVisitor.visit(LOGenerate gen) |
void |
ProjStarInUdfExpander.visit(LOGenerate gen) |
void |
LineageFindRelVisitor.visit(LOGenerate gen) |
void |
TypeCheckingRelVisitor.visit(LOInnerLoad innerLoad) |
void |
SchemaAliasVisitor.visit(LOInnerLoad load) |
void |
LineageFindRelVisitor.visit(LOInnerLoad innerLoad) |
void |
TypeCheckingRelVisitor.visit(LOJoin join)
LOJoin visitor
|
void |
SchemaAliasVisitor.visit(LOJoin join) |
void |
ProjectStarExpander.visit(LOJoin join) |
void |
LineageFindRelVisitor.visit(LOJoin join) |
void |
TypeCheckingRelVisitor.visit(LOLimit limit) |
void |
SchemaAliasVisitor.visit(LOLimit limit) |
void |
ScalarVariableValidator.visit(LOLimit limit) |
void |
LineageFindRelVisitor.visit(LOLimit loLimit) |
void |
LineageFindRelVisitor.visit(LOLoad load) |
void |
SchemaAliasVisitor.visit(LONative nativeMR) |
void |
TypeCheckingRelVisitor.visit(LORank rank)
The schema of rank output will be the same as input, plus a rank field.
|
void |
SchemaAliasVisitor.visit(LORank rank) |
void |
ProjectStarExpander.visit(LORank rank) |
void |
LineageFindRelVisitor.visit(LORank rank) |
void |
TypeCheckingRelVisitor.visit(LOSort sort)
The schema of sort output will be the same as sort input.
|
void |
SchemaAliasVisitor.visit(LOSort sort) |
void |
ProjectStarExpander.visit(LOSort sort) |
void |
LineageFindRelVisitor.visit(LOSort sort) |
void |
SchemaAliasVisitor.visit(LOSplit split) |
void |
LineageFindRelVisitor.visit(LOSplit relOp) |
void |
TypeCheckingRelVisitor.visit(LOSplitOutput op) |
void |
SchemaAliasVisitor.visit(LOSplitOutput splitOutput) |
void |
LineageFindRelVisitor.visit(LOSplitOutput split) |
void |
TypeCheckingRelVisitor.visit(LOStore store) |
void |
StoreAliasSetter.visit(LOStore store) |
void |
SortInfoSetter.visit(LOStore store) |
void |
SchemaAliasVisitor.visit(LOStore store) |
void |
LineageFindRelVisitor.visit(LOStore relOp) |
void |
InputOutputFileValidatorVisitor.visit(LOStore store) |
void |
SchemaAliasVisitor.visit(LOStream stream) |
void |
LineageFindRelVisitor.visit(LOStream stream) |
void |
UnionOnSchemaSetter.visit(LOUnion union) |
void |
TypeCheckingRelVisitor.visit(LOUnion u) |
void |
SchemaAliasVisitor.visit(LOUnion union) |
void |
LineageFindRelVisitor.visit(LOUnion relOp) |
void |
TypeCheckingExpVisitor.visit(MapLookupExpression map) |
void |
TypeCheckingExpVisitor.visit(ModExpression binOp) |
void |
TypeCheckingExpVisitor.visit(MultiplyExpression binOp) |
void |
TypeCheckingExpVisitor.visit(NegativeExpression negExp) |
void |
TypeCheckingExpVisitor.visit(NotEqualExpression binOp) |
void |
TypeCheckingExpVisitor.visit(NotExpression notExp) |
void |
TypeCheckingExpVisitor.visit(OrExpression orExp) |
void |
ResetProjectionAttachedRelationalOpVisitor.visit(ProjectExpression pe) |
void |
TypeCheckingExpVisitor.visit(RegexExpression rg)
RegexExpression expects CharArray as input
Itself always returns Boolean |
void |
TypeCheckingExpVisitor.visit(SubtractExpression binOp) |
void |
TypeCheckingExpVisitor.visit(UserFuncExpression func) |
Modifier and Type | Method and Description |
---|---|
abstract boolean |
Transformer.check(OperatorPlan matched)
check if the transform should be done.
|
List<OperatorPlan> |
Rule.match(OperatorPlan plan)
Search for all the sub-plans that matches the pattern
defined by this rule.
|
void |
PlanOptimizer.optimize()
Run the optimizer.
|
abstract void |
Transformer.transform(OperatorPlan matched)
Transform the tree
|
void |
PlanTransformListener.transformed(OperatorPlan fp,
OperatorPlan tp)
Notification that a plan has been transformed.
|
Modifier and Type | Class and Description |
---|---|
class |
ParserException |
Modifier and Type | Method and Description |
---|---|
static void |
QueryParserUtils.attachStorePlan(String scope,
LogicalPlan lp,
String fileName,
String func,
Operator input,
String alias,
PigContext pigContext) |
Modifier and Type | Method and Description |
---|---|
void |
POOptimizeDisabler.visit(LOCogroup cg) |
void |
LineageTrimmingVisitor.visit(LOCogroup cg) |
void |
AugmentBaseDataVisitor.visit(LOCogroup cg) |
void |
LineageTrimmingVisitor.visit(LOCross cs) |
void |
AugmentBaseDataVisitor.visit(LOCross cs) |
void |
LineageTrimmingVisitor.visit(LODistinct dt) |
void |
AugmentBaseDataVisitor.visit(LODistinct dt) |
void |
LineageTrimmingVisitor.visit(LOFilter filter) |
void |
AugmentBaseDataVisitor.visit(LOFilter filter) |
void |
LineageTrimmingVisitor.visit(LOForEach forEach) |
void |
AugmentBaseDataVisitor.visit(LOForEach forEach) |
void |
POOptimizeDisabler.visit(LOJoin join) |
void |
LineageTrimmingVisitor.visit(LOJoin join) |
void |
AugmentBaseDataVisitor.visit(LOJoin join) |
void |
LineageTrimmingVisitor.visit(LOLimit limOp) |
void |
AugmentBaseDataVisitor.visit(LOLimit lm) |
void |
LineageTrimmingVisitor.visit(LOLoad load) |
void |
AugmentBaseDataVisitor.visit(LOLoad load) |
void |
LineageTrimmingVisitor.visit(LOSort s) |
void |
AugmentBaseDataVisitor.visit(LOSort s) |
void |
LineageTrimmingVisitor.visit(LOSplit split) |
void |
AugmentBaseDataVisitor.visit(LOSplit split) |
void |
LineageTrimmingVisitor.visit(LOSplitOutput split) |
void |
LineageTrimmingVisitor.visit(LOStore store) |
void |
AugmentBaseDataVisitor.visit(LOStore store) |
void |
LineageTrimmingVisitor.visit(LOUnion u) |
void |
AugmentBaseDataVisitor.visit(LOUnion u) |
Constructor and Description |
---|
AugmentBaseDataVisitor(OperatorPlan plan,
Map<Operator,PhysicalOperator> logToPhysMap,
Map<LOLoad,DataBag> baseData,
Map<Operator,DataBag> derivedData) |
POOptimizeDisabler(OperatorPlan plan) |
Modifier and Type | Method and Description |
---|---|
static String |
DisplayExamples.printTabular(LogicalPlan lp,
Map<Operator,DataBag> exampleData,
Map<LOForEach,Map<LogicalRelationalOperator,DataBag>> forEachInnerLogToDataMap) |
void |
PreOrderDepthFirstWalker.walk(PlanVisitor visitor)
Begin traversing the graph.
|
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
DiffDate.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
UnixToISO.getArgToFuncMapping() |
List<FuncSpec> |
ISOToUnix.getArgToFuncMapping() |
List<FuncSpec> |
CustomFormatToISO.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
ISOYearsBetween.getArgToFuncMapping() |
List<FuncSpec> |
ISOSecondsBetween.getArgToFuncMapping() |
List<FuncSpec> |
ISOMonthsBetween.getArgToFuncMapping() |
List<FuncSpec> |
ISOMinutesBetween.getArgToFuncMapping() |
List<FuncSpec> |
ISOHoursBetween.getArgToFuncMapping() |
List<FuncSpec> |
ISODaysBetween.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
ISOToYear.getArgToFuncMapping() |
List<FuncSpec> |
ISOToWeek.getArgToFuncMapping() |
List<FuncSpec> |
ISOToSecond.getArgToFuncMapping() |
List<FuncSpec> |
ISOToMonth.getArgToFuncMapping() |
List<FuncSpec> |
ISOToMinute.getArgToFuncMapping() |
List<FuncSpec> |
ISOToHour.getArgToFuncMapping() |
List<FuncSpec> |
ISOToDay.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
nextAfter.getArgToFuncMapping() |
List<FuncSpec> |
getExponent.getArgToFuncMapping() |
List<FuncSpec> |
copySign.getArgToFuncMapping() |
List<FuncSpec> |
ULP.getArgToFuncMapping() |
List<FuncSpec> |
SIGNUM.getArgToFuncMapping() |
List<FuncSpec> |
SCALB.getArgToFuncMapping() |
List<FuncSpec> |
ROUND.getArgToFuncMapping()
Deprecated.
|
List<FuncSpec> |
NEXTUP.getArgToFuncMapping() |
List<FuncSpec> |
MIN.getArgToFuncMapping() |
List<FuncSpec> |
MAX.getArgToFuncMapping() |
List<FuncSpec> |
DoubleDoubleBase.getArgToFuncMapping() |
List<FuncSpec> |
DoubleBase.getArgToFuncMapping()
Deprecated.
|
List<FuncSpec> |
ABS.getArgToFuncMapping()
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
UPPER.getArgToFuncMapping()
Deprecated.
|
List<FuncSpec> |
RegexMatch.getArgToFuncMapping() |
List<FuncSpec> |
RegexExtractAll.getArgToFuncMapping()
Deprecated.
|
List<FuncSpec> |
RegexExtract.getArgToFuncMapping()
Deprecated.
|
List<FuncSpec> |
REPLACE_MULTI.getArgToFuncMapping() |
List<FuncSpec> |
HashFNV.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
Top.getArgToFuncMapping()
Deprecated.
|
List<FuncSpec> |
SearchQuery.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
SearchTermExtractor.getArgToFuncMapping() |
List<FuncSpec> |
SearchEngineExtractor.getArgToFuncMapping() |
List<FuncSpec> |
HostExtractor.getArgToFuncMapping() |
List<FuncSpec> |
DateExtractor.getArgToFuncMapping() |
Modifier and Type | Method and Description |
---|---|
List<FuncSpec> |
XPathAll.getArgToFuncMapping()
Returns argument schemas of the UDF.
|
List<FuncSpec> |
XPath.getArgToFuncMapping()
Returns argument schemas of the UDF.
|
Modifier and Type | Method and Description |
---|---|
LoadPushDown.RequiredFieldResponse |
HiveColumnarLoader.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) |
LoadPushDown.RequiredFieldResponse |
FixedWidthLoader.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) |
LoadPushDown.RequiredFieldResponse |
CSVLoader.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) |
LoadPushDown.RequiredFieldResponse |
CSVExcelStorage.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) |
LoadPushDown.RequiredFieldResponse |
AllLoader.pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) |
Modifier and Type | Method and Description |
---|---|
void |
ToolsPigServer.registerNoRun(String fileName,
Map<String,String> params,
List<String> paramFiles)
Register a script without running it.
|
List<ExecJob> |
ToolsPigServer.runPlan(LogicalPlan newPlan,
String jobName)
Given a (modified) new logical plan, run the script.
|
Modifier and Type | Method and Description |
---|---|
void |
PreprocessorContext.processShellCmd(String key,
String val)
This method generates parameter value by running specified command
|
void |
PreprocessorContext.processShellCmd(String key,
String val,
Boolean overwrite)
This method generates parameter value by running specified command
|
void |
PreprocessorContext.validate(String preprocessorCmd) |
Modifier and Type | Method and Description |
---|---|
abstract void |
JobStats.accept(PlanVisitor v) |
void |
PigStats.JobGraphPrinter.visit(JobStats op) |
Modifier and Type | Method and Description |
---|---|
void |
MRJobStats.accept(PlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
void |
SparkJobStats.accept(PlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
void |
TezVertexStats.accept(PlanVisitor v) |
void |
TezDAGStats.accept(PlanVisitor v) |
Modifier and Type | Method and Description |
---|---|
void |
BlackAndWhitelistValidator.validate() |
void |
PigCommandFilter.validate(PigCommandFilter.Command command)
Validates a Pig command as defined by
PigCommandFilter.Command . |
void |
BlackAndWhitelistFilter.validate(PigCommandFilter.Command command) |
Copyright © 2007-2017 The Apache Software Foundation