Modifier and Type | Method and Description |
---|---|
static void |
SparkUtil.createIndexerSparkNode(SparkOperator baseSparkOp,
String scope,
NodeIdGenerator nig) |
void |
UDFJarsFinder.visitSparkOp(SparkOperator sparkOp) |
void |
SparkPOUserFuncVisitor.visitSparkOp(SparkOperator sparkOperator) |
void |
JobGraphBuilder.visitSparkOp(SparkOperator sparkOp) |
Modifier and Type | Class and Description |
---|---|
class |
NativeSparkOperator
NativeSparkOperator:
|
Modifier and Type | Method and Description |
---|---|
static void |
MultiQueryOptimizerSpark.addSubPlanPropertiesToParent(SparkOperator parentOper,
SparkOperator subPlanOper) |
void |
SecondaryKeyOptimizerSpark.visitSparkOp(SparkOperator sparkOperator)
Secondary key sort optimization is enabled in group + foreach nested situation, like TestAccumlator#testAccumWithSort
After calling SecondaryKeyOptimizerUtil.applySecondaryKeySort, the POSort in the POForeach will be deleted in the spark plan.
|
void |
ParallelismSetter.visitSparkOp(SparkOperator sparkOp) |
void |
NoopFilterRemover.visitSparkOp(SparkOperator sparkOp) |
void |
MultiQueryOptimizerSpark.visitSparkOp(SparkOperator sparkOp) |
void |
JoinGroupOptimizerSpark.visitSparkOp(SparkOperator sparkOp) |
void |
CombinerOptimizer.visitSparkOp(SparkOperator sparkOp) |
void |
AccumulatorOptimizer.visitSparkOp(SparkOperator sparkOperator) |
Modifier and Type | Method and Description |
---|---|
void |
SparkOperator.copyFeatures(SparkOperator copyFrom,
List<org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkOperator.OPER_FEATURE> excludeFeatures) |
protected String[] |
DotSparkPrinter.getAttributes(SparkOperator op) |
protected String |
DotSparkPrinter.getName(SparkOperator op) |
protected Collection<DotSparkPrinter.InnerPlan> |
DotSparkPrinter.getNestedPlans(SparkOperator op) |
void |
SparkOperator.setRequestedParallelismByReference(SparkOperator oper) |
void |
XMLSparkPrinter.visitSparkOp(SparkOperator so) |
void |
SparkPrinter.visitSparkOp(SparkOperator sparkOp) |
void |
SparkPOPackageAnnotator.visitSparkOp(SparkOperator sparkOp) |
void |
SparkOpPlanVisitor.visitSparkOp(SparkOperator sparkOperator) |
Constructor and Description |
---|
SparkOpPlanVisitor(SparkOperPlan plan,
PlanWalker<SparkOperator,SparkOperPlan> walker) |
Modifier and Type | Method and Description |
---|---|
void |
SparkPigStats.addFailJobStats(POStore poStore,
SparkOperator sparkOperator,
String jobId,
JobMetricsListener jobMetricsListener,
JavaSparkContext sparkContext,
Exception e) |
static void |
SparkStatsUtil.addFailJobStats(String jobID,
POStore poStore,
SparkOperator sparkOperator,
SparkPigStats sparkPigStats,
Exception e) |
void |
SparkPigStats.addJobStats(POStore poStore,
SparkOperator sparkOperator,
int jobId,
JobMetricsListener jobMetricsListener,
JavaSparkContext sparkContext) |
String |
SparkScriptState.SparkScriptInfo.getAlias(SparkOperator sparkOp) |
String |
SparkScriptState.SparkScriptInfo.getAliasLocation(SparkOperator sparkOp) |
String |
SparkScriptState.SparkScriptInfo.getPigFeatures(SparkOperator sparkOp) |
void |
SparkJobStats.setAlias(SparkOperator sparkOperator) |
static void |
SparkStatsUtil.waitForJobAddStats(int jobID,
POStore poStore,
SparkOperator sparkOperator,
JobMetricsListener jobMetricsListener,
JavaSparkContext sparkContext,
SparkPigStats sparkPigStats) |
Copyright © 2007-2017 The Apache Software Foundation