| Interface | Description |
|---|---|
| SparkPigSplit |
Wrapper class for PigSplits in Spark mode
Spark only counts HDFS bytes read if we provide a FileSplit, so we have to wrap PigSplits and have the wrapper
class extend FileSplit
|
| Class | Description |
|---|---|
| JobGraphBuilder | |
| JobMetricsListener | |
| KryoSerializer |
Pulled class from Hive on Spark
|
| MapReducePartitionerWrapper |
Spark Partitioner that wraps a custom partitioner that implements
org.apache.hadoop.mapreduce.Partitioner interface.
|
| SparkEngineConf |
The object of SparkEngineConf is to solve the initialization problem of PigContext.properties.get("udf.import.list"),
UDFContext#udfConfs, UDFContext#clientSysProps in spark mode.
|
| SparkExecType | |
| SparkExecutionEngine | |
| SparkLauncher |
Main class that launches pig for Spark
|
| SparkLocalExecType |
SparkLocalExecType is the ExecType for local mode in Spark.
|
| SparkPigContext |
singleton class like PigContext
|
| SparkPigRecordReader |
Record reader for Spark mode - handles SparkPigSplit
|
| SparkPigSplit.FileSparkPigSplit | |
| SparkPigSplit.GenericSparkPigSplit | |
| SparkPOUserFuncVisitor | |
| SparkUtil | |
| UDFJarsFinder |
| Enum | Description |
|---|---|
| SparkLauncher.ResourceType |
Copyright © 2007-2017 The Apache Software Foundation