org.apache.pig.backend.hadoop.executionengine.spark

Class SparkExecutionEngine

java.lang.Object

org.apache.pig.backend.hadoop.executionengine.HExecutionEngine

org.apache.pig.backend.hadoop.executionengine.spark.SparkExecutionEngine

All Implemented Interfaces:

ExecutionEngine

public class SparkExecutionEngine
extends HExecutionEngine

Field Summary

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.HExecutionEngine

CORE_DEFAULT_SITE, CORE_SITE, ds, HADOOP_SITE, launcher, LOCAL, logicalToPhysicalKeys, MAPRED_DEFAULT_SITE, newLogToPhyMap, pigContext, YARN_DEFAULT_SITE, YARN_SITE

Constructor Summary

Constructors

Constructor and Description
SparkExecutionEngine(PigContext pigContext)

Method Summary

Modifier and Type	Method and Description
ExecutableManager	getExecutableManager() Returns the ExecutableManager to be used in Pig Streaming.
PigStats	instantiatePigStats() Creates a PigStats object which will be accessible as a ThreadLocal variable inside the PigStats class.
ScriptState	instantiateScriptState() Creates a ScriptState object which will be accessible as a ThreadLocal variable inside the ScriptState class.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.HExecutionEngine

compile, destroy, explain, getConfiguration, getDataStorage, getExecConf, getForEachInnerLogToPhyMap, getJobConf, getLocalConf, getLogToPhyMap, getS3Conf, init, kill, killJob, launchPig, setConfiguration, setProperty

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkExecutionEngine

public SparkExecutionEngine(PigContext pigContext)

Method Detail

instantiateScriptState

public ScriptState instantiateScriptState()

Description copied from interface: ExecutionEngine

Creates a ScriptState object which will be accessible as a ThreadLocal variable inside the ScriptState class. This method is called when first initializing the ScriptState as to delegate to the ExecutionEngine the version of ScriptState to use to manage the execution at hand.

Returns:

ScriptState object to manage execution of the script

getExecutableManager

public ExecutableManager getExecutableManager()

Description copied from interface: ExecutionEngine

Returns the ExecutableManager to be used in Pig Streaming.

Specified by:

getExecutableManager in interface ExecutionEngine

Overrides:

getExecutableManager in class HExecutionEngine

Returns:

ExecutableManager to be used in Pig Streaming.

instantiatePigStats

public PigStats instantiatePigStats()

Description copied from interface: ExecutionEngine

Creates a PigStats object which will be accessible as a ThreadLocal variable inside the PigStats class. This method is called when first initializing the PigStats.

Returns:

PigStats object.