org.apache.pig.backend.hadoop.executionengine.spark.streaming

Class SparkExecutableManager

java.lang.Object

org.apache.pig.impl.streaming.ExecutableManager

org.apache.pig.backend.hadoop.streaming.HadoopExecutableManager

org.apache.pig.backend.hadoop.executionengine.spark.streaming.SparkExecutableManager

public class SparkExecutableManager
extends HadoopExecutableManager

Field Summary

Fields inherited from class org.apache.pig.impl.streaming.ExecutableManager

command, exitCode, inputBytes, inputRecords, outerrThreadsError, outputBytes, outputRecords, stdin

Constructor Summary

Constructors

Constructor and Description
SparkExecutableManager()

Method Summary

Modifier and Type	Method and Description
protected boolean	writeErrorToHDFS(int limit, String taskId) Should the stderr data of this task be persisted on HDFS?

Methods inherited from class org.apache.pig.backend.hadoop.streaming.HadoopExecutableManager

close, configure, exec, processError

Methods inherited from class org.apache.pig.impl.streaming.ExecutableManager

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkExecutableManager

public SparkExecutableManager()

Method Detail

writeErrorToHDFS

protected boolean writeErrorToHDFS(int limit,
                                   String taskId)

Description copied from class: HadoopExecutableManager

Should the stderr data of this task be persisted on HDFS?

Overrides:

writeErrorToHDFS in class HadoopExecutableManager

Parameters:

limit - maximum number of tasks whose stderr log-files are persisted

taskId - id of the task

Returns:

true if stderr data of task should be persisted on HDFS, false otherwise