org.apache.pig.backend.hadoop.executionengine.spark

Class SparkPigSplit.GenericSparkPigSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.pig.backend.hadoop.executionengine.spark.SparkPigSplit.GenericSparkPigSplit

All Implemented Interfaces:

Serializable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.Writable, SparkPigSplit

Enclosing interface:

SparkPigSplit

public static class SparkPigSplit.GenericSparkPigSplit
extends org.apache.hadoop.mapreduce.InputSplit
implements SparkPigSplit

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.pig.backend.hadoop.executionengine.spark.SparkPigSplit

SparkPigSplit.FileSparkPigSplit, SparkPigSplit.GenericSparkPigSplit

Constructor Summary

Constructors

Constructor and Description
GenericSparkPigSplit() Spark executor's deserializer calls this, and we have to instantiate a default wrapped object
GenericSparkPigSplit(PigSplit pigSplit)

Method Summary

Modifier and Type	Method and Description
boolean	disableCounter()
org.apache.hadoop.conf.Configuration	getConf()
long	getLength()
long	getLength(int idx)
org.apache.hadoop.mapred.SplitLocationInfo[]	getLocationInfo()
String[]	getLocations()
int	getNumPaths()
int	getSplitIndex()
PigSplit	getWrappedPigSplit()
org.apache.hadoop.mapreduce.InputSplit	getWrappedSplit()
org.apache.hadoop.mapreduce.InputSplit	getWrappedSplit(int idx)
boolean	isMultiInputs()
void	readFields(DataInput is)
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setCurrentIdx(int idx)
void	setDisableCounter(boolean disableCounter)
void	setMultiInputs(boolean b)
String	toString()
void	write(DataOutput os)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

GenericSparkPigSplit

public GenericSparkPigSplit()

Spark executor's deserializer calls this, and we have to instantiate a default wrapped object

GenericSparkPigSplit

public GenericSparkPigSplit(PigSplit pigSplit)

Method Detail

getLocationInfo

public org.apache.hadoop.mapred.SplitLocationInfo[] getLocationInfo()
                                                             throws IOException

Specified by:

getLocationInfo in interface SparkPigSplit

Overrides:

getLocationInfo in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

getLength

public long getLength()
               throws IOException,
                      InterruptedException

Specified by:

getLength in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

InterruptedException

getLocations

public String[] getLocations()
                      throws IOException,
                             InterruptedException

Specified by:

getLocations in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

InterruptedException

getWrappedSplit

public org.apache.hadoop.mapreduce.InputSplit getWrappedSplit()

Specified by:

getWrappedSplit in interface SparkPigSplit

getWrappedSplit

public org.apache.hadoop.mapreduce.InputSplit getWrappedSplit(int idx)

Specified by:

getWrappedSplit in interface SparkPigSplit

getLength

public long getLength(int idx)
               throws IOException,
                      InterruptedException

Specified by:

getLength in interface SparkPigSplit

Throws:

IOException

InterruptedException

readFields

public void readFields(DataInput is)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException

write

public void write(DataOutput os)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

getSplitIndex

public int getSplitIndex()

Specified by:

getSplitIndex in interface SparkPigSplit

setMultiInputs

public void setMultiInputs(boolean b)

Specified by:

setMultiInputs in interface SparkPigSplit

isMultiInputs

public boolean isMultiInputs()

Specified by:

isMultiInputs in interface SparkPigSplit

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getNumPaths

public int getNumPaths()

Specified by:

getNumPaths in interface SparkPigSplit

setDisableCounter

public void setDisableCounter(boolean disableCounter)

Specified by:

setDisableCounter in interface SparkPigSplit

disableCounter

public boolean disableCounter()

Specified by:

disableCounter in interface SparkPigSplit

setCurrentIdx

public void setCurrentIdx(int idx)

Specified by:

setCurrentIdx in interface SparkPigSplit

getWrappedPigSplit

public PigSplit getWrappedPigSplit()

Specified by:

getWrappedPigSplit in interface SparkPigSplit