org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Class PigSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.Writable

public class PigSplit
extends org.apache.hadoop.mapreduce.InputSplit
implements org.apache.hadoop.io.Writable, org.apache.hadoop.conf.Configurable

The main split class that maintains important information about the input split. The reason this class implements Configurable is so that Hadoop will call Configurable.setConf(Configuration) on the backend so we can use the Configuration to create the SerializationFactory to deserialize the wrapped InputSplit.

Constructor Summary

Constructors

Constructor and Description
PigSplit()
PigSplit(org.apache.hadoop.mapreduce.InputSplit[] wrappedSplits, int inputIndex, List<OperatorKey> targetOps, int splitIndex)

Method Summary

Modifier and Type	Method and Description
boolean	disableCounter()
org.apache.hadoop.conf.Configuration	getConf()
long	getLength()
long	getLength(int idx) Return the length of a wrapped split
org.apache.hadoop.mapred.SplitLocationInfo[]	getLocationInfo()
String[]	getLocations()
int	getNumPaths()
int	getSplitIndex()
List<OperatorKey>	getTargetOps()
org.apache.hadoop.mapreduce.InputSplit	getWrappedSplit() This methods returns the actual InputSplit (as returned by the InputFormat) which this class is wrapping.
org.apache.hadoop.mapreduce.InputSplit	getWrappedSplit(int idx)
boolean	isMultiInputs() Returns true if the map has multiple inputs, else false
void	readFields(DataInput is)
void	setConf(org.apache.hadoop.conf.Configuration conf) (non-Javadoc)
void	setCurrentIdx(int idx)
void	setDisableCounter(boolean disableCounter)
void	setMultiInputs(boolean b) Indicates this map has multiple input (such as the result of a join operation).
String	toString()
void	write(DataOutput os)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PigSplit

public PigSplit()

PigSplit

public PigSplit(org.apache.hadoop.mapreduce.InputSplit[] wrappedSplits,
                int inputIndex,
                List<OperatorKey> targetOps,
                int splitIndex)

Method Detail

getTargetOps

public List<OperatorKey> getTargetOps()

getWrappedSplit

public org.apache.hadoop.mapreduce.InputSplit getWrappedSplit()

This methods returns the actual InputSplit (as returned by the InputFormat) which this class is wrapping.

Returns:

the wrappedSplit

getWrappedSplit

public org.apache.hadoop.mapreduce.InputSplit getWrappedSplit(int idx)

Parameters:

idx - the index into the wrapped splits

Returns:

the specified wrapped split

getLocations

public String[] getLocations()
                      throws IOException,
                             InterruptedException

Specified by:

getLocations in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

InterruptedException

getLocationInfo

public org.apache.hadoop.mapred.SplitLocationInfo[] getLocationInfo()
                                                             throws IOException

Overrides:

getLocationInfo in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

getLength

public long getLength()
               throws IOException,
                      InterruptedException

Specified by:

getLength in class org.apache.hadoop.mapreduce.InputSplit

Throws:

IOException

InterruptedException

getLength

public long getLength(int idx)
               throws IOException,
                      InterruptedException

Return the length of a wrapped split

Parameters:

idx - the index into the wrapped splits

Returns:

number of wrapped splits

Throws:

IOException

InterruptedException

readFields

public void readFields(DataInput is)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException

write

public void write(DataOutput os)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

getSplitIndex

public int getSplitIndex()

setMultiInputs

public void setMultiInputs(boolean b)

Indicates this map has multiple input (such as the result of a join operation).

Parameters:

b - true if the map has multiple inputs

isMultiInputs

public boolean isMultiInputs()

Returns true if the map has multiple inputs, else false

Returns:

true if the map has multiple inputs, else false

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

(non-Javadoc)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

See Also:

This will be called by {@link PigInputFormat#getSplits(org.apache.hadoop.mapreduce.JobContext)} to be used in {@link #write(DataOutput)} for serializing the wrappedSplit This will be called by Hadoop in the backend to set the right Job Configuration (hadoop will invoke this method because PigSplit implements {@link Configurable} - we need this Configuration in readFields() to deserialize the wrappedSplit

getNumPaths

public int getNumPaths()

Returns:

the number of wrapped splits

toString

public String toString()

Overrides:

toString in class Object

setDisableCounter

public void setDisableCounter(boolean disableCounter)

disableCounter

public boolean disableCounter()

setCurrentIdx

public void setCurrentIdx(int idx)