org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Class PigRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

SparkPigRecordReader

public class PigRecordReader
extends org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

A wrapper around the actual RecordReader and loadfunc - this is needed for two reasons 1) To intercept the initialize call from hadoop and initialize the underlying actual RecordReader with the right Context object - this is achieved by looking up the Context corresponding to the input split this Reader is supposed to process 2) We need to give hadoop consistent key-value types - text and tuple respectively - so PigRecordReader will call underlying Loader's getNext() to get the Tuple value - the key is null text since key is not used in input to map() in Pig.

Constructor Summary

Constructors

Constructor and Description
PigRecordReader(org.apache.hadoop.mapreduce.InputFormat<?,?> inputformat, PigSplit pigSplit, LoadFuncDecorator decorator, org.apache.hadoop.mapreduce.TaskAttemptContext context, long limit)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.hadoop.io.Text	getCurrentKey()
Tuple	getCurrentValue()
float	getProgress()
void	initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
protected boolean	initNextRecordReader() Get the record reader for the next chunk in this CombineFileSplit.
boolean	nextKeyValue()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PigRecordReader

public PigRecordReader(org.apache.hadoop.mapreduce.InputFormat<?,?> inputformat,
                       PigSplit pigSplit,
                       LoadFuncDecorator decorator,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context,
                       long limit)
                throws java.io.IOException,
                       java.lang.InterruptedException

Parameters:

context -

Throws:

java.io.IOException

java.lang.InterruptedException

Method Detail

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

Throws:

java.io.IOException

getCurrentKey

public org.apache.hadoop.io.Text getCurrentKey()
                                        throws java.io.IOException,
                                               java.lang.InterruptedException

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

Throws:

java.io.IOException

java.lang.InterruptedException

getCurrentValue

public Tuple getCurrentValue()
                      throws java.io.IOException,
                             java.lang.InterruptedException

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

Throws:

java.io.IOException

java.lang.InterruptedException

getProgress

public float getProgress()
                  throws java.io.IOException,
                         java.lang.InterruptedException

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

Throws:

java.io.IOException

java.lang.InterruptedException

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws java.io.IOException,
                       java.lang.InterruptedException

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

Throws:

java.io.IOException

java.lang.InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws java.io.IOException,
                            java.lang.InterruptedException

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

Throws:

java.io.IOException

java.lang.InterruptedException

initNextRecordReader

protected boolean initNextRecordReader()
                                throws java.io.IOException,
                                       java.lang.InterruptedException

Get the record reader for the next chunk in this CombineFileSplit.

Throws:

java.io.IOException

java.lang.InterruptedException