org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigRecordReader

java.lang.Object
  extended by org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader
All Implemented Interfaces:
Closeable

public class PigRecordReader
extends org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>

A wrapper around the actual RecordReader and loadfunc - this is needed for two reasons 1) To intercept the initialize call from hadoop and initialize the underlying actual RecordReader with the right Context object - this is achieved by looking up the Context corresponding to the input split this Reader is supposed to process 2) We need to give hadoop consistent key-value types - text and tuple respectively - so PigRecordReader will call underlying Loader's getNext() to get the Tuple value - the key is null text since key is not used in input to map() in Pig.


Constructor Summary
PigRecordReader(org.apache.hadoop.mapreduce.InputFormat<?,?> inputformat, PigSplit pigSplit, LoadFunc loadFunc, org.apache.hadoop.mapreduce.TaskAttemptContext context, long limit)
           
 
Method Summary
 void close()
           
 org.apache.hadoop.io.Text getCurrentKey()
           
 Tuple getCurrentValue()
           
 float getProgress()
           
 void initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
           
protected  boolean initNextRecordReader()
          Get the record reader for the next chunk in this CombineFileSplit.
 boolean nextKeyValue()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PigRecordReader

public PigRecordReader(org.apache.hadoop.mapreduce.InputFormat<?,?> inputformat,
                       PigSplit pigSplit,
                       LoadFunc loadFunc,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context,
                       long limit)
                throws IOException,
                       InterruptedException
Parameters:
context -
Throws:
IOException
InterruptedException
Method Detail

close

public void close()
           throws IOException
Specified by:
close in interface Closeable
Specified by:
close in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
Throws:
IOException

getCurrentKey

public org.apache.hadoop.io.Text getCurrentKey()
                                        throws IOException,
                                               InterruptedException
Specified by:
getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
Throws:
IOException
InterruptedException

getCurrentValue

public Tuple getCurrentValue()
                      throws IOException,
                             InterruptedException
Specified by:
getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
Throws:
IOException
InterruptedException

getProgress

public float getProgress()
                  throws IOException,
                         InterruptedException
Specified by:
getProgress in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
Throws:
IOException
InterruptedException

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws IOException,
                       InterruptedException
Specified by:
initialize in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
Throws:
IOException
InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException
Specified by:
nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>
Throws:
IOException
InterruptedException

initNextRecordReader

protected boolean initNextRecordReader()
                                throws IOException,
                                       InterruptedException
Get the record reader for the next chunk in this CombineFileSplit.

Throws:
IOException
InterruptedException


Copyright © 2007-2012 The Apache Software Foundation