org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce

java.lang.Object
  extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce

public class PigMapReduce
extends Object

This class is the static Mapper & Reducer classes that are used by Pig to execute Pig Map Reduce jobs. Since there is a reduce phase, the leaf is bound to be a POLocalRearrange. So the map phase has to separate the key and tuple and collect it into the output collector. The shuffle and sort phase sorts these keys & tuples and creates key, List<Tuple> and passes the key and iterator to the list. The deserialized POPackage operator is used to package the key, List<Tuple> into pigKey, Bag<Tuple> where pigKey is of the appropriate pig type and then the result of the package is attached to the reduce plan which is executed if its not empty. Either the result of the reduce plan or the package res is collected into the output collector. The index of the tuple (that is, which bag it should be placed in by the package) is packed into the key. This is done so that hadoop sorts the keys in order of index for join.


Nested Class Summary
static class PigMapReduce.Map
           
static class PigMapReduce.MapWithComparator
          This "specialized" map class is ONLY to be used in pig queries with order by a udf.
static class PigMapReduce.MapWithPartitionIndex
          Used by Skewed Join
static class PigMapReduce.Reduce
           
static class PigMapReduce.ReduceWithComparator
          This "specialized" reduce class is ONLY to be used in pig queries with order by a udf.
 
Field Summary
static org.apache.hadoop.conf.Configuration sJobConf
          Deprecated. Use UDFContext instead in the following way to get the job's Configuration:
UdfContext.getUdfContext().getJobConf()
static ThreadLocal<org.apache.hadoop.conf.Configuration> sJobConfInternal
           
static org.apache.hadoop.mapreduce.JobContext sJobContext
           
 
Constructor Summary
PigMapReduce()
           
 
Method Summary
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

sJobContext

public static org.apache.hadoop.mapreduce.JobContext sJobContext

sJobConf

@Deprecated
public static org.apache.hadoop.conf.Configuration sJobConf
Deprecated. Use UDFContext instead in the following way to get the job's Configuration:
UdfContext.getUdfContext().getJobConf()

sJobConfInternal

public static final ThreadLocal<org.apache.hadoop.conf.Configuration> sJobConfInternal
Constructor Detail

PigMapReduce

public PigMapReduce()


Copyright © ${year} The Apache Software Foundation