org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce.ReduceWithComparator

java.lang.Object
  extended by org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
          extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.ReduceWithComparator
Enclosing class:
PigMapReduce

public static class PigMapReduce.ReduceWithComparator
extends PigMapReduce.Reduce

This "specialized" reduce class is ONLY to be used in pig queries with order by a udf. A UDF used for comparison in the order by expects to be handed tuples. Hence a specialized map class (PigMapReduce.MapWithComparator) ensures that the "key" used in the order by is wrapped into a tuple (if it isn't already a tuple). This reduce class unwraps this tuple in the case where the map had wrapped into a tuple and handes the "unwrapped" key to the POPackage for processing


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer
org.apache.hadoop.mapreduce.Reducer.Context
 
Field Summary
 
Fields inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
errorInReduce, initialized, log, outputCollector, pack, rp, stores
 
Constructor Summary
PigMapReduce.ReduceWithComparator()
           
 
Method Summary
protected  void reduce(PigNullableWritable key, Iterable<NullableTuple> tupIter, org.apache.hadoop.mapreduce.Reducer.Context context)
          The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type.
protected  void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
          Configures the Reduce plan, the POPackage operator and the reporter thread
 
Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
cleanup, processOnePackageOutput, runPipeline
 
Methods inherited from class org.apache.hadoop.mapreduce.Reducer
run
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PigMapReduce.ReduceWithComparator

public PigMapReduce.ReduceWithComparator()
Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException
Configures the Reduce plan, the POPackage operator and the reporter thread

Overrides:
setup in class PigMapReduce.Reduce
Throws:
IOException
InterruptedException

reduce

protected void reduce(PigNullableWritable key,
                      Iterable<NullableTuple> tupIter,
                      org.apache.hadoop.mapreduce.Reducer.Context context)
               throws IOException,
                      InterruptedException
The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type. The package result is either collected as is, if the reduce plan is empty or after passing through the reduce plan.

Overrides:
reduce in class PigMapReduce.Reduce
Throws:
IOException
InterruptedException


Copyright © ${year} The Apache Software Foundation