org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Class PigGenericMapReduce.ReduceWithComparator

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.Reduce

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.ReduceWithComparator

Enclosing class:

PigGenericMapReduce

public static class PigGenericMapReduce.ReduceWithComparator
extends PigMapReduce.Reduce

This "specialized" reduce class is ONLY to be used in pig queries with order by a udf. A UDF used for comparison in the order by expects to be handed tuples. Hence a specialized map class (PigMapReduce.MapWithComparator) ensures that the "key" used in the order by is wrapped into a tuple (if it isn't already a tuple). This reduce class unwraps this tuple in the case where the map had wrapped into a tuple and handes the "unwrapped" key to the POPackage for processing

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

PigMapReduce.Reduce.IllustratorContextImpl

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.Reduce

errorInReduce, initialized, log, outputCollector, pack, rp, stores

Constructor Summary

Constructors

Constructor and Description
ReduceWithComparator()

Method Summary

Modifier and Type	Method and Description
protected void	reduce(PigNullableWritable key, Iterable<NullableTuple> tupIter, org.apache.hadoop.mapreduce.Reducer.Context context) The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type.
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context) Configures the Reduce plan, the POPackage operator and the reporter thread

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

getIllustratorContext, getPack, inIllustrator

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.Reduce

cleanup, processOnePackageOutput, runPipeline, setReducePlan

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReduceWithComparator

public ReduceWithComparator()

Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException

Configures the Reduce plan, the POPackage operator and the reporter thread

Overrides:

setup in class PigGenericMapReduce.Reduce

Throws:

IOException

InterruptedException

reduce

protected void reduce(PigNullableWritable key,
                      Iterable<NullableTuple> tupIter,
                      org.apache.hadoop.mapreduce.Reducer.Context context)
               throws IOException,
                      InterruptedException

The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type. The package result is either collected as is, if the reduce plan is empty or after passing through the reduce plan.

Overrides:

reduce in class PigGenericMapReduce.Reduce

Throws:

IOException

InterruptedException