org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Class PigCombiner.Combine

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner.Combine

Enclosing class:

PigCombiner

public static class PigCombiner.Combine
extends org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Constructor Summary

Constructors

Constructor and Description
Combine()

Method Summary

Modifier and Type	Method and Description
protected void	cleanup(org.apache.hadoop.mapreduce.Reducer.Context context) Will be called once all the intermediate keys and values are processed.
byte	getKeyType()
boolean	processOnePackageOutput(org.apache.hadoop.mapreduce.Reducer.Context oc)
protected void	reduce(PigNullableWritable key, java.lang.Iterable<NullableTuple> tupIter, org.apache.hadoop.mapreduce.Reducer.Context context) The reduce function which packages the key and List <Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type.
void	setKeyType(byte keyType)
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context) Configures the Reduce plan, the POPackage operator and the reporter thread
static void	staticDataCleanup()

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Combine

public Combine()

Method Detail

staticDataCleanup

public static void staticDataCleanup()

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws java.io.IOException,
                     java.lang.InterruptedException

Configures the Reduce plan, the POPackage operator and the reporter thread

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

Throws:

java.io.IOException

java.lang.InterruptedException

reduce

protected void reduce(PigNullableWritable key,
                      java.lang.Iterable<NullableTuple> tupIter,
                      org.apache.hadoop.mapreduce.Reducer.Context context)
               throws java.io.IOException,
                      java.lang.InterruptedException

The reduce function which packages the key and List <Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type. The package result is either collected as is, if the reduce plan is empty or after passing through the reduce plan.

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

Throws:

java.io.IOException

java.lang.InterruptedException

processOnePackageOutput

public boolean processOnePackageOutput(org.apache.hadoop.mapreduce.Reducer.Context oc)
                                throws java.io.IOException,
                                       java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.InterruptedException

cleanup

protected void cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
                throws java.io.IOException,
                       java.lang.InterruptedException

Will be called once all the intermediate keys and values are processed. cleanup references to the PhysicalPlan

Overrides:

cleanup in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

Throws:

java.io.IOException

java.lang.InterruptedException

getKeyType

public byte getKeyType()

Returns:

the keyType

setKeyType

public void setKeyType(byte keyType)

Parameters:

keyType - the keyType to set