org.apache.pig.backend.hadoop.executionengine.mapReduceLayer

Class PigMapReduceCounter.PigReduceCounter

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.Reduce

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduceCounter.PigReduceCounter

Enclosing class:

PigMapReduceCounter

public static class PigMapReduceCounter.PigReduceCounter
extends org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

This class is the used for RANK BY operations, independently if it is dense or not.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce.IllustratorContextImpl

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.mapreduce.Reducer.Context	context
static PhysicalOperator	leaf
static List<PhysicalOperator>	leaves
static String	taskID

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.Reduce

errorInReduce, initialized, outputCollector, pack, rp, stores

Constructor Summary

Constructors

Constructor and Description
PigMapReduceCounter.PigReduceCounter()

Method Summary

Methods

Modifier and Type	Method and Description
static void	incrementCounter(Long increment) On this case, global counters are accessed during reduce phase (immediately after a sorting phase) and the increment for global counters are dependent if it is dense rank or not.
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context) Here is set up the task id, in order to be attached to each tuple

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce

getIllustratorContext, getPack, inIllustrator

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce.Reduce

cleanup, processOnePackageOutput, reduce, runPipeline, setReducePlan

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

taskID

public static String taskID

context

public static org.apache.hadoop.mapreduce.Reducer.Context context

leaves

public static List<PhysicalOperator> leaves

leaf

public static PhysicalOperator leaf

Constructor Detail

PigMapReduceCounter.PigReduceCounter

public PigMapReduceCounter.PigReduceCounter()

Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException

Here is set up the task id, in order to be attached to each tuple

Overrides:

setup in class PigGenericMapReduce.Reduce

Throws:

IOException

InterruptedException

incrementCounter

public static void incrementCounter(Long increment)

On this case, global counters are accessed during reduce phase (immediately after a sorting phase) and the increment for global counters are dependent if it is dense rank or not. If it is a dense rank, increment is done by 1. if it is not increment depends on the size of the size of bag in the tuple.

Parameters:

increment - is the value to add to the corresponding global counter.