org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners
Class SkewedPartitioner

java.lang.Object
  extended by org.apache.hadoop.mapreduce.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable

public class SkewedPartitioner
extends org.apache.hadoop.mapreduce.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>
implements org.apache.hadoop.conf.Configurable

This class is used by skewed join. For the partitioned table, the skewedpartitioner reads the key distribution data from the sampler file and returns the reducer index in a round robin fashion. For ex: if the key distribution file contains (k1, 5, 3) as an entry, reducers from 5 to 3 are returned in a round robin manner.


Constructor Summary
SkewedPartitioner()
           
 
Method Summary
 org.apache.hadoop.conf.Configuration getConf()
           
 int getPartition(PigNullableWritable wrappedKey, org.apache.hadoop.io.Writable value, int numPartitions)
           
 void setConf(org.apache.hadoop.conf.Configuration job)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SkewedPartitioner

public SkewedPartitioner()
Method Detail

getPartition

public int getPartition(PigNullableWritable wrappedKey,
                        org.apache.hadoop.io.Writable value,
                        int numPartitions)
Specified by:
getPartition in class org.apache.hadoop.mapreduce.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>

setConf

public void setConf(org.apache.hadoop.conf.Configuration job)
Specified by:
setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()
Specified by:
getConf in interface org.apache.hadoop.conf.Configurable


Copyright © 2007-2012 The Apache Software Foundation