org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Interface PigReducerEstimator

All Known Implementing Classes:
InputSizeReducerEstimator

@InterfaceAudience.Public
@InterfaceStability.Evolving
public interface PigReducerEstimator

Interface to implement when you want to use a custom approach to estimating the number of reducers for a job.

See Also:
InputSizeReducerEstimator

Field Summary
static String BYTES_PER_REDUCER_PARAM
           
static long DEFAULT_BYTES_PER_REDUCER
           
static int DEFAULT_MAX_REDUCER_COUNT_PARAM
           
static String MAX_REDUCER_COUNT_PARAM
           
 
Method Summary
 int estimateNumberOfReducers(org.apache.hadoop.mapreduce.Job job, MapReduceOper mapReduceOper)
          Estimate the number of reducers for a given job based on the collection of load funcs passed.
 

Field Detail

BYTES_PER_REDUCER_PARAM

static final String BYTES_PER_REDUCER_PARAM
See Also:
Constant Field Values

MAX_REDUCER_COUNT_PARAM

static final String MAX_REDUCER_COUNT_PARAM
See Also:
Constant Field Values

DEFAULT_BYTES_PER_REDUCER

static final long DEFAULT_BYTES_PER_REDUCER
See Also:
Constant Field Values

DEFAULT_MAX_REDUCER_COUNT_PARAM

static final int DEFAULT_MAX_REDUCER_COUNT_PARAM
See Also:
Constant Field Values
Method Detail

estimateNumberOfReducers

int estimateNumberOfReducers(org.apache.hadoop.mapreduce.Job job,
                             MapReduceOper mapReduceOper)
                             throws IOException
Estimate the number of reducers for a given job based on the collection of load funcs passed.

Parameters:
job - job instance
mapReduceOper - map reducer operator of the job
Returns:
the number of reducers to use, or -1 if the count couldn't be estimated
Throws:
IOException


Copyright © 2007-2012 The Apache Software Foundation