org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigOutputFormat

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat

public class PigOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>

The better half of PigInputFormat which is responsible for the Store functionality. It is the exact mirror image of PigInputFormat having RecordWriter instead of a RecordReader.


Nested Class Summary
static class PigOutputFormat.PigRecordWriter
          Wrapper class which will delegate calls to the actual RecordWriter - this should only get called in the single store case.
 
Field Summary
static String MAPRED_OUTPUT_DIR
          hadoop job output directory
static String MAPRED_TASK_PARTITION
          hadoop partition number
static String PIG_MAPRED_OUTPUT_DIR
          the temporary directory for the multi store
static String PIG_TMP_PATH
          the relative path that can be used to build a temporary place to store the output from a number of map-reduce tasks
 
Constructor Summary
PigOutputFormat()
           
 
Method Summary
 void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobcontext)
           
 org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext)
           
 org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable,Tuple> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext)
           
static void setLocation(org.apache.hadoop.mapreduce.JobContext jobContext, POStore store)
          Before delegating calls to underlying OutputFormat or OutputCommitter Pig needs to ensure the Configuration in the JobContext contains the output location and StoreFunc for the specific store - so set these up in the context for this specific store
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

MAPRED_OUTPUT_DIR

public static final String MAPRED_OUTPUT_DIR
hadoop job output directory

See Also:
Constant Field Values

MAPRED_TASK_PARTITION

public static final String MAPRED_TASK_PARTITION
hadoop partition number

See Also:
Constant Field Values

PIG_MAPRED_OUTPUT_DIR

public static final String PIG_MAPRED_OUTPUT_DIR
the temporary directory for the multi store

See Also:
Constant Field Values

PIG_TMP_PATH

public static final String PIG_TMP_PATH
the relative path that can be used to build a temporary place to store the output from a number of map-reduce tasks

See Also:
Constant Field Values
Constructor Detail

PigOutputFormat

public PigOutputFormat()
Method Detail

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable,Tuple> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext)
                                                                                                        throws IOException,
                                                                                                               InterruptedException
Specified by:
getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
Throws:
IOException
InterruptedException

setLocation

public static void setLocation(org.apache.hadoop.mapreduce.JobContext jobContext,
                               POStore store)
                        throws IOException
Before delegating calls to underlying OutputFormat or OutputCommitter Pig needs to ensure the Configuration in the JobContext contains the output location and StoreFunc for the specific store - so set these up in the context for this specific store

Parameters:
jobContext - the JobContext
store - the POStore
Throws:
IOException - on failure

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobcontext)
                      throws IOException,
                             InterruptedException
Specified by:
checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
Throws:
IOException
InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskattemptcontext)
                                                               throws IOException,
                                                                      InterruptedException
Specified by:
getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable,Tuple>
Throws:
IOException
InterruptedException


Copyright © 2007-2012 The Apache Software Foundation