org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class MapReduceLauncher

java.lang.Object
  extended by org.apache.pig.backend.hadoop.executionengine.Launcher
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher

public class MapReduceLauncher
extends Launcher

Main class that launches pig for Map Reduce


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.Launcher
Launcher.JobControlThreadExceptionHandler
 
Field Summary
static String SUCCEEDED_FILE_NAME
           
static String SUCCESSFUL_JOB_OUTPUT_DIR_MARKER
           
 
Fields inherited from class org.apache.pig.backend.hadoop.executionengine.Launcher
failureMap, jc, jobControlException, jobControlExceptionStackTrace, totalHadoopTimeSpent
 
Constructor Summary
MapReduceLauncher()
           
 
Method Summary
 MROperPlan compile(PhysicalPlan php, PigContext pc)
           
 void explain(PhysicalPlan php, PigContext pc, PrintStream ps, String format, boolean verbose)
          Explain how a pig job will be executed on the underlying infrastructure.
 Exception getError(FileSpec spec)
          Get the exception that caused a failure on the backend for a store location (if any).
 void kill()
           
 void killJob(String jobID, org.apache.hadoop.conf.Configuration conf)
           
 PigStats launchPig(PhysicalPlan php, String grpName, PigContext pc)
          Method to launch pig for hadoop either for a cluster's job tracker or for a local job runner.
 
Methods inherited from class org.apache.pig.backend.hadoop.executionengine.Launcher
calculateProgress, computeTimeSpent, getErrorMessages, getExceptionFromString, getStackTraceElement, getTotalHadoopTimeSpent, isComplete, reset
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

SUCCEEDED_FILE_NAME

public static final String SUCCEEDED_FILE_NAME
See Also:
Constant Field Values

SUCCESSFUL_JOB_OUTPUT_DIR_MARKER

public static final String SUCCESSFUL_JOB_OUTPUT_DIR_MARKER
See Also:
Constant Field Values
Constructor Detail

MapReduceLauncher

public MapReduceLauncher()
Method Detail

kill

public void kill()
Specified by:
kill in class Launcher

killJob

public void killJob(String jobID,
                    org.apache.hadoop.conf.Configuration conf)
             throws BackendException
Specified by:
killJob in class Launcher
Throws:
BackendException

getError

public Exception getError(FileSpec spec)
Get the exception that caused a failure on the backend for a store location (if any).


launchPig

public PigStats launchPig(PhysicalPlan php,
                          String grpName,
                          PigContext pc)
                   throws PlanException,
                          VisitorException,
                          IOException,
                          ExecException,
                          JobCreationException,
                          Exception
Description copied from class: Launcher
Method to launch pig for hadoop either for a cluster's job tracker or for a local job runner. THe only difference between the two is the job client. Depending on the pig context the job client will be initialize to one of the two. Launchers for other frameworks can overide these methods. Given an input PhysicalPlan, it compiles it to get a MapReduce Plan. The MapReduce plan which has multiple MapReduce operators each one of which has to be run as a map reduce job with dependency information stored in the plan. It compiles the MROperPlan into a JobControl object. Each Map Reduce operator is converted into a Job and added to the JobControl object. Each Job also has a set of dependent Jobs that are created using the MROperPlan. The JobControl object is obtained from the JobControlCompiler Then a new thread is spawned that submits these jobs while respecting the dependency information. The parent thread monitors the submitted jobs' progress and after it is complete, stops the JobControl thread.

Specified by:
launchPig in class Launcher
Throws:
Exception
PlanException
VisitorException
IOException
ExecException
JobCreationException

explain

public void explain(PhysicalPlan php,
                    PigContext pc,
                    PrintStream ps,
                    String format,
                    boolean verbose)
             throws PlanException,
                    VisitorException,
                    IOException
Description copied from class: Launcher
Explain how a pig job will be executed on the underlying infrastructure.

Specified by:
explain in class Launcher
Parameters:
php - PhysicalPlan to explain
pc - PigContext to use for configuration
ps - PrintStream to write output on.
format - Format to write in
verbose - Amount of information to print
Throws:
VisitorException
IOException
PlanException

compile

public MROperPlan compile(PhysicalPlan php,
                          PigContext pc)
                   throws PlanException,
                          IOException,
                          VisitorException
Throws:
PlanException
IOException
VisitorException


Copyright © 2007-2012 The Apache Software Foundation