org.apache.pig.tools.pigstats

Class PigStats

java.lang.Object

org.apache.pig.tools.pigstats.PigStats

Direct Known Subclasses:

EmptyPigStats, SimplePigStats, TezPigScriptStats

@InterfaceAudience.Public
@InterfaceStability.Evolving
public abstract class PigStats
extends Object

PigStats encapsulates the statistics collected from a running script. It includes status of the execution, the DAG of its Hadoop jobs, as well as information about outputs and inputs of the script.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PigStats.JobGraph JobGraph is an OperatorPlan whose members are JobStats
static class	PigStats.JobGraphPrinter This class prints a JobGraph

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,OutputStats>	aliasOuputMap
protected static String	DATE_FORMAT
protected long	endTime
protected int	errorCode
protected String	errorMessage
protected Throwable	errorThrowable
protected PigStats.JobGraph	jobPlan
protected PigContext	pigContext
protected int	returnCode
protected long	startTime
protected String	userId

Constructor Summary

Constructors

Constructor and Description
PigStats()

Method Summary

Methods

Modifier and Type	Method and Description
static PigStats	get()
abstract List<String>	getAllErrorMessages()
abstract Map<String,List<PigStats>>	getAllStats()
long	getBytesWritten() Returns the total bytes written to user specified HDFS locations of this script.
long	getDuration()
int	getErrorCode() Returns the error code of PigException
String	getErrorMessage() Returns error message string
Throwable	getErrorThrowable() Returns the error code of PigException
String	getFeatures()
String	getFileName()
String	getHadoopVersion()
List<InputStats>	getInputStats()
abstract org.apache.hadoop.mapred.JobClient	getJobClient() Deprecated.
PigStats.JobGraph	getJobGraph() Returns the DAG of jobs spawned by the script
long	getNumberBytes(String location) Returns the number of bytes for the given output location, -1 for invalid location or name.
int	getNumberFailedJobs()
int	getNumberJobs() Returns the number of jobs for this script
long	getNumberRecords(String location) Returns the number of records for the given output location, -1 for invalid location or name.
int	getNumberSuccessfulJobs()
String	getOutputAlias(String location) Returns the alias associated with this output location
List<String>	getOutputLocations() Returns the list of output locations in the script
List<String>	getOutputNames() Returns the list of output names in the script
List<OutputStats>	getOutputStats()
PigContext	getPigContext()
Properties	getPigProperties() Returns the properties associated with the script
String	getPigVersion()
abstract long	getProactiveSpillCountObjects() Returns the total number of bags that spilled proactively
abstract long	getProactiveSpillCountRecords() Returns the total number of records that spilled proactively
long	getRecordWritten() Returns the total number of records in user specified output locations of this script.
int	getReturnCode() Returns code are defined in PigRunner.ReturnCode
String	getScriptId()
abstract long	getSMMSpillCount() Returns the total spill counts from SpillableMemoryManager.
abstract boolean	isEmbedded()
boolean	isSuccessful()
OutputStats	result(String alias)
void	setBackendException(String jobId, Exception e)
void	setErrorCode(int errorCode)
void	setErrorMessage(String errorMessage)
void	setErrorThrowable(Throwable t)
void	setReturnCode(int returnCode)
void	start()
static PigStats	start(PigStats stats)
void	stop()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DATE_FORMAT

protected static final String DATE_FORMAT

See Also:

Constant Field Values

startTime

protected long startTime

endTime

protected long endTime

userId

protected String userId

jobPlan

protected PigStats.JobGraph jobPlan

pigContext

protected PigContext pigContext

aliasOuputMap

protected Map<String,OutputStats> aliasOuputMap

errorCode

protected int errorCode

errorMessage

protected String errorMessage

errorThrowable

protected Throwable errorThrowable

returnCode

protected int returnCode

Constructor Detail

PigStats

public PigStats()

Method Detail

get

public static PigStats get()

start

public static PigStats start(PigStats stats)

getReturnCode

public int getReturnCode()

Returns code are defined in PigRunner.ReturnCode

getErrorMessage

public String getErrorMessage()

Returns error message string

getErrorCode

public int getErrorCode()

Returns the error code of PigException

getErrorThrowable

public Throwable getErrorThrowable()

Returns the error code of PigException

getJobClient

@Deprecated
public abstract org.apache.hadoop.mapred.JobClient getJobClient()

Deprecated.

isEmbedded

public abstract boolean isEmbedded()

isSuccessful

public boolean isSuccessful()

getAllStats

public abstract Map<String,List<PigStats>> getAllStats()

getAllErrorMessages

public abstract List<String> getAllErrorMessages()

getPigProperties

public Properties getPigProperties()

Returns the properties associated with the script

getJobGraph

public PigStats.JobGraph getJobGraph()

Returns the DAG of jobs spawned by the script

getOutputLocations

public List<String> getOutputLocations()

Returns the list of output locations in the script

getOutputNames

public List<String> getOutputNames()

Returns the list of output names in the script

getNumberBytes

public long getNumberBytes(String location)

Returns the number of bytes for the given output location, -1 for invalid location or name.

getNumberRecords

public long getNumberRecords(String location)

Returns the number of records for the given output location, -1 for invalid location or name.

getOutputAlias

public String getOutputAlias(String location)

Returns the alias associated with this output location

getSMMSpillCount

public abstract long getSMMSpillCount()

Returns the total spill counts from SpillableMemoryManager.

getProactiveSpillCountObjects

public abstract long getProactiveSpillCountObjects()

Returns the total number of bags that spilled proactively

getProactiveSpillCountRecords

public abstract long getProactiveSpillCountRecords()

Returns the total number of records that spilled proactively

getBytesWritten

public long getBytesWritten()

Returns the total bytes written to user specified HDFS locations of this script.

getRecordWritten

public long getRecordWritten()

Returns the total number of records in user specified output locations of this script.

getHadoopVersion

public String getHadoopVersion()

getPigVersion

public String getPigVersion()

getScriptId

public String getScriptId()

getFileName

public String getFileName()

getFeatures

public String getFeatures()

getDuration

public long getDuration()

getNumberJobs

public int getNumberJobs()

Returns the number of jobs for this script

getOutputStats

public List<OutputStats> getOutputStats()

result

public OutputStats result(String alias)

getInputStats

public List<InputStats> getInputStats()

setErrorMessage

public void setErrorMessage(String errorMessage)

setErrorCode

public void setErrorCode(int errorCode)

setErrorThrowable

public void setErrorThrowable(Throwable t)

setReturnCode

public void setReturnCode(int returnCode)

setBackendException

@InterfaceAudience.Private
public void setBackendException(String jobId,
                                                 Exception e)

getPigContext

@InterfaceAudience.Private
public PigContext getPigContext()

start

public void start()

stop

public void stop()

getNumberSuccessfulJobs

public int getNumberSuccessfulJobs()

getNumberFailedJobs

public int getNumberFailedJobs()