Modifier and Type | Method and Description |
---|---|
void |
OverwritableStoreFunc.cleanupOutput(POStore store,
org.apache.hadoop.mapreduce.Job job)
This method is called to cleanup the store/output location of this
StoreFunc . |
Modifier and Type | Method and Description |
---|---|
POStore |
ExecJob.getPOStore() |
Modifier and Type | Field and Description |
---|---|
protected POStore |
HJob.poStore |
Modifier and Type | Method and Description |
---|---|
POStore |
HJob.getPOStore() |
Constructor and Description |
---|
HJob(ExecJob.JOB_STATUS status,
PigContext pigContext,
POStore store,
String alias) |
HJob(ExecJob.JOB_STATUS status,
PigContext pigContext,
POStore store,
String alias,
PigStats stats) |
Modifier and Type | Method and Description |
---|---|
StoreFuncInterface |
FetchPOStoreImpl.createStoreFunc(POStore store) |
Modifier and Type | Field and Description |
---|---|
protected List<POStore> |
PigOutputFormat.mapStores |
protected List<POStore> |
PigOutputFormat.reduceStores |
protected List<POStore> |
PigGenericMapReduce.Reduce.stores |
protected List<POStore> |
PigGenericMapBase.stores |
Modifier and Type | Method and Description |
---|---|
List<POStore> |
JobControlCompiler.getStores(org.apache.hadoop.mapred.jobcontrol.Job job)
Returns all store locations of a previously compiled job
|
Modifier and Type | Method and Description |
---|---|
StoreFuncInterface |
MapReducePOStoreImpl.createStoreFunc(POStore store) |
long |
PigStatsOutputSizeReader.getOutputSize(POStore sto,
org.apache.hadoop.conf.Configuration conf)
Returns the size of output in bytes.
|
long |
FileBasedOutputSizeReader.getOutputSize(POStore sto,
org.apache.hadoop.conf.Configuration conf)
Returns the total size of output files in bytes
|
static void |
PigOutputFormat.setLocation(org.apache.hadoop.mapreduce.JobContext jobContext,
POStore store)
Before delegating calls to underlying OutputFormat or OutputCommitter
Pig needs to ensure the Configuration in the JobContext contains
the output location and StoreFunc
for the specific store - so set these up in the context for this specific
store
|
static org.apache.hadoop.mapreduce.JobContext |
PigOutputCommitter.setUpContext(org.apache.hadoop.mapreduce.JobContext context,
POStore store) |
static void |
PigOutputCommitter.storeCleanup(POStore store,
org.apache.hadoop.conf.Configuration conf) |
boolean |
PigStatsOutputSizeReader.supports(POStore sto,
org.apache.hadoop.conf.Configuration conf)
Returns whether the given PSStore is supported by this output size reader
or not.
|
boolean |
FileBasedOutputSizeReader.supports(POStore sto,
org.apache.hadoop.conf.Configuration conf)
Returns whether the given POStore is supported by this output size reader
or not.
|
void |
PhyPlanSetter.visitStore(POStore st) |
void |
MRCompiler.visitStore(POStore op) |
Constructor and Description |
---|
PigOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context,
List<POStore> mapStores,
List<POStore> reduceStores) |
PigOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context,
List<POStore> mapStores,
List<POStore> reduceStores) |
Modifier and Type | Method and Description |
---|---|
void |
UdfCacheShipFilesVisitor.visitStore(POStore st) |
void |
PlanPrinter.visitStore(POStore op) |
void |
PhyPlanVisitor.visitStore(POStore st) |
Modifier and Type | Method and Description |
---|---|
abstract StoreFuncInterface |
POStoreImpl.createStoreFunc(POStore store)
Set up the storer
|
Constructor and Description |
---|
POStore(POStore copy) |
Modifier and Type | Method and Description |
---|---|
<any> |
StoreConverter.convert(List<<any>> predecessors,
POStore op) |
Modifier and Type | Method and Description |
---|---|
void |
SparkCompiler.visitStore(POStore op) |
Modifier and Type | Method and Description |
---|---|
POStore |
TezOperator.VertexGroupInfo.getStore() |
Modifier and Type | Method and Description |
---|---|
void |
TezCompiler.visitStore(POStore op) |
Constructor and Description |
---|
VertexGroupInfo(POStore store) |
Modifier and Type | Class and Description |
---|---|
class |
POStoreTez
POStoreTez is used to write to a Tez MROutput
|
Constructor and Description |
---|
POStoreTez(POStore copy) |
Constructor and Description |
---|
PigOutputCommitterTez(org.apache.hadoop.mapreduce.TaskAttemptContext context,
List<POStore> mapStores,
List<POStore> reduceStores) |
PigOutputCommitterTez(org.apache.hadoop.mapreduce.TaskAttemptContext context,
List<POStore> mapStores,
List<POStore> reduceStores) |
Modifier and Type | Method and Description |
---|---|
static POStore |
TezCompilerUtil.getStore(String scope,
NodeIdGenerator nig) |
Modifier and Type | Method and Description |
---|---|
void |
TezUDFContextSeparator.serializeUDFContext(org.apache.hadoop.conf.Configuration conf,
TezOperator tezOp,
POStore store) |
Modifier and Type | Method and Description |
---|---|
static void |
MapRedUtil.setupStreamingDirsConfSingle(POStore st,
PigContext pigContext,
org.apache.hadoop.conf.Configuration conf)
Sets up output and log dir paths for a single-store streaming job
|
Modifier and Type | Method and Description |
---|---|
void |
PigStorage.cleanupOutput(POStore store,
org.apache.hadoop.mapreduce.Job job) |
Modifier and Type | Method and Description |
---|---|
void |
UDFContextSeparator.serializeUDFContext(org.apache.hadoop.conf.Configuration conf,
String planOpKey,
POStore store) |
void |
UDFContextSeparator.visitStore(POStore st) |
Modifier and Type | Method and Description |
---|---|
void |
IllustratorAttacher.visitStore(POStore st) |
Modifier and Type | Method and Description |
---|---|
POStore |
OutputStats.getPOStore() |
Modifier and Type | Method and Description |
---|---|
static String |
PigStatsUtil.getMultiStoreCounterName(POStore store)
Returns the counter name for the given
POStore |
static long |
JobStats.getOutputSize(POStore sto,
org.apache.hadoop.conf.Configuration conf)
Looks up the output size reader from OUTPUT_SIZE_READER_KEY and invokes
it to get the size of output.
|
void |
OutputStats.setPOStore(POStore store) |
Constructor and Description |
---|
EmptyPigStats(PigContext pigContext,
POStore poStore) |
Modifier and Type | Method and Description |
---|---|
void |
SparkPigStats.addFailJobStats(POStore poStore,
SparkOperator sparkOperator,
String jobId,
JobMetricsListener jobMetricsListener,
JavaSparkContext sparkContext,
Exception e) |
static void |
SparkStatsUtil.addFailJobStats(String jobID,
POStore poStore,
SparkOperator sparkOperator,
SparkPigStats sparkPigStats,
Exception e) |
void |
SparkPigStats.addJobStats(POStore poStore,
SparkOperator sparkOperator,
int jobId,
JobMetricsListener jobMetricsListener,
JavaSparkContext sparkContext) |
void |
SparkJobStats.addOutputInfo(POStore poStore,
boolean success,
JobMetricsListener jobMetricsListener) |
static String |
SparkStatsUtil.getCounterName(POStore store) |
static long |
SparkStatsUtil.getRecordCount(POStore store) |
static void |
SparkStatsUtil.waitForJobAddStats(int jobID,
POStore poStore,
SparkOperator sparkOperator,
JobMetricsListener jobMetricsListener,
JavaSparkContext sparkContext,
SparkPigStats sparkPigStats) |
Copyright © 2007-2017 The Apache Software Foundation