public class LoadFuncMetadataWrapper extends LoadFuncWrapper implements LoadMetadata
Modifier | Constructor and Description |
---|---|
protected |
LoadFuncMetadataWrapper() |
Modifier and Type | Method and Description |
---|---|
String[] |
getPartitionKeys(String location,
org.apache.hadoop.mapreduce.Job job)
Find what columns are partition keys for this input.
|
ResourceSchema |
getSchema(String location,
org.apache.hadoop.mapreduce.Job job)
Get a schema for the data to be loaded.
|
ResourceStatistics |
getStatistics(String location,
org.apache.hadoop.mapreduce.Job job)
Get statistics about the data to be loaded.
|
protected void |
setLoadFunc(LoadMetadata loadFunc)
The wrapped LoadMetadata object must be set before method calls are made on this object.
|
void |
setPartitionFilter(Expression partitionFilter)
Set the filter for partitioning.
|
getInputFormat, getLoadCaster, getMethodName, getNext, loadFunc, prepareToRead, relativeToAbsolutePath, setLoadFunc, setLocation, setUDFContextSignature
getAbsolutePath, getCacheFiles, getPathStrings, getShipFiles, join, warn
protected void setLoadFunc(LoadMetadata loadFunc)
loadFunc
- public ResourceSchema getSchema(String location, org.apache.hadoop.mapreduce.Job job) throws IOException
LoadMetadata
getSchema
in interface LoadMetadata
location
- Location as returned by
LoadFunc.relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)
job
- The Job
object - this should be used only to obtain
cluster properties through JobContextImpl.getConfiguration()
and not to set/query
any runtime job information.IOException
- if an exception occurs while determining the schemapublic ResourceStatistics getStatistics(String location, org.apache.hadoop.mapreduce.Job job) throws IOException
LoadMetadata
LoadFunc
, then LoadFunc.setLocation(String, org.apache.hadoop.mapreduce.Job)
is guaranteed to be called before this method.getStatistics
in interface LoadMetadata
location
- Location as returned by
LoadFunc.relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)
job
- The Job
object - this should be used only to obtain
cluster properties through JobContextImpl.getConfiguration()
and not to set/query
any runtime job information.IOException
- if an exception occurs while retrieving statisticspublic String[] getPartitionKeys(String location, org.apache.hadoop.mapreduce.Job job) throws IOException
LoadMetadata
getPartitionKeys
in interface LoadMetadata
location
- Location as returned by
LoadFunc.relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)
job
- The Job
object - this should be used only to obtain
cluster properties through JobContextImpl.getConfiguration()
and not to set/query
any runtime job information.IOException
- if an exception occurs while retrieving partition keyspublic void setPartitionFilter(Expression partitionFilter) throws IOException
LoadMetadata
LoadMetadata.getPartitionKeys(String, Job)
, then this method is not
called by Pig runtime. This method is also not called by the Pig runtime
if there are no partition filter conditions.setPartitionFilter
in interface LoadMetadata
partitionFilter
- that describes filter for partitioningIOException
- if the filter is not compatible with the storage
mechanism or contains non-partition fields.Copyright © 2007-2017 The Apache Software Foundation