org.apache.pig

Interface LoadPushDown

All Known Implementing Classes:

AllLoader, AvroStorage, CSVExcelStorage, CSVLoader, FixedWidthLoader, HBaseStorage, HiveColumnarLoader, HiveColumnarStorage, IndexedStorage, nl.basjes.pig.input.apachehttpdlog.Loader, LogFormatLoader, OrcStorage, ParquetLoader, PigStorage, PigStorageSchema, TrevniStorage

@InterfaceAudience.Public
@InterfaceStability.Evolving
public interface LoadPushDown

This interface defines how to communicate to Pig what functionality can be pushed into the loader. If a given loader does not implement this interface it will be assumed that it is unable to accept any functionality for push down.

Since:

Pig 0.7

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	LoadPushDown.OperatorSet Set of possible operations that Pig can push down to a loader.
static class	LoadPushDown.RequiredField Describes a field that is required to execute a scripts.
static class	LoadPushDown.RequiredFieldList List of fields that Pig knows to be required to executed a script.
static class	LoadPushDown.RequiredFieldResponse Indicates whether the loader will return the requested fields or all fields.

Method Summary

Methods

Modifier and Type	Method and Description
List<LoadPushDown.OperatorSet>	getFeatures() Determine the operators that can be pushed to the loader.
LoadPushDown.RequiredFieldResponse	pushProjection(LoadPushDown.RequiredFieldList requiredFieldList) Indicate to the loader fields that will be needed.

Method Detail

getFeatures

List<LoadPushDown.OperatorSet> getFeatures()

Determine the operators that can be pushed to the loader. Note that by indicating a loader can accept a certain operator (such as selection) the loader is not promising that it can handle all selections. When it is passed the actual operators to push down it will still have a chance to reject them.

Returns:

list of all features that the loader can support

pushProjection

LoadPushDown.RequiredFieldResponse pushProjection(LoadPushDown.RequiredFieldList requiredFieldList)
                                                  throws FrontendException

Indicate to the loader fields that will be needed. This can be useful for loaders that access data that is stored in a columnar format where indicating columns to be accessed a head of time will save scans. This method will not be invoked by the Pig runtime if all fields are required. So implementations should assume that if this method is not invoked, then all fields from the input are required. If the loader function cannot make use of this information, it is free to ignore it by returning an appropriate Response

Parameters:

requiredFieldList - RequiredFieldList indicating which columns will be needed. This structure is read only. User cannot make change to it inside pushProjection.

Returns:

Indicates which fields will be returned

Throws:

FrontendException