org.apache.pig

Interface LoadPredicatePushdown

All Known Implementing Classes:

OrcStorage

@InterfaceAudience.Private
@InterfaceStability.Evolving
public interface LoadPredicatePushdown

This interface defines how a loader can support predicate pushdown. If a given loader implements this interface, pig will pushdown predicates based on type of operations supported by the loader on given set of fields. This interface is private in Pig 0.14 and will be made public in Pig 0.15 after PIG-4093. It is to be used only by builtin LoadFunc implementations till it is made public as PIG-4093 will cause API changes to this interface and make it backward incompatible.

Since:

Pig 0.14

Method Summary

Methods

Modifier and Type	Method and Description
java.util.List<java.lang.String>	getPredicateFields(java.lang.String location, org.apache.hadoop.mapreduce.Job job) Find what fields of the data can support predicate pushdown.
java.util.List<Expression.OpType>	getSupportedExpressionTypes() Indicate operations on fields supported by the loader for predicate pushdown
void	setPushdownPredicate(Expression predicate) Push down expression to the loader

Method Detail

getPredicateFields

java.util.List<java.lang.String> getPredicateFields(java.lang.String location,
                                                  org.apache.hadoop.mapreduce.Job job)
                                                    throws java.io.IOException

Find what fields of the data can support predicate pushdown.

Parameters:

location - Location as returned by LoadFunc.relativeToAbsolutePath(String, org.apache.hadoop.fs.Path)

job - The Job object - this should be used only to obtain cluster properties through JobContext.getConfiguration() and not to set/query any runtime job information.

Returns:

list of field names that can be pushed down. Implementations should return null to indicate that there are no fields that support predicate pushdown

Throws:

java.io.IOException - if an exception occurs while retrieving predicate fields

getSupportedExpressionTypes

java.util.List<Expression.OpType> getSupportedExpressionTypes()

Indicate operations on fields supported by the loader for predicate pushdown

Returns:

List of operations supported by the predicate pushdown loader

setPushdownPredicate

void setPushdownPredicate(Expression predicate)
                          throws java.io.IOException

Push down expression to the loader

Parameters:

predicate - expression to be filtered by the loader.

Throws:

java.io.IOException