public class MapKeysPruneHelper extends Object
Map> where Integer is the column number
of the field and Set is the set of Keys in this field ( field is a map field only ).
It does this for only the top level schema in load.
Algorithm:
Traverse the Plan in ReverseDependency order ( ie. Sink to Source )
For LogicalRelationalOperators having MapLookupExpression in their
expressionPlan collect uid and keys related to it. This is
retained in the visitor
For ForEach having nested LogicalPlan use the same visitor hence
there is no distinction required
At Sources find all the uids provided by this source and annotate this
LogicalRelationalOperator ( load ) with Map>
containing only the column numbers that this LogicalRelationalOperator generates
NOTE: This is a simple Map Pruner. If a map key is mentioned in the script
then this pruner assumes you need the key. This pruner is not as optimized
as column pruner ( which removes a column if it is mentioned but never used )
Modifier and Type | Class and Description |
---|---|
static class |
MapKeysPruneHelper.FullMapCollector |
static class |
MapKeysPruneHelper.MapMarker
This class collects all the information required to create
the list of keys required for a map
|
Modifier and Type | Field and Description |
---|---|
static String |
REQUIRED_MAPKEYS |
Constructor and Description |
---|
MapKeysPruneHelper(OperatorPlan currentPlan) |
Modifier and Type | Method and Description |
---|---|
boolean |
check() |
OperatorPlan |
reportChanges() |
public static final String REQUIRED_MAPKEYS
public MapKeysPruneHelper(OperatorPlan currentPlan)
public boolean check() throws FrontendException
FrontendException
public OperatorPlan reportChanges()
Copyright © 2007-2017 The Apache Software Foundation