org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Class POLoad

java.lang.Object

org.apache.pig.impl.plan.Operator<PhyPlanVisitor>

org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.lang.Comparable<Operator>, Illustrable

Direct Known Subclasses:

POSimpleTezLoad

public class POLoad
extends PhysicalOperator

The load operator which is used in two ways: 1) As a local operator it can be used to load files 2) In the Map Reduce setting, it is used to create jobs from MapReduce operators which keep the loads and stores in the Map and Reduce Plans till the job is created

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

PhysicalOperator.OriginalLocation

Field Summary

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

alias, illustrator, input, inputAttached, inputs, lineageTracer, outputs, parentPlan, pigLogger, reporter, requestedParallelism, res, RESULT_EMPTY, RESULT_EOP, resultType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

Constructors

Constructor and Description
POLoad(OperatorKey k)
POLoad(OperatorKey k, FileSpec lFile)
POLoad(OperatorKey k, int rp, FileSpec lFile)
POLoad(OperatorKey k, LoadFunc lf)

Method Summary

Methods

Modifier and Type	Method and Description
java.util.List<java.lang.String>	getCacheFiles()
FileSpec	getLFile()
long	getLimit()
LoadFunc	getLoadFunc()
Result	getNextTuple() The main method used by this operator's successor to read tuples from the specified file using the specified load function.
PigContext	getPc()
java.util.List<java.lang.String>	getShipFiles()
java.lang.String	getSignature()
Tuple	illustratorMarkup(java.lang.Object in, java.lang.Object out, int eqClassIndex) input tuple mark up to be illustrate-able
boolean	isTmpLoad()
java.lang.String	name()
void	setCacheFiles(java.util.List<java.lang.String> cf)
void	setIsTmpLoad(boolean tmp)
void	setLFile(FileSpec file)
void	setLimit(long limit)
void	setPc(PigContext pc)
void	setShipFiles(java.util.List<java.lang.String> sf)
void	setSignature(java.lang.String signature)
void	setUp() Set up the loader by 1) Instantiating the load func 2) Opening an input stream to the specified file and 3) Binding to the input stream at the specified offset.
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
boolean	supportsMultipleOutputs() Indicates whether this operator supports multiple outputs.
void	tearDown() At the end of processing, the inputstream is closed using this method
void	visit(PhyPlanVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator

addOriginalLocation, addOriginalLocation, attachInput, clone, cloneHelper, copyAliasFrom, detachInput, getAlias, getAliasString, getIllustrator, getInputs, getLogger, getNext, getNextBigDecimal, getNextBigInteger, getNextBoolean, getNextDataBag, getNextDataByteArray, getNextDateTime, getNextDouble, getNextFloat, getNextInteger, getNextLong, getNextMap, getNextString, getOriginalLocations, getPigLogger, getReporter, getRequestedParallelism, getResultType, isAccumStarted, isAccumulative, isBlocking, isInputAttached, processInput, reset, setAccumEnd, setAccumStart, setAccumulative, setIllustrator, setInputs, setParentPlan, setPigLogger, setReporter, setRequestedParallelism, setResultType, staticDataCleanup

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, getOperatorKey, getProjectionMap, hashCode, regenerateProjectionMap, rewire, toString, unsetProjectionMap

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

POLoad

public POLoad(OperatorKey k)

POLoad

public POLoad(OperatorKey k,
      FileSpec lFile)

POLoad

public POLoad(OperatorKey k,
      int rp,
      FileSpec lFile)

POLoad

public POLoad(OperatorKey k,
      LoadFunc lf)

Method Detail

setUp

public void setUp()
           throws java.io.IOException

Set up the loader by 1) Instantiating the load func 2) Opening an input stream to the specified file and 3) Binding to the input stream at the specified offset.

Throws:

java.io.IOException

tearDown

public void tearDown()
              throws java.io.IOException

At the end of processing, the inputstream is closed using this method

Throws:

java.io.IOException

getNextTuple

public Result getNextTuple()
                    throws ExecException

The main method used by this operator's successor to read tuples from the specified file using the specified load function.

Overrides:

getNextTuple in class PhysicalOperator

Returns:

Whatever the loader returns A null from the loader is indicative of EOP and hence the tearDown of connection

Throws:

ExecException

name

public java.lang.String name()

Specified by:

name in class Operator<PhyPlanVisitor>

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

supportsMultipleOutputs

public boolean supportsMultipleOutputs()

Description copied from class: Operator

Indicates whether this operator supports multiple outputs.

Specified by:

supportsMultipleOutputs in class Operator<PhyPlanVisitor>

Returns:

true if it does, otherwise false.

visit

public void visit(PhyPlanVisitor v)
           throws VisitorException

Description copied from class: Operator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class PhysicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

getLFile

public FileSpec getLFile()

setLFile

public void setLFile(FileSpec file)

setIsTmpLoad

public void setIsTmpLoad(boolean tmp)

isTmpLoad

public boolean isTmpLoad()

getPc

public PigContext getPc()

setPc

public void setPc(PigContext pc)

getSignature

public java.lang.String getSignature()

setSignature

public void setSignature(java.lang.String signature)

getLoadFunc

public LoadFunc getLoadFunc()

illustratorMarkup

public Tuple illustratorMarkup(java.lang.Object in,
                      java.lang.Object out,
                      int eqClassIndex)

Description copied from interface: Illustrable

input tuple mark up to be illustrate-able

Parameters:

in - input tuple

out - output tuple before wrapped in ExampleTuple

eqClassIndex - index into equivalence classes in illustrator

Returns:

tuple

getLimit

public long getLimit()

setLimit

public void setLimit(long limit)

getCacheFiles

public java.util.List<java.lang.String> getCacheFiles()

setCacheFiles

public void setCacheFiles(java.util.List<java.lang.String> cf)

getShipFiles

public java.util.List<java.lang.String> getShipFiles()

setShipFiles

public void setShipFiles(java.util.List<java.lang.String> sf)