org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators

Interface AccumulativeTupleBuffer

All Known Implementing Classes:

POPackage.POPackageTupleBuffer

public interface AccumulativeTupleBuffer

This interface is used during Reduce phrase to process tuples in batch mode. It is used by POPackage when all of the UDFs can be called in accumulative mode. Tuples are not pulled all at once, instead, each time, only a specified number of tuples are pulled out of iterator and put in an buffer. Then this buffer is wrapped into a bag to be passed to the operators in reduce plan. The purpose of doing this is to reduce memory usage and avoid spilling.

Method Summary

Methods

Modifier and Type	Method and Description
void	clear() Clear internal buffer, this should be called after all data are retreived
java.util.Iterator<Tuple>	getTuples(int index) Get iterator of tuples in the buffer
boolean	hasNextBatch() Whether there are more tuples to pull out of iterator
void	nextBatch() Pull next batch of tuples from iterator and put them into this buffer

Method Detail

nextBatch

void nextBatch()
               throws java.io.IOException

Pull next batch of tuples from iterator and put them into this buffer

Throws:

java.io.IOException

hasNextBatch

boolean hasNextBatch()

Whether there are more tuples to pull out of iterator

clear

void clear()

Clear internal buffer, this should be called after all data are retreived

getTuples

java.util.Iterator<Tuple> getTuples(int index)

Get iterator of tuples in the buffer

Parameters:

index - the index of tuple