org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators
Interface AccumulativeTupleBuffer


public interface AccumulativeTupleBuffer

This interface is used during Reduce phrase to process tuples in batch mode. It is used by POPackage when all of the UDFs can be called in accumulative mode. Tuples are not pulled all at once, instead, each time, only a specified number of tuples are pulled out of iterator and put in an buffer. Then this buffer is wrapped into a bag to be passed to the operators in reduce plan. The purpose of doing this is to reduce memory usage and avoid spilling.


Method Summary
 void clear()
          Clear internal buffer, this should be called after all data are retreived
 Iterator<Tuple> getTuples(int index)
          Get iterator of tuples in the buffer
 boolean hasNextBatch()
          Whether there are more tuples to pull out of iterator
 void nextBatch()
          Pull next batch of tuples from iterator and put them into this buffer
 

Method Detail

nextBatch

void nextBatch()
               throws IOException
Pull next batch of tuples from iterator and put them into this buffer

Throws:
IOException

hasNextBatch

boolean hasNextBatch()
Whether there are more tuples to pull out of iterator


clear

void clear()
Clear internal buffer, this should be called after all data are retreived


getTuples

Iterator<Tuple> getTuples(int index)
Get iterator of tuples in the buffer

Parameters:
index - the index of tuple


Copyright © 2007-2012 The Apache Software Foundation