org.apache.pig.data
Class NonSpillableDataBag

java.lang.Object
  extended by org.apache.pig.data.NonSpillableDataBag
All Implemented Interfaces:
Serializable, Comparable, Iterable<Tuple>, org.apache.hadoop.io.Writable, org.apache.hadoop.io.WritableComparable, DataBag, Spillable

public class NonSpillableDataBag
extends Object
implements DataBag

An unordered collection of Tuples (possibly) with multiples. The tuples are stored in an ArrayList, since there is no concern for order or distinctness. The implicit assumption is that the user of this class is storing only those many tuples as will fit in memory - no spilling will be done on this bag to disk.

See Also:
Serialized Form

Constructor Summary
NonSpillableDataBag()
           
NonSpillableDataBag(int tupleCount)
          Use this constructor if you know upfront how many tuples you are going to put in this bag.
NonSpillableDataBag(List<Tuple> listOfTuples)
          This constructor creates a bag out of an existing list of tuples by taking ownership of the list and NOT copying the contents of the list.
 
Method Summary
 void add(Tuple t)
          Add a tuple to the bag.
 void addAll(DataBag b)
          Add contents of a bag to the bag.
 void clear()
          Clear out the contents of the bag, both on disk and in memory.
 int compareTo(Object other)
           
 boolean equals(Object obj)
           
 long getMemorySize()
          Requests that an object return an estimate of its in memory size.
 int hashCode()
           
 boolean isDistinct()
          Find out if the bag is distinct.
 boolean isSorted()
          Find out if the bag is sorted.
 Iterator<Tuple> iterator()
          Get an iterator to the bag.
 void markStale(boolean stale)
          This is used by FuncEvalSpec.FakeDataBag.
 void readFields(DataInput in)
          Read a bag from disk.
protected  void reportProgress()
          Report progress to HDFS.
 long size()
          Get the number of elements in the bag, both in memory and on disk.
 long spill()
          Instructs an object to spill whatever it can to disk and release references to any data structures it spills.
 String toString()
          Write the bag into a string.
 void write(DataOutput out)
          Write a bag's contents to disk.
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

NonSpillableDataBag

public NonSpillableDataBag()

NonSpillableDataBag

public NonSpillableDataBag(int tupleCount)
Use this constructor if you know upfront how many tuples you are going to put in this bag.

Parameters:
tupleCount -

NonSpillableDataBag

public NonSpillableDataBag(List<Tuple> listOfTuples)
This constructor creates a bag out of an existing list of tuples by taking ownership of the list and NOT copying the contents of the list.

Parameters:
listOfTuples - List containing the tuples
Method Detail

isSorted

public boolean isSorted()
Description copied from interface: DataBag
Find out if the bag is sorted.

Specified by:
isSorted in interface DataBag
Returns:
true if this is a sorted data bag, false otherwise.

isDistinct

public boolean isDistinct()
Description copied from interface: DataBag
Find out if the bag is distinct.

Specified by:
isDistinct in interface DataBag
Returns:
true if the bag is a distinct bag, false otherwise.

iterator

public Iterator<Tuple> iterator()
Description copied from interface: DataBag
Get an iterator to the bag. For default and distinct bags, no particular order is guaranteed. For sorted bags the order is guaranteed to be sorted according to the provided comparator.

Specified by:
iterator in interface Iterable<Tuple>
Specified by:
iterator in interface DataBag
Returns:
tuple iterator

reportProgress

protected void reportProgress()
Report progress to HDFS.


add

public void add(Tuple t)
Description copied from interface: DataBag
Add a tuple to the bag.

Specified by:
add in interface DataBag
Parameters:
t - tuple to add.

addAll

public void addAll(DataBag b)
Description copied from interface: DataBag
Add contents of a bag to the bag.

Specified by:
addAll in interface DataBag
Parameters:
b - bag to add contents of.

clear

public void clear()
Description copied from interface: DataBag
Clear out the contents of the bag, both on disk and in memory. Any attempts to read after this is called will produce undefined results.

Specified by:
clear in interface DataBag

markStale

public void markStale(boolean stale)
Description copied from interface: DataBag
This is used by FuncEvalSpec.FakeDataBag.

Specified by:
markStale in interface DataBag
Parameters:
stale - Set stale state.

size

public long size()
Description copied from interface: DataBag
Get the number of elements in the bag, both in memory and on disk.

Specified by:
size in interface DataBag
Returns:
number of elements in the bag

getMemorySize

public long getMemorySize()
Description copied from interface: Spillable
Requests that an object return an estimate of its in memory size.

Specified by:
getMemorySize in interface Spillable
Returns:
estimated in memory size.

spill

public long spill()
Description copied from interface: Spillable
Instructs an object to spill whatever it can to disk and release references to any data structures it spills.

Specified by:
spill in interface Spillable
Returns:
number of objects spilled.

write

public void write(DataOutput out)
           throws IOException
Write a bag's contents to disk.

Specified by:
write in interface org.apache.hadoop.io.Writable
Parameters:
out - DataOutput to write data to.
Throws:
IOException - (passes it on from underlying calls).

readFields

public void readFields(DataInput in)
                throws IOException
Read a bag from disk.

Specified by:
readFields in interface org.apache.hadoop.io.Writable
Parameters:
in - DataInput to read data from.
Throws:
IOException - (passes it on from underlying calls).

equals

public boolean equals(Object obj)
Overrides:
equals in class Object

hashCode

public int hashCode()
Overrides:
hashCode in class Object

compareTo

public int compareTo(Object other)
Specified by:
compareTo in interface Comparable

toString

public String toString()
Write the bag into a string.

Overrides:
toString in class Object


Copyright © ${year} The Apache Software Foundation