org.apache.pig.data

Class LimitedSortedDataBag

java.lang.Object

org.apache.pig.data.LimitedSortedDataBag

All Implemented Interfaces:

Serializable, Comparable, Iterable<Tuple>, org.apache.hadoop.io.Writable, org.apache.hadoop.io.WritableComparable, DataBag, Spillable

public class LimitedSortedDataBag
extends Object
implements DataBag

An ordered collection of Tuples (possibly) with multiples. Data is stored in a priority queue as it comes in, and only sorted when iterator is requested. LimitedSortedDataBag is not spillable. We allow a user defined comparator, but provide a default comparator in cases where the user doesn't specify one.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
LimitedSortedDataBag(Comparator<Tuple> comp, long limit)

Method Summary

Modifier and Type	Method and Description
void	add(Tuple t) Add a tuple to the bag.
void	addAll(DataBag b) Add contents of a bag to the bag.
void	clear() Clear out the contents of the bag, both on disk and in memory.
int	compareTo(Object other)
long	getMemorySize() Not implemented.
boolean	isDistinct() Find out if the bag is distinct.
boolean	isSorted() Find out if the bag is sorted.
Iterator<Tuple>	iterator() Get an iterator to the bag.
void	markStale(boolean stale) Not implemented.
void	readFields(DataInput in) Read a bag from disk.
protected void	reportProgress() Report progress to HDFS.
long	size() Get the number of elements in the bag in memory.
long	spill() Not implemented.
String	toString() Write the bag into a string.
void	write(DataOutput out) Write a bag's contents to disk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

LimitedSortedDataBag

public LimitedSortedDataBag(Comparator<Tuple> comp,
                            long limit)

Parameters:

comp - Comparator to use to do the sorting. If null, DefaultComparator will be used.

Method Detail

size

public long size()

Get the number of elements in the bag in memory.

Specified by:

size in interface DataBag

Returns:

number of elements in the bag

isSorted

public boolean isSorted()

Find out if the bag is sorted.

Specified by:

isSorted in interface DataBag

Returns:

true if this is a sorted data bag, false otherwise.

isDistinct

public boolean isDistinct()

Find out if the bag is distinct.

Specified by:

isDistinct in interface DataBag

Returns:

true if the bag is a distinct bag, false otherwise.

iterator

public Iterator<Tuple> iterator()

Get an iterator to the bag. For default and distinct bags, no particular order is guaranteed. For sorted bags the order is guaranteed to be sorted according to the provided comparator.

Specified by:

iterator in interface Iterable<Tuple>

Specified by:

iterator in interface DataBag

Returns:

tuple iterator

add

public void add(Tuple t)

Add a tuple to the bag.

Specified by:

add in interface DataBag

Parameters:

t - tuple to add.

addAll

public void addAll(DataBag b)

Add contents of a bag to the bag.

Specified by:

addAll in interface DataBag

Parameters:

b - bag to add contents of.

clear

public void clear()

Clear out the contents of the bag, both on disk and in memory. Any attempts to read after this is called will produce undefined results.

Specified by:

clear in interface DataBag

write

public void write(DataOutput out)
           throws IOException

Write a bag's contents to disk.

Specified by:

write in interface org.apache.hadoop.io.Writable

Parameters:

out - DataOutput to write data to.

Throws:

IOException - (passes it on from underlying calls).

readFields

public void readFields(DataInput in)
                throws IOException

Read a bag from disk.

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Parameters:

in - DataInput to read data from.

Throws:

IOException - (passes it on from underlying calls).

toString

public String toString()

Write the bag into a string.

Overrides:

toString in class Object

compareTo

public int compareTo(Object other)

Specified by:

compareTo in interface Comparable

markStale

public void markStale(boolean stale)

Not implemented. This is used by FuncEvalSpec.FakeDataBag.

Specified by:

markStale in interface DataBag

Parameters:

stale - Set stale state.

spill

public long spill()

Not implemented.

Specified by:

spill in interface Spillable

Returns:

number of objects spilled.

getMemorySize

public long getMemorySize()

Not implemented.

Specified by:

getMemorySize in interface Spillable

Returns:

estimated in memory size.

reportProgress

protected void reportProgress()

Report progress to HDFS.