org.apache.pig.piggybank.storage

Class IndexedStorage.IndexManager

java.lang.Object

org.apache.pig.piggybank.storage.IndexedStorage.IndexManager

Enclosing class:

IndexedStorage

public static class IndexedStorage.IndexManager
extends Object

IndexManager manages the index file (both writing and reading) It keeps track of the last index read during reading.

Constructor Summary

Constructors

Constructor and Description
IndexedStorage.IndexManager(org.apache.hadoop.fs.FileStatus ifile) Constructor (called during reading)
IndexedStorage.IndexManager(int[] offsetsToIndexKeys) Constructor (called during writing)

Method Summary

Methods

Modifier and Type	Method and Description
void	Close() Close the index file
void	createIndexFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file) Open the index file for writing for given data file
Tuple	getIndexKeyTuple(Tuple indexTuple) Extracts the index key from the index tuple
long	getIndexKeyTupleCount(Tuple indexTuple) Extracts the number of records that share the current key from the index tuple.
long	getOffset(Tuple indexTuple) Extracts the offset into the data file from the index tuple.
void	openIndexFile(org.apache.hadoop.fs.FileSystem fs) Opens the index file.
Tuple	ReadIndex() Reads the next index from the index file (or null if EOF) and extracts the index fields.
void	ReadIndexFooter() Reads the index footer
void	ReadIndexHeader() Read index header
Tuple	ScanIndex(Tuple keys) Scans the index looking for a given key.
void	SeekToIndexFooter() Seeks to the index footer
void	WriteIndex() Writes the current index.
void	WriteIndexHeader() Write index header
void	WriterIndexFooter() Writes the index footer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IndexedStorage.IndexManager

public IndexedStorage.IndexManager(org.apache.hadoop.fs.FileStatus ifile)

Constructor (called during reading)

Parameters:

ifile - index file to read

IndexedStorage.IndexManager

public IndexedStorage.IndexManager(int[] offsetsToIndexKeys)

Constructor (called during writing)

Parameters:

offsetsToIndexKeys -

Method Detail

createIndexFile

public void createIndexFile(org.apache.hadoop.fs.FileSystem fs,
                   org.apache.hadoop.fs.Path file)
                     throws IOException

Open the index file for writing for given data file

Parameters:

fs -

file -

Throws:

IOException

openIndexFile

public void openIndexFile(org.apache.hadoop.fs.FileSystem fs)
                   throws IOException

Opens the index file.

Throws:

IOException

Close

public void Close()
           throws IOException

Close the index file

Throws:

IOException

WriteIndexHeader

public void WriteIndexHeader()
                      throws IOException

Write index header

Parameters:

indexOut - - Stream to write to

ih - - Index header to write

Throws:

IOException

ReadIndexHeader

public void ReadIndexHeader()
                     throws IOException

Read index header

Parameters:

indexIn - - Stream to read from

Throws:

IOException

WriterIndexFooter

public void WriterIndexFooter()
                       throws IOException

Writes the index footer

Throws:

IOException

ReadIndexFooter

public void ReadIndexFooter()
                     throws IOException

Reads the index footer

Throws:

IOException

SeekToIndexFooter

public void SeekToIndexFooter()
                       throws IOException

Seeks to the index footer

Throws:

IOException

WriteIndex

public void WriteIndex()
                throws IOException

Writes the current index.

Throws:

IOException

getIndexKeyTuple

public Tuple getIndexKeyTuple(Tuple indexTuple)
                       throws IOException

Extracts the index key from the index tuple

Throws:

IOException

getIndexKeyTupleCount

public long getIndexKeyTupleCount(Tuple indexTuple)
                           throws IOException

Extracts the number of records that share the current key from the index tuple.

Throws:

IOException

getOffset

public long getOffset(Tuple indexTuple)
               throws IOException

Extracts the offset into the data file from the index tuple.

Throws:

IOException

ReadIndex

public Tuple ReadIndex()
                throws IOException

Reads the next index from the index file (or null if EOF) and extracts the index fields.

Throws:

IOException

ScanIndex

public Tuple ScanIndex(Tuple keys)
                throws IOException

Scans the index looking for a given key.

Returns:

the matching index tuple OR the last index tuple greater than the requested key if no match is found.

Throws:

IOException