org.apache.pig.builtin

Class TrevniStorage

java.lang.Object

org.apache.pig.LoadFunc

org.apache.pig.builtin.AvroStorage

org.apache.pig.builtin.TrevniStorage

All Implemented Interfaces:

LoadMetadata, LoadPushDown, StoreFuncInterface, StoreResources

public class TrevniStorage
extends AvroStorage
implements LoadPushDown

Pig Store/Load Function for Trevni.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.pig.LoadPushDown

LoadPushDown.OperatorSet, LoadPushDown.RequiredField, LoadPushDown.RequiredFieldList, LoadPushDown.RequiredFieldResponse

Field Summary

Fields inherited from class org.apache.pig.builtin.AvroStorage

allowRecursive, doubleColonsToDoubleUnderscores, INPUT_AVRO_SCHEMA, log, OUTPUT_AVRO_SCHEMA, requiredFieldList, schema, udfContextSignature

Constructor Summary

Constructors

Constructor and Description
TrevniStorage() Create new instance of TrevniStorage with no arguments (useful for loading files without specifying parameters).
TrevniStorage(String sn, String opts) Create new instance of TrevniStorage.

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.avro.Schema	getAvroSchema(org.apache.hadoop.fs.Path[] p, org.apache.hadoop.mapreduce.Job job) Reads the avro schemas at the specified location.
org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.avro.generic.GenericData.Record>	getInputFormat() This will be called during planning on the front end.
org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Object>	getOutputFormat() Return the OutputFormat associated with StoreFuncInterface.

Methods inherited from class org.apache.pig.builtin.AvroStorage

checkSchema, cleanupOnFailure, cleanupOnSuccess, getAvroSchema, getFeatures, getInputAvroSchema, getNext, getOutputAvroSchema, getPartitionKeys, getProperties, getProperties, getSchema, getShipFiles, getStatistics, prepareToRead, prepareToWrite, pushProjection, putNext, relToAbsPathForStoreLocation, setInputAvroSchema, setLocation, setOutputAvroSchema, setPartitionFilter, setStoreFuncUDFContextSignature, setStoreLocation, setUDFContextSignature

Methods inherited from class org.apache.pig.LoadFunc

getAbsolutePath, getCacheFiles, getLoadCaster, getPathStrings, join, relativeToAbsolutePath, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pig.LoadPushDown

getFeatures, pushProjection

Methods inherited from interface org.apache.pig.StoreResources

getCacheFiles

Constructor Detail

TrevniStorage

public TrevniStorage()

Create new instance of TrevniStorage with no arguments (useful for loading files without specifying parameters).

TrevniStorage

public TrevniStorage(String sn,
             String opts)

Create new instance of TrevniStorage.

Parameters:

sn - Specifies the input/output schema or record name.

opts - Options for AvroStorage:

-namespace Namespace for an automatically generated output schema.

-schemafile Specifies URL for avro schema file from which to read the input schema (can be local file, hdfs, url, etc).

-examplefile Specifies URL for avro data file from which to copy the input schema (can be local file, hdfs, url, etc).

-allowrecursive Option to allow recursive schema definitions (default is false).

Method Detail

getInputFormat

public org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.avro.generic.GenericData.Record> getInputFormat()
                                                                                                                                     throws IOException

Description copied from class: LoadFunc

This will be called during planning on the front end. This is the instance of InputFormat (rather than the class name) because the load function may need to instantiate the InputFormat in order to control how it is constructed.

Overrides:

getInputFormat in class AvroStorage

Returns:

the InputFormat associated with this loader.

Throws:

IOException - if there is an exception during InputFormat construction

See Also:

LoadFunc.getInputFormat()

getOutputFormat

public org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Object> getOutputFormat()
                                                                                                   throws IOException

Description copied from interface: StoreFuncInterface

Return the OutputFormat associated with StoreFuncInterface. This will be called on the front end during planning and on the backend during execution.

Specified by:

getOutputFormat in interface StoreFuncInterface

Overrides:

getOutputFormat in class AvroStorage

Returns:

the OutputFormat associated with StoreFuncInterface

Throws:

IOException - if an exception occurs while constructing the OutputFormat

getAvroSchema

public org.apache.avro.Schema getAvroSchema(org.apache.hadoop.fs.Path[] p,
                                   org.apache.hadoop.mapreduce.Job job)
                                     throws IOException

Description copied from class: AvroStorage

Reads the avro schemas at the specified location.

Overrides:

getAvroSchema in class AvroStorage

Parameters:

p - Location of file

job - Hadoop job object

Returns:

an Avro Schema object derived from the specified file

Throws:

IOException