org.apache.pig.builtin
Class TrevniStorage

java.lang.Object
  extended by org.apache.pig.LoadFunc
      extended by org.apache.pig.builtin.AvroStorage
          extended by org.apache.pig.builtin.TrevniStorage
All Implemented Interfaces:
LoadMetadata, LoadPushDown, StoreFuncInterface

public class TrevniStorage
extends AvroStorage
implements LoadPushDown

Pig Store/Load Function for Trevni.


Nested Class Summary
 
Nested classes/interfaces inherited from interface org.apache.pig.LoadPushDown
LoadPushDown.OperatorSet, LoadPushDown.RequiredField, LoadPushDown.RequiredFieldList, LoadPushDown.RequiredFieldResponse
 
Field Summary
 
Fields inherited from class org.apache.pig.builtin.AvroStorage
allowRecursive, doubleColonsToDoubleUnderscores, INPUT_AVRO_SCHEMA, log, OUTPUT_AVRO_SCHEMA, requiredFieldList, schema, udfContextSignature, VISIBLE_FILES
 
Constructor Summary
TrevniStorage()
          Create new instance of TrevniStorage with no arguments (useful for loading files without specifying parameters).
TrevniStorage(String sn, String opts)
          Create new instance of TrevniStorage.
 
Method Summary
 org.apache.avro.Schema getAvroSchema(org.apache.hadoop.fs.Path[] p, org.apache.hadoop.mapreduce.Job job)
          Reads the avro schemas at the specified location.
 org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.avro.generic.GenericData.Record> getInputFormat()
          This will be called during planning on the front end.
 org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Object> getOutputFormat()
          Return the OutputFormat associated with StoreFuncInterface.
 
Methods inherited from class org.apache.pig.builtin.AvroStorage
checkSchema, cleanupOnFailure, cleanupOnSuccess, depthFirstSearchForFile, getAvroSchema, getFeatures, getInputAvroSchema, getNext, getOutputAvroSchema, getPartitionKeys, getProperties, getProperties, getSchema, getStatistics, prepareToRead, prepareToWrite, pushProjection, putNext, relToAbsPathForStoreLocation, setInputAvroSchema, setLocation, setOutputAvroSchema, setPartitionFilter, setStoreFuncUDFContextSignature, setStoreLocation, setUDFContextSignature
 
Methods inherited from class org.apache.pig.LoadFunc
getAbsolutePath, getLoadCaster, getPathStrings, join, relativeToAbsolutePath, warn
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.pig.LoadPushDown
getFeatures, pushProjection
 

Constructor Detail

TrevniStorage

public TrevniStorage()
Create new instance of TrevniStorage with no arguments (useful for loading files without specifying parameters).


TrevniStorage

public TrevniStorage(String sn,
                     String opts)
Create new instance of TrevniStorage.

Parameters:
sn - Specifies the input/output schema or record name.
opts - Options for AvroStorage:
  • -namespace Namespace for an automatically generated output schema.
  • -schemafile Specifies URL for avro schema file from which to read the input schema (can be local file, hdfs, url, etc).
  • -examplefile Specifies URL for avro data file from which to copy the input schema (can be local file, hdfs, url, etc).
  • -allowrecursive Option to allow recursive schema definitions (default is false).
  • Method Detail

    getInputFormat

    public org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.avro.generic.GenericData.Record> getInputFormat()
                                                                                                                                         throws IOException
    Description copied from class: LoadFunc
    This will be called during planning on the front end. This is the instance of InputFormat (rather than the class name) because the load function may need to instantiate the InputFormat in order to control how it is constructed.

    Overrides:
    getInputFormat in class AvroStorage
    Returns:
    the InputFormat associated with this loader.
    Throws:
    IOException - if there is an exception during InputFormat construction
    See Also:
    LoadFunc.getInputFormat()

    getOutputFormat

    public org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Object> getOutputFormat()
                                                                                                       throws IOException
    Description copied from interface: StoreFuncInterface
    Return the OutputFormat associated with StoreFuncInterface. This will be called on the front end during planning and on the backend during execution.

    Specified by:
    getOutputFormat in interface StoreFuncInterface
    Overrides:
    getOutputFormat in class AvroStorage
    Returns:
    the OutputFormat associated with StoreFuncInterface
    Throws:
    IOException - if an exception occurs while constructing the OutputFormat

    getAvroSchema

    public org.apache.avro.Schema getAvroSchema(org.apache.hadoop.fs.Path[] p,
                                                org.apache.hadoop.mapreduce.Job job)
                                         throws IOException
    Description copied from class: AvroStorage
    Reads the avro schemas at the specified location.

    Overrides:
    getAvroSchema in class AvroStorage
    Parameters:
    p - Location of file
    job - Hadoop job object
    Returns:
    an Avro Schema object derived from the specified file
    Throws:
    IOException


    Copyright © 2007-2012 The Apache Software Foundation