org.apache.pig.piggybank.storage.avro
Class PigSchema2Avro

java.lang.Object
  extended by org.apache.pig.piggybank.storage.avro.PigSchema2Avro

public class PigSchema2Avro
extends Object

This class contains functions to convert Pig schema to Avro. It consists of two sets of methods: 1. Convert a Pig schema to Avro schema; 2. Validate whether a Pig schema is compatible with a given Avro schema. Notice that the Avro schema doesn't need to cover all fields in Pig schema, and the missing fields are converted using methods in set 1.


Field Summary
static String FIELD_NAME
           
static String TUPLE_NAME
           
static int tupleIndex
           
 
Constructor Summary
PigSchema2Avro()
           
 
Method Summary
protected static org.apache.avro.Schema convert(ResourceSchema.ResourceFieldSchema pigSchema, boolean nullable)
          Convert a Pig ResourceFieldSchema to avro schema
static org.apache.avro.Schema convert(ResourceSchema pigSchema, boolean nullable)
          Convert a pig ResourceSchema to avro schema
protected static org.apache.avro.Schema convertPrimitiveType(byte pigType)
          Convert Pig primitive type to Avro type
protected static org.apache.avro.Schema convertRecord(ResourceSchema.ResourceFieldSchema[] pigFields, boolean nullable)
          Convert pig data to Avro record
protected static boolean isCompatible(org.apache.avro.Schema avroSchema, ResourceSchema.ResourceFieldSchema pigSchema)
          Check whether Avro type is compatible with Pig type
static void setTupleIndex(int index)
          This is a painful hack to make unit tests pass.
protected static org.apache.avro.Schema validateAndConvert(org.apache.avro.Schema avroSchema, ResourceSchema.ResourceFieldSchema pigSchema)
          Validate whether pigSchema is compatible with avroSchema and convert those Pig fields with to corresponding Avro schemas.
static org.apache.avro.Schema validateAndConvert(org.apache.avro.Schema avroSchema, ResourceSchema pigSchema)
          Validate whether pigSchema is compatible with avroSchema
protected static org.apache.avro.Schema validateAndConvertRecord(org.apache.avro.Schema avroSchema, ResourceSchema.ResourceFieldSchema[] pigFields)
          Validate a Pig tuple is compatible with Avro record.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

TUPLE_NAME

public static final String TUPLE_NAME
See Also:
Constant Field Values

FIELD_NAME

public static final String FIELD_NAME
See Also:
Constant Field Values

tupleIndex

public static int tupleIndex
Constructor Detail

PigSchema2Avro

public PigSchema2Avro()
Method Detail

convert

public static org.apache.avro.Schema convert(ResourceSchema pigSchema,
                                             boolean nullable)
                                      throws IOException
Convert a pig ResourceSchema to avro schema

Throws:
IOException

convert

protected static org.apache.avro.Schema convert(ResourceSchema.ResourceFieldSchema pigSchema,
                                                boolean nullable)
                                         throws IOException
Convert a Pig ResourceFieldSchema to avro schema

Throws:
IOException

convertRecord

protected static org.apache.avro.Schema convertRecord(ResourceSchema.ResourceFieldSchema[] pigFields,
                                                      boolean nullable)
                                               throws IOException
Convert pig data to Avro record

Throws:
IOException

setTupleIndex

public static void setTupleIndex(int index)
This is a painful hack to make unit tests pass. The static counter holds state between unit tests, so it needs to be reset. Otherwise tests will fail if their order is swapped or a new test is added.

Parameters:
index -

convertPrimitiveType

protected static org.apache.avro.Schema convertPrimitiveType(byte pigType)
                                                      throws IOException
Convert Pig primitive type to Avro type

Throws:
IOException

validateAndConvert

public static org.apache.avro.Schema validateAndConvert(org.apache.avro.Schema avroSchema,
                                                        ResourceSchema pigSchema)
                                                 throws IOException
Validate whether pigSchema is compatible with avroSchema

Throws:
IOException

validateAndConvert

protected static org.apache.avro.Schema validateAndConvert(org.apache.avro.Schema avroSchema,
                                                           ResourceSchema.ResourceFieldSchema pigSchema)
                                                    throws IOException
Validate whether pigSchema is compatible with avroSchema and convert those Pig fields with to corresponding Avro schemas.

Throws:
IOException

validateAndConvertRecord

protected static org.apache.avro.Schema validateAndConvertRecord(org.apache.avro.Schema avroSchema,
                                                                 ResourceSchema.ResourceFieldSchema[] pigFields)
                                                          throws IOException
Validate a Pig tuple is compatible with Avro record. If the Avro schema is not complete (with uncovered fields), then convert those fields using methods in set 1. Notice that users can get rid of Pig tuple wrappers, e.g. an Avro schema "int" is compatible with a Pig schema "T:(int)"

Throws:
IOException

isCompatible

protected static boolean isCompatible(org.apache.avro.Schema avroSchema,
                                      ResourceSchema.ResourceFieldSchema pigSchema)
Check whether Avro type is compatible with Pig type



Copyright © 2007-2012 The Apache Software Foundation