org.apache.pig.impl.util.avro

Class AvroStorageSchemaConversionUtilities

java.lang.Object

org.apache.pig.impl.util.avro.AvroStorageSchemaConversionUtilities

public class AvroStorageSchemaConversionUtilities
extends Object

Static methods for converting from Avro Schema object to Pig Schema objects, and vice versa.

Constructor Summary

Constructors

Constructor and Description
AvroStorageSchemaConversionUtilities()

Method Summary

Modifier and Type	Method and Description
static ResourceSchema	avroSchemaToResourceSchema(org.apache.avro.Schema s, Boolean allowRecursiveSchema) Translates an Avro schema to a Resource Schema (for Pig).
static byte	getPigType(org.apache.avro.Schema s) Determines the pig object type of the Avro schema.
static boolean	isNullableUnion(org.apache.avro.Schema s) Checks to see if an avro schema is a combination of null and another object.
static boolean	isUnionOfSimpleTypes(org.apache.avro.Schema s)
static org.apache.avro.Schema	newSchemaFromRequiredFieldList(org.apache.avro.Schema oldSchema, List<LoadPushDown.RequiredField> rfl) Takes an Avro Schema and a Pig RequiredFieldList and returns a new schema with only the required fields, or no if the function can't extract only those fields.
static org.apache.avro.Schema	newSchemaFromRequiredFieldList(org.apache.avro.Schema oldSchema, LoadPushDown.RequiredFieldList rfl) Takes an Avro Schema and a Pig RequiredFieldList and returns a new schema with only the requried fields, or no if the function can't extract only those fields.
static org.apache.avro.Schema	removeSimpleUnion(org.apache.avro.Schema s) Given an input schema that is a union of an avro schema and null (or just a union with one type), return the avro schema.
static org.apache.avro.Schema	resourceSchemaToAvroSchema(ResourceSchema rs, String recordName, String recordNameSpace, Map<String,List<org.apache.avro.Schema>> definedRecordNames, Boolean doubleColonsToDoubleUnderscores) Translated a ResourceSchema to an Avro Schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroStorageSchemaConversionUtilities

public AvroStorageSchemaConversionUtilities()

Method Detail

getPigType

public static byte getPigType(org.apache.avro.Schema s)
                       throws ExecException

Determines the pig object type of the Avro schema.

Parameters:

s - The avro schema for which to determine the type

Returns:

the byte representing the schema type

Throws:

ExecException

See Also:

Schema.Type

isUnionOfSimpleTypes

public static boolean isUnionOfSimpleTypes(org.apache.avro.Schema s)

avroSchemaToResourceSchema

public static ResourceSchema avroSchemaToResourceSchema(org.apache.avro.Schema s,
                                                        Boolean allowRecursiveSchema)
                                                 throws IOException

Translates an Avro schema to a Resource Schema (for Pig).

Parameters:

s - The avro schema for which to determine the type

allowRecursiveSchema - Flag indicating whether to throw an error if a recursive schema definition is found

Returns:

the corresponding pig schema

Throws:

IOException

resourceSchemaToAvroSchema

public static org.apache.avro.Schema resourceSchemaToAvroSchema(ResourceSchema rs,
                                                                String recordName,
                                                                String recordNameSpace,
                                                                Map<String,List<org.apache.avro.Schema>> definedRecordNames,
                                                                Boolean doubleColonsToDoubleUnderscores)
                                                         throws IOException

Translated a ResourceSchema to an Avro Schema.

Parameters:

rs - Input schema.

recordName - Record name

recordNameSpace - Namespace

definedRecordNames - Map of already defined record names to schema objects

Returns:

the translated schema

Throws:

IOException

isNullableUnion

public static boolean isNullableUnion(org.apache.avro.Schema s)

Checks to see if an avro schema is a combination of null and another object.

Parameters:

s - The object to check

Returns:

whether it's a nullable union

removeSimpleUnion

public static org.apache.avro.Schema removeSimpleUnion(org.apache.avro.Schema s)

Given an input schema that is a union of an avro schema and null (or just a union with one type), return the avro schema.

Parameters:

s - The input schema object

Returns:

The non-null part of the union

newSchemaFromRequiredFieldList

public static org.apache.avro.Schema newSchemaFromRequiredFieldList(org.apache.avro.Schema oldSchema,
                                                                    LoadPushDown.RequiredFieldList rfl)

Takes an Avro Schema and a Pig RequiredFieldList and returns a new schema with only the requried fields, or no if the function can't extract only those fields. Useful for push down projections.

Parameters:

oldSchema - The avro schema from which to extract the schema

rfl - the Pig required field list

Returns:

the new schema, or null

newSchemaFromRequiredFieldList

public static org.apache.avro.Schema newSchemaFromRequiredFieldList(org.apache.avro.Schema oldSchema,
                                                                    List<LoadPushDown.RequiredField> rfl)

Takes an Avro Schema and a Pig RequiredFieldList and returns a new schema with only the required fields, or no if the function can't extract only those fields. Useful for push down projections.

Parameters:

oldSchema - The avro schema from which to extract the schema

rfl - List of required fields

Returns:

the new schema