org.apache.pig.piggybank.storage.hiverc

Class HiveRCSchemaUtil

java.lang.Object

org.apache.pig.piggybank.storage.hiverc.HiveRCSchemaUtil

public class HiveRCSchemaUtil
extends java.lang.Object

Implements helper methods for:

• Parsing the hive table schema string.
• Converting from hive to pig types
• Converting from pig to hive types

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.util.regex.Pattern	ptypes Regex to filter out column types

Constructor Summary

Constructors

Constructor and Description
HiveRCSchemaUtil()

Method Summary

Methods

Modifier and Type	Method and Description
static java.util.Set<java.lang.String>	compileSet(java.lang.String columnsToRead) Returns a set of columns, with the column names strimmed
static java.lang.String	extractDayDate(java.lang.String fileName) Extract the date from the hive file names e.g /user/hive/warehouse/table/daydate=2009-10-01/upload001/0002.dat This method will extract the 2009-10-01 from this name.
static java.lang.Object	extractPigTypeFromHiveType(java.lang.Object value) Converts from a hive type to a pig type
static byte	findPigDataType(java.lang.String hiveType) Returns the pig DataType for the hive type
static java.lang.String	listToString(java.util.List<java.lang.String> list) Trims items in the list.
static Tuple	parseLazyArrayToPigArray(org.apache.hadoop.hive.serde2.lazy.LazyArray arr) Converts the LazyArray to a Tuple.
static InternalMap	parseLazyMapToPigMap(org.apache.hadoop.hive.serde2.lazy.LazyMap map) Converts the LazyMap to a InternalMap.
static java.util.List<java.lang.String>	parseSchema(java.util.regex.Pattern pattern, java.lang.String schema) General schema parsing method, is used to parse the column names.
static java.util.List<java.lang.String>	parseSchemaTypes(java.lang.String schema) Parses the schema types and returns a List of these.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ptypes

protected static final java.util.regex.Pattern ptypes

Regex to filter out column types

Constructor Detail

HiveRCSchemaUtil

public HiveRCSchemaUtil()

Method Detail

parseSchema

public static java.util.List<java.lang.String> parseSchema(java.util.regex.Pattern pattern,
                                           java.lang.String schema)

General schema parsing method, is used to parse the column names.

Parameters:

pattern - String

schema - String

Returns:

List of String

parseSchemaTypes

public static java.util.List<java.lang.String> parseSchemaTypes(java.lang.String schema)

Parses the schema types and returns a List of these.

Parameters:

schema -

Returns:

List of String

listToString

public static final java.lang.String listToString(java.util.List<java.lang.String> list)

Trims items in the list.

Parameters:

list -

Returns:

String

extractDayDate

public static final java.lang.String extractDayDate(java.lang.String fileName)

Extract the date from the hive file names e.g /user/hive/warehouse/table/daydate=2009-10-01/upload001/0002.dat
This method will extract the 2009-10-01 from this name.

Parameters:

fileName -

Returns:

String

compileSet

public static final java.util.Set<java.lang.String> compileSet(java.lang.String columnsToRead)

Returns a set of columns, with the column names strimmed

Parameters:

columnsToRead -

Returns:

Set

findPigDataType

public static byte findPigDataType(java.lang.String hiveType)

Returns the pig DataType for the hive type

Parameters:

hiveType -

Returns:

byte from DataType

extractPigTypeFromHiveType

public static java.lang.Object extractPigTypeFromHiveType(java.lang.Object value)

Converts from a hive type to a pig type

Parameters:

value - Object hive type

Returns:

Object pig type

parseLazyMapToPigMap

public static InternalMap parseLazyMapToPigMap(org.apache.hadoop.hive.serde2.lazy.LazyMap map)

Converts the LazyMap to a InternalMap.

Parameters:

map - LazyMap

Returns:

InternalMap

parseLazyArrayToPigArray

public static Tuple parseLazyArrayToPigArray(org.apache.hadoop.hive.serde2.lazy.LazyArray arr)

Converts the LazyArray to a Tuple.

Parameters:

arr - LazyArray

Returns:

Tuple