org.apache.pig.piggybank.storage.hiverc

Class HiveRCSchemaUtil

java.lang.Object

org.apache.pig.piggybank.storage.hiverc.HiveRCSchemaUtil

public class HiveRCSchemaUtil
extends Object

Implements helper methods for:

• Parsing the hive table schema string.
• Converting from hive to pig types
• Converting from pig to hive types

Field Summary

Fields

Modifier and Type	Field and Description
protected static Pattern	ptypes Regex to filter out column types

Constructor Summary

Constructors

Constructor and Description
HiveRCSchemaUtil()

Method Summary

Modifier and Type	Method and Description
static Set<String>	compileSet(String columnsToRead) Returns a set of columns, with the column names strimmed
static String	extractDayDate(String fileName) Extract the date from the hive file names e.g /user/hive/warehouse/table/daydate=2009-10-01/upload001/0002.dat This method will extract the 2009-10-01 from this name.
static Object	extractPigTypeFromHiveType(Object value) Converts from a hive type to a pig type
static byte	findPigDataType(String hiveType) Returns the pig DataType for the hive type
static String	listToString(List<String> list) Trims items in the list.
static Tuple	parseLazyArrayToPigArray(org.apache.hadoop.hive.serde2.lazy.LazyArray arr) Converts the LazyArray to a Tuple.
static InternalMap	parseLazyMapToPigMap(org.apache.hadoop.hive.serde2.lazy.LazyMap map) Converts the LazyMap to a InternalMap.
static List<String>	parseSchema(Pattern pattern, String schema) General schema parsing method, is used to parse the column names.
static List<String>	parseSchemaTypes(String schema) Parses the schema types and returns a List of these.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ptypes

protected static final Pattern ptypes

Regex to filter out column types

Constructor Detail

HiveRCSchemaUtil

public HiveRCSchemaUtil()

Method Detail

parseSchema

public static List<String> parseSchema(Pattern pattern,
                                       String schema)

General schema parsing method, is used to parse the column names.

Parameters:

pattern - String

schema - String

Returns:

List of String

parseSchemaTypes

public static List<String> parseSchemaTypes(String schema)

Parses the schema types and returns a List of these.

Parameters:

schema -

Returns:

List of String

listToString

public static final String listToString(List<String> list)

Trims items in the list.

Parameters:

list -

Returns:

String

extractDayDate

public static final String extractDayDate(String fileName)

Extract the date from the hive file names e.g /user/hive/warehouse/table/daydate=2009-10-01/upload001/0002.dat
This method will extract the 2009-10-01 from this name.

Parameters:

fileName -

Returns:

String

compileSet

public static final Set<String> compileSet(String columnsToRead)

Returns a set of columns, with the column names strimmed

Parameters:

columnsToRead -

Returns:

Set

findPigDataType

public static byte findPigDataType(String hiveType)

Returns the pig DataType for the hive type

Parameters:

hiveType -

Returns:

byte from DataType

extractPigTypeFromHiveType

public static Object extractPigTypeFromHiveType(Object value)

Converts from a hive type to a pig type

Parameters:

value - Object hive type

Returns:

Object pig type

parseLazyMapToPigMap

public static InternalMap parseLazyMapToPigMap(org.apache.hadoop.hive.serde2.lazy.LazyMap map)

Converts the LazyMap to a InternalMap.

Parameters:

map - LazyMap

Returns:

InternalMap

parseLazyArrayToPigArray

public static Tuple parseLazyArrayToPigArray(org.apache.hadoop.hive.serde2.lazy.LazyArray arr)

Converts the LazyArray to a Tuple.

Parameters:

arr - LazyArray

Returns:

Tuple