org.apache.pig.impl.util

Class UDFContext

java.lang.Object

org.apache.pig.impl.util.UDFContext

public class UDFContext
extends Object

Method Summary

Modifier and Type	Method and Description
void	addJobConf(org.apache.hadoop.conf.Configuration conf) Adds the JobConf to this singleton.
UDFContext	clone() Make a shallow copy of the context.
void	deserialize() Populate the udfConfs field.
void	deserializeForSpark(String udfConfsStr, String clientSysPropsStr)
Properties	getClientSysProps()
Properties	getClientSystemProps() Get the System Properties (Read only) as on the client machine from where Pig was launched.
org.apache.hadoop.conf.Configuration	getJobConf() Get the JobConf.
HashMap<org.apache.pig.impl.util.UDFContext.UDFContextKey,Properties>	getUdfConfs()
static UDFContext	getUDFContext()
Properties	getUDFProperties(Class c) Get a properties object that is specific to this UDF.
Properties	getUDFProperties(Class c, String[] args) Get a properties object that is specific to this UDF.
boolean	isFrontend() Convenience method for UDF code to check where it runs (see PIG-2576)
boolean	isUDFConfEmpty()
void	reset()
String	serialize()
void	serialize(org.apache.hadoop.conf.Configuration conf) Serialize the UDF specific information into an instance of JobConf.
void	setClientSystemProps(Properties properties)
static void	setUdfContext(UDFContext udfContext)
static void	staticDataCleanup()

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getUDFContext

public static UDFContext getUDFContext()

Returns:

a Thread Local UDFContext

setUdfContext

public static void setUdfContext(UDFContext udfContext)

staticDataCleanup

public static void staticDataCleanup()

getUdfConfs

public HashMap<org.apache.pig.impl.util.UDFContext.UDFContextKey,Properties> getUdfConfs()

setClientSystemProps

public void setClientSystemProps(Properties properties)

getClientSystemProps

public Properties getClientSystemProps()

Get the System Properties (Read only) as on the client machine from where Pig was launched. This will include command line properties passed at launch time

Returns:

client side System Properties including command line properties

addJobConf

public void addJobConf(org.apache.hadoop.conf.Configuration conf)

Adds the JobConf to this singleton. Will be called on the backend by the Map and Reduce functions so that UDFs can obtain the JobConf on the backend.

getJobConf

public org.apache.hadoop.conf.Configuration getJobConf()

Get the JobConf. This should only be called on the backend. It will return null on the frontend.

Returns:

JobConf for this job. This is a copy of the JobConf. Nothing written here will be kept by the system. getUDFConf should be used for recording UDF specific information.

getUDFProperties

public Properties getUDFProperties(Class c,
                                   String[] args)

Get a properties object that is specific to this UDF. Note that if a given UDF is called multiple times in a script, and each instance passes different arguments, then each will be provided with different configuration object. This can be used by loaders to pass their input object path or URI and separate themselves from other instances of the same loader. Constructor arguments could also be used, as they are available on both the front and back end. Note that this can only be used to share information across instantiations of the same function in the front end and between front end and back end. It cannot be used to share information between instantiations (that is, between map and/or reduce instances) on the back end at runtime.

Parameters:

c - of the UDF obtaining the properties object.

args - String arguments that make this instance of the UDF unique.

Returns:

A reference to the properties object specific to the calling UDF. This is a reference, not a copy. Any changes to this object will automatically be propogated to other instances of the UDF calling this function.

getUDFProperties

public Properties getUDFProperties(Class c)

Get a properties object that is specific to this UDF. Note that if a given UDF is called multiple times in a script, they will all be provided the same configuration object. It is up to the UDF to make sure the multiple instances do not stomp on each other. It is guaranteed that this properties object will be separate from that provided to any other UDF. Note that this can only be used to share information across instantiations of the same function in the front end and between front end and back end. It cannot be used to share information between instantiations (that is, between map and/or reduce instances) on the back end at runtime.

Parameters:

c - of the UDF obtaining the properties object.

Returns:

A reference to the properties object specific to the calling UDF. This is a reference, not a copy. Any changes to this object will automatically be propogated to other instances of the UDF calling this function.

serialize

public void serialize(org.apache.hadoop.conf.Configuration conf)
               throws IOException

Serialize the UDF specific information into an instance of JobConf. This function is intended to be called on the front end in preparation for sending the data to the backend.

Parameters:

conf - JobConf to serialize into

Throws:

IOException - if underlying serialization throws it

serialize

public String serialize()

deserialize

public void deserialize()
                 throws IOException

Populate the udfConfs field. This function is intended to be called by Map.configure or Reduce.configure on the backend. It assumes that addJobConf has already been called.

Throws:

IOException - if underlying deseralization throws it

deserializeForSpark

public void deserializeForSpark(String udfConfsStr,
                                String clientSysPropsStr)
                         throws IOException

Throws:

IOException

reset

public void reset()

isUDFConfEmpty

public boolean isUDFConfEmpty()

isFrontend

public boolean isFrontend()

Convenience method for UDF code to check where it runs (see PIG-2576)

Returns:

boolean type value

clone

public UDFContext clone()

Make a shallow copy of the context.

Overrides:

clone in class Object

getClientSysProps

public Properties getClientSysProps()