org.apache.pig.backend.hadoop.executionengine.spark.converter

Class LoadConverter

java.lang.Object

org.apache.pig.backend.hadoop.executionengine.spark.converter.LoadConverter

All Implemented Interfaces:

RDDConverter<Tuple,Tuple,POLoad>

public class LoadConverter
extends Object
implements RDDConverter<Tuple,Tuple,POLoad>

Converter that loads data via POLoad and converts it to RRD<Tuple>. Abuses the interface a bit in that there is no input RRD to convert in this case. Instead input is the source path of the POLoad.

Constructor Summary

Constructors

Constructor and Description
LoadConverter(PigContext pigContext, PhysicalPlan physicalPlan, SparkContext sparkContext, org.apache.hadoop.mapred.JobConf jobConf, SparkEngineConf sparkEngineConf)

Method Summary

Modifier and Type	Method and Description
<any>	convert(List<<any>> predecessorRdds, POLoad op)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LoadConverter

public LoadConverter(PigContext pigContext,
                     PhysicalPlan physicalPlan,
                     SparkContext sparkContext,
                     org.apache.hadoop.mapred.JobConf jobConf,
                     SparkEngineConf sparkEngineConf)

Method Detail

convert

public <any> convert(List<<any>> predecessorRdds,
                     POLoad op)
              throws IOException

Specified by:

convert in interface RDDConverter<Tuple,Tuple,POLoad>

Throws:

IOException