org.apache.pig.backend.hadoop.executionengine.spark.running

Class PigInputFormatSpark

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.Text,Tuple>

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat

org.apache.pig.backend.hadoop.executionengine.spark.running.PigInputFormatSpark

public class PigInputFormatSpark
extends PigInputFormat

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat

PigInputFormat.RecordReaderFactory

Field Summary

Fields inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat

log, PIG_INPUT_LIMITS, PIG_INPUT_SIGNATURES, PIG_INPUT_TARGETS, PIG_LOADS

Constructor Summary

Constructors

Constructor and Description
PigInputFormatSpark()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
java.util.List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext jobcontext) This is where we have to wrap PigSplits into SparkPigSplits

Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat

getPigSplits

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PigInputFormatSpark

public PigInputFormatSpark()

Method Detail

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                    org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                             throws java.io.IOException,
                                                                                                    java.lang.InterruptedException

Overrides:

createRecordReader in class PigInputFormat

Throws:

java.io.IOException

java.lang.InterruptedException

getSplits

public java.util.List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext jobcontext)
                                                                 throws java.io.IOException,
                                                                        java.lang.InterruptedException

This is where we have to wrap PigSplits into SparkPigSplits

Overrides:

getSplits in class PigInputFormat

Parameters:

jobcontext -

Returns:

Throws:

java.io.IOException

java.lang.InterruptedException