org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce.ReduceWithComparator

java.lang.Object
  extended by org.apache.hadoop.mapred.MapReduceBase
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
          extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.ReduceWithComparator
All Implemented Interfaces:
Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
Enclosing class:
PigMapReduce

public static class PigMapReduce.ReduceWithComparator
extends PigMapReduce.Reduce

This "specialized" reduce class is ONLY to be used in pig queries with order by a udf. A UDF used for comparison in the order by expects to be handed tuples. Hence a specialized map class (PigMapReduce.MapWithComparator) ensures that the "key" used in the order by is wrapped into a tuple (if it isn't already a tuple). This reduce class unwraps this tuple in the case where the map had wrapped into a tuple and handes the "unwrapped" key to the POPackage for processing


Field Summary
 
Fields inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
errorInReduce, initialized, log, outputCollector, pack, rp
 
Constructor Summary
PigMapReduce.ReduceWithComparator()
           
 
Method Summary
 void configure(org.apache.hadoop.mapred.JobConf jConf)
          Configures the Reduce plan, the POPackage operator and the reporter thread
 void reduce(PigNullableWritable key, Iterator<NullableTuple> tupIter, org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc, org.apache.hadoop.mapred.Reporter reporter)
          The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type.
 
Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
close, processOnePackageOutput, runPipeline
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PigMapReduce.ReduceWithComparator

public PigMapReduce.ReduceWithComparator()
Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf jConf)
Configures the Reduce plan, the POPackage operator and the reporter thread

Specified by:
configure in interface org.apache.hadoop.mapred.JobConfigurable
Overrides:
configure in class PigMapReduce.Reduce

reduce

public void reduce(PigNullableWritable key,
                   Iterator<NullableTuple> tupIter,
                   org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc,
                   org.apache.hadoop.mapred.Reporter reporter)
            throws IOException
The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type. The package result is either collected as is, if the reduce plan is empty or after passing through the reduce plan.

Specified by:
reduce in interface org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
Overrides:
reduce in class PigMapReduce.Reduce
Throws:
IOException


Copyright © ${year} The Apache Software Foundation