org.apache.pig.impl.logicalLayer
Class LOLoad

java.lang.Object
  org.apache.pig.impl.plan.Operator<LOVisitor>
      org.apache.pig.impl.logicalLayer.LogicalOperator
          org.apache.pig.impl.logicalLayer.RelationalOperator
              org.apache.pig.impl.logicalLayer.LOLoad

All Implemented Interfaces:

Serializable, Cloneable, Comparable<Operator>

public class LOLoad
extends RelationalOperator

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.pig.impl.logicalLayer.LogicalOperator

mAlias, mIsProjectionMapComputed, mIsSchemaComputed, mPinnedOptions, mPlan, mProjectionMap, mRequestedParallelism, mSchema, mType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

LOLoad(LogicalPlan plan, OperatorKey key, FileSpec inputFileSpec, org.apache.hadoop.conf.Configuration conf)

Method Summary

org.apache.hadoop.conf.Configuration	getConfiguration()
Schema	getDeterminedSchema()
Schema	getEnforcedSchema()
FileSpec	getInputFile()
LoadFunc	getLoadFunc()
ProjectionMap	getProjectionMap() Produce a map describing how this operator modifies its projection.
List<RequiredFields>	getRelevantInputs(int output, int column) Get relevant input columns of a particular output column.
LoadPushDown.RequiredFieldList	getRequiredFieldList()
List<RequiredFields>	getRequiredFields() Get a list of fields that this operator requires.
Schema	getSchema() Get a copy of the schema for the output of this operator.
String	getSchemaFile()
byte	getType() Get the type of this operator.
String	name()
boolean	pruneColumns(List<Pair<Integer,Integer>> columns)
LoadPushDown.RequiredFieldResponse	pushProjection(LoadPushDown.RequiredFieldList requiredFieldList)
void	setAlias(String newAlias)
void	setEnforcedSchema(Schema enforcedSchema) Set this when user enforces schema
void	setInputFile(FileSpec inputFileSpec)
void	setSchema(Schema schema) Set the output schema for this operator.
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
void	visit(LOVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.impl.logicalLayer.RelationalOperator

insertPlainForEachAfter, pruneColumnInPlan, regenerateProjectionMap, unsetProjectionMap

Methods inherited from class org.apache.pig.impl.logicalLayer.LogicalOperator

clone, forceSchema, getAlias, getOperatorKey, getPlan, getRequestedParallelism, isPinnedOption, pinOption, reconcileSchema, regenerateSchema, setCanonicalNames, setPlan, setRequestedParallelism, setSchemaComputed, setType, supportsMultipleOutputs, toString, unsetSchema

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, hashCode, rewire

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

LOLoad

public LOLoad(LogicalPlan plan,
              OperatorKey key,
              FileSpec inputFileSpec,
              org.apache.hadoop.conf.Configuration conf)
       throws IOException

Parameters:

plan - LogicalPlan this operator is a part of.

key - OperatorKey for this operator

inputFileSpec - the file to be loaded *

conf - the read-only configuration object

Throws:

IOException

Method Detail

getInputFile

public FileSpec getInputFile()

setInputFile

public void setInputFile(FileSpec inputFileSpec)
                  throws IOException

Throws:

IOException

getSchemaFile

public String getSchemaFile()

getLoadFunc

public LoadFunc getLoadFunc()

name

public String name()

Specified by:

name in class Operator<LOVisitor>

getSchema

public Schema getSchema()
                 throws FrontendException

Description copied from class: LogicalOperator

Get a copy of the schema for the output of this operator.

Specified by:

getSchema in class LogicalOperator

Throws:

FrontendException

setSchema

public void setSchema(Schema schema)
               throws FrontendException

Description copied from class: LogicalOperator

Set the output schema for this operator. If a schema already exists, an attempt will be made to reconcile it with this new schema.

Overrides:

setSchema in class LogicalOperator

Parameters:

schema - Schema to set.

Throws:

FrontendException

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<LOVisitor>

Returns:

true if it does, otherwise false.

visit

public void visit(LOVisitor v)
           throws VisitorException

Description copied from class: LogicalOperator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class LogicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

getEnforcedSchema

public Schema getEnforcedSchema()

setEnforcedSchema

public void setEnforcedSchema(Schema enforcedSchema)

Set this when user enforces schema

Parameters:

enforcedSchema -

getType

public byte getType()

Description copied from class: LogicalOperator

Get the type of this operator.

Overrides:

getType in class LogicalOperator

getDeterminedSchema

public Schema getDeterminedSchema()

Returns:

the DeterminedSchema

getProjectionMap

public ProjectionMap getProjectionMap()

Description copied from class: RelationalOperator

Produce a map describing how this operator modifies its projection.

Overrides:

getProjectionMap in class RelationalOperator

Returns:

ProjectionMap null indicates it does not know how the projection changes, for example a join of two inputs where one input does not have a schema.

getRequiredFields

public List<RequiredFields> getRequiredFields()

Description copied from class: RelationalOperator

Get a list of fields that this operator requires. This is not necessarily equivalent to the list of fields the operator projects. For example, a filter will project anything passed to it, but requires only the fields explicitly referenced in its filter expression.

Overrides:

getRequiredFields in class RelationalOperator

Returns:

list of RequiredFields null indicates that the operator does not need any fields from its input.

getRelevantInputs

public List<RequiredFields> getRelevantInputs(int output,
                                              int column)
                                       throws FrontendException

Description copied from class: RelationalOperator

Get relevant input columns of a particular output column. The resulting input columns are necessary components only to the output column. Input columns needed by the entire RelationalOperator thus indirectly contribute to the output columns are not counted. Those are required columns. eg1: A = load 'a' AS (a0, a1, a2); B = filter a by a0=='1'; Relevant input columns for B.$1 is A.a1 because A.a1 direct generate B.$1. A.a0 is needed by the filter operator and it is considered as required fields for the relational operator. eg2: A = load 'a' AS (a0, a1); B = load 'b' AS (b0, b1); C = join A by a0, B by b0; Relevant input columns for C.$0 is A.a0. Relevant input columns for C.$1 is A.a1. eg3: A = load 'a' AS (a0, a1); B = load 'b' AS (b0, b1); C = cogroup A by a0, B by b0; Relevant input columns for C.$0 is A.a0, B.b0. Relevant input columns for C.$1 is A.*. Relevant input columns for C.$2 is B.*. eg4: A = load 'a' AS (a0, a1, a2); B = foreach A generate a1, a0+a2; Relevant input columns for B.$0 is A.a1. Relevant input columns for B.$1 is A.a0 and A.a2. eg5: A = load 'a' AS (a0, a1, a2); B = foreach A generate a1, *; Relevant input columns for B.$0 is A.a1. Relevant input columns for B.$1 is A.a0. Relevant input columns for B.$2 is A.a1. Relevant input columns for B.$3 is A.a2.

Specified by:

getRelevantInputs in class RelationalOperator

Parameters:

output - output index. Only LOSplit have output other than 0 currently

column - output column

Returns:

List of relevant input columns. null if Pig cannot determine relevant inputs or any error occurs

Throws:

FrontendException

pushProjection

public LoadPushDown.RequiredFieldResponse pushProjection(LoadPushDown.RequiredFieldList requiredFieldList)
                                                  throws FrontendException

Throws:

FrontendException

setAlias

public void setAlias(String newAlias)

Overrides:

setAlias in class LogicalOperator

pruneColumns

public boolean pruneColumns(List<Pair<Integer,Integer>> columns)
                     throws FrontendException

Overrides:

pruneColumns in class RelationalOperator

Throws:

FrontendException

getRequiredFieldList

public LoadPushDown.RequiredFieldList getRequiredFieldList()

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()