org.apache.pig.impl.logicalLayer
Class LOStream

java.lang.Object
  org.apache.pig.impl.plan.Operator<LOVisitor>
      org.apache.pig.impl.logicalLayer.LogicalOperator
          org.apache.pig.impl.logicalLayer.RelationalOperator
              org.apache.pig.impl.logicalLayer.LOStream

All Implemented Interfaces:

Serializable, Cloneable, Comparable<Operator>

public class LOStream
extends RelationalOperator

LOStream represents the specification of an external command to be executed in a Pig Query. LOStream encapsulates all relevant details of the command specified by the user either directly via the STREAM operator or indirectly via a DEFINE operator. It includes details such as input/output/error specifications and also files to be shipped to the cluster and files to be cached.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.pig.impl.logicalLayer.LogicalOperator

mAlias, mIsProjectionMapComputed, mIsSchemaComputed, mPinnedOptions, mPlan, mProjectionMap, mRequestedParallelism, mSchema, mType

Fields inherited from class org.apache.pig.impl.plan.Operator

mKey

Constructor Summary

LOStream(LogicalPlan plan, OperatorKey k, LogicalOperator input, ExecutableManager exeManager, StreamingCommand cmd)
Create a new LOStream with the given command.

Method Summary

ExecutableManager	getExecutableManager()
ProjectionMap	getProjectionMap() Produce a map describing how this operator modifies its projection.
List<RequiredFields>	getRelevantInputs(int output, int column) Get relevant input columns of a particular output column.
List<RequiredFields>	getRequiredFields() Get a list of fields that this operator requires.
Schema	getSchema() Get a copy of the schema for the output of this operator.
StreamingCommand	getStreamingCommand() Get the StreamingCommand object associated with this operator
String	name()
void	setOptimizedSpec(StreamingCommand.Handle handle, String spec) Set the optimized StreamingCommand.HandleSpec for the given StreamingCommand.Handle of the StreamSpec.
boolean	supportsMultipleInputs() Indicates whether this operator supports multiple inputs.
void	visit(LOVisitor v) Visit this node with the provided visitor.

Methods inherited from class org.apache.pig.impl.logicalLayer.RelationalOperator

insertPlainForEachAfter, pruneColumnInPlan, pruneColumns, regenerateProjectionMap, unsetProjectionMap

Methods inherited from class org.apache.pig.impl.logicalLayer.LogicalOperator

clone, forceSchema, getAlias, getOperatorKey, getPlan, getRequestedParallelism, getType, isPinnedOption, pinOption, reconcileSchema, regenerateSchema, setAlias, setCanonicalNames, setPlan, setRequestedParallelism, setSchema, setSchemaComputed, setType, supportsMultipleOutputs, toString, unsetSchema

Methods inherited from class org.apache.pig.impl.plan.Operator

compareTo, equals, hashCode, rewire

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

LOStream

public LOStream(LogicalPlan plan,
                OperatorKey k,
                LogicalOperator input,
                ExecutableManager exeManager,
                StreamingCommand cmd)

Create a new LOStream with the given command.

Parameters:

plan - the logical plan this operator is a part of

k - the operator key for this operator

input - operator that is input to this command

exeManager - ExecutableManager used by this streaming command.

cmd - StreamingCommand for this stream to run.

Method Detail

getStreamingCommand

public StreamingCommand getStreamingCommand()

Get the StreamingCommand object associated with this operator

Returns:

the StreamingCommand object

getSchema

public Schema getSchema()
                 throws FrontendException

Description copied from class: LogicalOperator

Get a copy of the schema for the output of this operator.

Specified by:

getSchema in class LogicalOperator

Throws:

FrontendException

setOptimizedSpec

public void setOptimizedSpec(StreamingCommand.Handle handle,
                             String spec)

Set the optimized StreamingCommand.HandleSpec for the given StreamingCommand.Handle of the StreamSpec.

Parameters:

handle - Handle to optimize

spec - optimized specification for the handle

visit

public void visit(LOVisitor v)
           throws VisitorException

Description copied from class: LogicalOperator

Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:

visit in class LogicalOperator

Parameters:

v - Visitor to visit with.

Throws:

VisitorException - if the visitor has a problem.

name

public String name()

Specified by:

name in class Operator<LOVisitor>

supportsMultipleInputs

public boolean supportsMultipleInputs()

Description copied from class: Operator

Indicates whether this operator supports multiple inputs.

Specified by:

supportsMultipleInputs in class Operator<LOVisitor>

Returns:

true if it does, otherwise false.

getExecutableManager

public ExecutableManager getExecutableManager()

Returns:

the ExecutableManager

getProjectionMap

public ProjectionMap getProjectionMap()

Description copied from class: RelationalOperator

Produce a map describing how this operator modifies its projection.

Overrides:

getProjectionMap in class RelationalOperator

Returns:

ProjectionMap null indicates it does not know how the projection changes, for example a join of two inputs where one input does not have a schema.

getRequiredFields

public List<RequiredFields> getRequiredFields()

Description copied from class: RelationalOperator

Get a list of fields that this operator requires. This is not necessarily equivalent to the list of fields the operator projects. For example, a filter will project anything passed to it, but requires only the fields explicitly referenced in its filter expression.

Overrides:

getRequiredFields in class RelationalOperator

Returns:

list of RequiredFields null indicates that the operator does not need any fields from its input.

getRelevantInputs

public List<RequiredFields> getRelevantInputs(int output,
                                              int column)
                                       throws FrontendException

Description copied from class: RelationalOperator

Get relevant input columns of a particular output column. The resulting input columns are necessary components only to the output column. Input columns needed by the entire RelationalOperator thus indirectly contribute to the output columns are not counted. Those are required columns. eg1: A = load 'a' AS (a0, a1, a2); B = filter a by a0=='1'; Relevant input columns for B.$1 is A.a1 because A.a1 direct generate B.$1. A.a0 is needed by the filter operator and it is considered as required fields for the relational operator. eg2: A = load 'a' AS (a0, a1); B = load 'b' AS (b0, b1); C = join A by a0, B by b0; Relevant input columns for C.$0 is A.a0. Relevant input columns for C.$1 is A.a1. eg3: A = load 'a' AS (a0, a1); B = load 'b' AS (b0, b1); C = cogroup A by a0, B by b0; Relevant input columns for C.$0 is A.a0, B.b0. Relevant input columns for C.$1 is A.*. Relevant input columns for C.$2 is B.*. eg4: A = load 'a' AS (a0, a1, a2); B = foreach A generate a1, a0+a2; Relevant input columns for B.$0 is A.a1. Relevant input columns for B.$1 is A.a0 and A.a2. eg5: A = load 'a' AS (a0, a1, a2); B = foreach A generate a1, *; Relevant input columns for B.$0 is A.a1. Relevant input columns for B.$1 is A.a0. Relevant input columns for B.$2 is A.a1. Relevant input columns for B.$3 is A.a2.

Specified by:

getRelevantInputs in class RelationalOperator

Parameters:

output - output index. Only LOSplit have output other than 0 currently

column - output column

Returns:

List of relevant input columns. null if Pig cannot determine relevant inputs or any error occurs

Throws:

FrontendException