org.apache.hadoop.hive.ql.io
Class SymlinkTextInputFormat
java.lang.Object
org.apache.hadoop.hive.ql.io.SymbolicInputFormat
org.apache.hadoop.hive.ql.io.SymlinkTextInputFormat
- All Implemented Interfaces:
- ContentSummaryInputFormat, ReworkMapredInputFormat
public class SymlinkTextInputFormat
- extends SymbolicInputFormat
- implements ContentSummaryInputFormat, ReworkMapredInputFormat
Symlink file is a text file which contains a list of filename / dirname.
This input method reads symlink files from specified job input paths and
takes the files / directories specified in those symlink files as
actual map-reduce input. The target input data should be in TextInputFormat.
Nested Class Summary |
static class |
SymlinkTextInputFormat.SymlinkTextInputSplit
This input split wraps the FileSplit generated from
TextInputFormat.getSplits(), while setting the original link file path
as job input path. |
Method Summary |
void |
configure(JobConf job)
|
ContentSummary |
getContentSummary(Path p,
JobConf job)
|
|
getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
InputSplit[] |
getSplits(JobConf job,
int numSplits)
Parses all target paths from job input directory which contains symlink
files, and splits the target data using TextInputFormat. |
void |
validateInput(JobConf job)
For backward compatibility with hadoop 0.17. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
SymlinkTextInputFormat
public SymlinkTextInputFormat()
getRecordReader
public getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
throws IOException
- Throws:
IOException
getSplits
public InputSplit[] getSplits(JobConf job,
int numSplits)
throws IOException
- Parses all target paths from job input directory which contains symlink
files, and splits the target data using TextInputFormat.
- Throws:
IOException
configure
public void configure(JobConf job)
validateInput
public void validateInput(JobConf job)
throws IOException
- For backward compatibility with hadoop 0.17.
- Throws:
IOException
getContentSummary
public ContentSummary getContentSummary(Path p,
JobConf job)
throws IOException
- Specified by:
getContentSummary
in interface ContentSummaryInputFormat
- Throws:
IOException
Copyright © 2012 The Apache Software Foundation