|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
public interface HadoopShims
In order to be compatible with multiple versions of Hadoop, all parts of the Hadoop interface that are not cross-version compatible are encapsulated in an implementation of this class. Users should use the ShimLoader class as a factory to obtain an implementation of HadoopShims corresponding to the version of Hadoop currently on the classpath.
Nested Class Summary | |
---|---|
static interface |
HadoopShims.CombineFileInputFormatShim<K,V>
CombineFileInputFormatShim. |
static interface |
HadoopShims.InputSplitShim
InputSplitShim. |
static interface |
HadoopShims.MiniDFSShim
Shim around the functions in MiniDFSCluster that Hive uses. |
Field Summary | |
---|---|
static org.apache.commons.logging.Log |
LOG
|
Method Summary | |
---|---|
int |
compareText(org.apache.hadoop.io.Text a,
org.apache.hadoop.io.Text b)
We define this function here to make the code compatible between hadoop 0.17 and hadoop 0.20. |
int |
createHadoopArchive(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path parentDir,
org.apache.hadoop.fs.Path destDir,
String archiveName)
|
boolean |
fileSystemDeleteOnExit(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
Calls fs.deleteOnExit(path) if such a function exists. |
long |
getAccessTime(org.apache.hadoop.fs.FileStatus file)
return the last access time of the given file. |
HadoopShims.CombineFileInputFormatShim |
getCombineFileInputFormat()
|
String |
getInputFormatClassName()
|
HadoopShims.MiniDFSShim |
getMiniDfs(org.apache.hadoop.conf.Configuration conf,
int numDataNodes,
boolean format,
String[] racks)
Returns a shim to wrap MiniDFSCluster. |
String[] |
getTaskJobIDs(org.apache.hadoop.mapred.TaskCompletionEvent t)
getTaskJobIDs returns an array of String with two elements. |
String |
getTokenStrForm(String tokenSignature)
Get the string form of the token given a token signature. |
org.apache.hadoop.security.UserGroupInformation |
getUGIForConf(org.apache.hadoop.conf.Configuration conf)
Get the UGI that the given job configuration will run as. |
void |
inputFormatValidateInput(org.apache.hadoop.mapred.InputFormat fmt,
org.apache.hadoop.mapred.JobConf conf)
Calls fmt.validateInput(conf) if such a function exists. |
boolean |
isJobPreparing(org.apache.hadoop.mapred.RunningJob job)
Return true if the job has not switched to RUNNING state yet and is still in PREP state |
boolean |
isSecureShimImpl()
Return true if the Shim is based on Hadoop Security APIs. |
void |
setFloatConf(org.apache.hadoop.conf.Configuration conf,
String varName,
float val)
Wrapper for Configuration.setFloat, which was not introduced until 0.20. |
void |
setNullOutputFormat(org.apache.hadoop.mapred.JobConf conf)
Hive uses side effect files exclusively for it's output. |
void |
setTmpFiles(String prop,
String files)
If JobClient.getCommandLineConfig exists, sets the given property/value pair in that Configuration object. |
boolean |
usesJobShell()
Return true if the current version of Hadoop uses the JobShell for command line interpretation. |
Field Detail |
---|
static final org.apache.commons.logging.Log LOG
Method Detail |
---|
boolean usesJobShell()
boolean isJobPreparing(org.apache.hadoop.mapred.RunningJob job) throws IOException
IOException
boolean fileSystemDeleteOnExit(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) throws IOException
IOException
void inputFormatValidateInput(org.apache.hadoop.mapred.InputFormat fmt, org.apache.hadoop.mapred.JobConf conf) throws IOException
IOException
void setTmpFiles(String prop, String files)
long getAccessTime(org.apache.hadoop.fs.FileStatus file)
file
-
HadoopShims.MiniDFSShim getMiniDfs(org.apache.hadoop.conf.Configuration conf, int numDataNodes, boolean format, String[] racks) throws IOException
IOException
int compareText(org.apache.hadoop.io.Text a, org.apache.hadoop.io.Text b)
HadoopShims.CombineFileInputFormatShim getCombineFileInputFormat()
String getInputFormatClassName()
void setFloatConf(org.apache.hadoop.conf.Configuration conf, String varName, float val)
String[] getTaskJobIDs(org.apache.hadoop.mapred.TaskCompletionEvent t)
int createHadoopArchive(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path parentDir, org.apache.hadoop.fs.Path destDir, String archiveName) throws Exception
Exception
void setNullOutputFormat(org.apache.hadoop.mapred.JobConf conf)
org.apache.hadoop.security.UserGroupInformation getUGIForConf(org.apache.hadoop.conf.Configuration conf) throws LoginException, IOException
LoginException
IOException
boolean isSecureShimImpl()
String getTokenStrForm(String tokenSignature) throws IOException
tokenSignature
-
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |