|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.hbase.regionserver.wal.HLogSplitter
public class HLogSplitter
This class is responsible for splitting up a bunch of regionserver commit log files that are no longer being written to, into new files, one per region for region to replay on startup. Delete the old log files when finished.
Field Summary | |
---|---|
protected org.apache.hadoop.conf.Configuration |
conf
|
protected org.apache.hadoop.fs.FileSystem |
fs
|
protected org.apache.hadoop.fs.Path |
oldLogDir
|
static String |
RECOVERED_EDITS
Name of file that holds recovered edits written by the wal log splitting code, one per region |
protected org.apache.hadoop.fs.Path |
rootDir
|
protected org.apache.hadoop.fs.Path |
srcDir
|
protected AtomicReference<Throwable> |
thrown
|
Constructor Summary | |
---|---|
HLogSplitter(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.Path srcDir,
org.apache.hadoop.fs.Path oldLogDir,
org.apache.hadoop.fs.FileSystem fs)
|
Method Summary | |
---|---|
static HLogSplitter |
createLogSplitter(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.Path srcDir,
org.apache.hadoop.fs.Path oldLogDir,
org.apache.hadoop.fs.FileSystem fs)
Create a new HLogSplitter using the given Configuration and the
hbase.hlog.splitter.impl property to derived the instance
class to use. |
protected HLog.Writer |
createWriter(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path logfile,
org.apache.hadoop.conf.Configuration conf)
Create a new HLog.Writer for writing log splits. |
protected HLog.Reader |
getReader(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.FileStatus file,
org.apache.hadoop.conf.Configuration conf,
boolean skipErrors)
Create a new HLog.Reader for reading logs to split. |
protected HLog.Reader |
getReader(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path curLogFile,
org.apache.hadoop.conf.Configuration conf)
Create a new HLog.Reader for reading logs to split. |
long |
getSize()
|
long |
getTime()
|
static void |
moveRecoveredEditsFromTemp(String tmpname,
org.apache.hadoop.fs.Path rootdir,
org.apache.hadoop.fs.Path oldLogDir,
String logfile,
org.apache.hadoop.conf.Configuration conf)
|
static void |
moveRecoveredEditsFromTemp(String tmpname,
String logfile,
org.apache.hadoop.conf.Configuration conf)
Completes the work done by splitLogFileToTemp by moving the recovered.edits from the staging area to the respective region server's directories. |
List<org.apache.hadoop.fs.Path> |
splitLog()
Split up a bunch of regionserver commit log files that are no longer being written to, into new files, one per region for region to replay on startup. |
boolean |
splitLogFileToTemp(org.apache.hadoop.fs.FileStatus logfile,
String tmpname,
CancelableProgressable reporter)
|
static boolean |
splitLogFileToTemp(org.apache.hadoop.fs.Path rootDir,
String tmpname,
org.apache.hadoop.fs.FileStatus logfile,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration conf,
CancelableProgressable reporter)
Splits a HLog file into a temporary staging area. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final String RECOVERED_EDITS
protected final org.apache.hadoop.fs.Path rootDir
protected final org.apache.hadoop.fs.Path srcDir
protected final org.apache.hadoop.fs.Path oldLogDir
protected final org.apache.hadoop.fs.FileSystem fs
protected final org.apache.hadoop.conf.Configuration conf
protected AtomicReference<Throwable> thrown
Constructor Detail |
---|
public HLogSplitter(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path srcDir, org.apache.hadoop.fs.Path oldLogDir, org.apache.hadoop.fs.FileSystem fs)
Method Detail |
---|
public static HLogSplitter createLogSplitter(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path srcDir, org.apache.hadoop.fs.Path oldLogDir, org.apache.hadoop.fs.FileSystem fs)
Configuration
and the
hbase.hlog.splitter.impl
property to derived the instance
class to use.
conf
- rootDir
- hbase directorysrcDir
- logs directoryoldLogDir
- directory where processed logs are archived tofs
- FileSystem
public List<org.apache.hadoop.fs.Path> splitLog() throws IOException
IOException
- will throw if corrupted hlogs aren't toleratedpublic long getTime()
public long getSize()
public static boolean splitLogFileToTemp(org.apache.hadoop.fs.Path rootDir, String tmpname, org.apache.hadoop.fs.FileStatus logfile, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, CancelableProgressable reporter) throws IOException
If the log file has N regions then N recovered.edits files will be produced. There is no buffering in this code. Instead it relies on the buffering in the SequenceFileWriter.
rootDir
- tmpname
- logfile
- fs
- conf
- reporter
-
IOException
public boolean splitLogFileToTemp(org.apache.hadoop.fs.FileStatus logfile, String tmpname, CancelableProgressable reporter) throws IOException
IOException
public static void moveRecoveredEditsFromTemp(String tmpname, String logfile, org.apache.hadoop.conf.Configuration conf) throws IOException
It is invoked by SplitLogManager once it knows that one of the SplitLogWorkers have completed the splitLogFileToTemp() part. If the master crashes then this function might get called multiple times.
tmpname
- conf
-
IOException
public static void moveRecoveredEditsFromTemp(String tmpname, org.apache.hadoop.fs.Path rootdir, org.apache.hadoop.fs.Path oldLogDir, String logfile, org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
protected HLog.Reader getReader(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus file, org.apache.hadoop.conf.Configuration conf, boolean skipErrors) throws IOException, org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.CorruptedLogFileException
HLog.Reader
for reading logs to split.
fs
- file
- conf
-
IOException
CorruptedLogFile
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.CorruptedLogFileException
protected HLog.Writer createWriter(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path logfile, org.apache.hadoop.conf.Configuration conf) throws IOException
HLog.Writer
for writing log splits.
IOException
protected HLog.Reader getReader(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path curLogFile, org.apache.hadoop.conf.Configuration conf) throws IOException
HLog.Reader
for reading logs to split.
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |