|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder
public class FastDiffDeltaEncoder
Encoder similar to DiffKeyDeltaEncoder
but supposedly faster.
Compress using:
- store size of common prefix
- save column family once in the first KeyValue
- use integer compression for key, value and prefix (7-bit encoding)
- use bits to avoid duplication key length, value length
and type if it same as previous
- store in 3 bits length of prefix timestamp
with previous KeyValue's timestamp
- one bit which allow to omit value if it is the same
Format:
- 1 byte: flag
- 1-5 bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag)
- 1-5 bytes: value length (only if FLAG_SAME_VALUE_LENGTH is not set in flag)
- 1-5 bytes: prefix length
- ... bytes: rest of the row (if prefix length is small enough)
- ... bytes: qualifier (or suffix depending on prefix length)
- 1-8 bytes: timestamp suffix
- 1 byte: type (only if FLAG_SAME_TYPE is not set in the flag)
- ... bytes: value (only if FLAG_SAME_VALUE is not set in the flag)
Nested Class Summary | |
---|---|
protected static class |
BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState>
|
protected static class |
BufferedDataBlockEncoder.SeekerState
|
protected static class |
FastDiffDeltaEncoder.FastDiffSeekerState
|
Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.encoding.DataBlockEncoder |
---|
DataBlockEncoder.EncodedSeeker |
Constructor Summary | |
---|---|
FastDiffDeltaEncoder()
|
Method Summary | |
---|---|
protected void |
afterDecodingKeyValue(DataInputStream source,
ByteBuffer dest,
boolean includesMemstoreTS)
|
protected void |
afterEncodingKeyValue(ByteBuffer in,
DataOutputStream out,
boolean includesMemstoreTS)
|
void |
compressKeyValues(DataOutputStream out,
ByteBuffer in,
boolean includesMemstoreTS)
Compress KeyValues and write them to output buffer. |
DataBlockEncoder.EncodedSeeker |
createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator,
boolean includesMemstoreTS)
Create a HFileBlock seeker which find KeyValues within a block. |
ByteBuffer |
getFirstKeyInBlock(ByteBuffer block)
Return first key in block. |
String |
toString()
|
ByteBuffer |
uncompressKeyValues(DataInputStream source,
boolean includesMemstoreTS)
Uncompress. |
ByteBuffer |
uncompressKeyValues(DataInputStream source,
int allocHeaderLength,
int skipLastBytes,
boolean includesMemstoreTS)
Uncompress. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Constructor Detail |
---|
public FastDiffDeltaEncoder()
Method Detail |
---|
public void compressKeyValues(DataOutputStream out, ByteBuffer in, boolean includesMemstoreTS) throws IOException
DataBlockEncoder
out
- Where to write compressed data.in
- Source of KeyValue for compression.includesMemstoreTS
- true if including memstore timestamp after every
key-value pair
IOException
- If there is an error writing to output stream.public ByteBuffer uncompressKeyValues(DataInputStream source, int allocHeaderLength, int skipLastBytes, boolean includesMemstoreTS) throws IOException
DataBlockEncoder
source
- Compressed stream of KeyValues.allocHeaderLength
- allocate this many bytes for the header.skipLastBytes
- Do not copy n last bytes.includesMemstoreTS
- true if including memstore timestamp after every
key-value pair
IOException
- If there is an error in source.public ByteBuffer getFirstKeyInBlock(ByteBuffer block)
DataBlockEncoder
block
- encoded block we want index, the position will not change
public String toString()
toString
in class Object
public DataBlockEncoder.EncodedSeeker createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator, boolean includesMemstoreTS)
DataBlockEncoder
comparator
- what kind of comparison should be usedincludesMemstoreTS
- true if including memstore timestamp after every
key-value pair
public ByteBuffer uncompressKeyValues(DataInputStream source, boolean includesMemstoreTS) throws IOException
DataBlockEncoder
uncompressKeyValues
in interface DataBlockEncoder
source
- Compressed stream of KeyValues.includesMemstoreTS
- true if including memstore timestamp after every
key-value pair
IOException
- If there is an error in source.protected final void afterEncodingKeyValue(ByteBuffer in, DataOutputStream out, boolean includesMemstoreTS)
protected final void afterDecodingKeyValue(DataInputStream source, ByteBuffer dest, boolean includesMemstoreTS)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |