com.cloudera.sqoop.io
Class SplittingOutputStream

java.lang.Object
  extended by java.io.OutputStream
      extended by com.cloudera.sqoop.io.SplittingOutputStream
All Implemented Interfaces:
java.io.Closeable, java.io.Flushable

public class SplittingOutputStream
extends java.io.OutputStream

An output stream that writes to an underlying filesystem, opening a new file after a specified number of bytes have been written to the current one.


Field Summary
static org.apache.commons.logging.Log LOG
           
 
Constructor Summary
SplittingOutputStream(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path destDir, java.lang.String filePrefix, long cutoff, boolean doGzip)
          Create a new SplittingOutputStream.
 
Method Summary
 void allowSplit()
          Defines a point in the stream when it is acceptable to split to a new file; e.g., the end of a record.
 void close()
           
 void flush()
           
 boolean wouldSplit()
           
 void write(byte[] b)
           
 void write(byte[] b, int off, int len)
           
 void write(int b)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Constructor Detail

SplittingOutputStream

public SplittingOutputStream(org.apache.hadoop.conf.Configuration conf,
                             org.apache.hadoop.fs.Path destDir,
                             java.lang.String filePrefix,
                             long cutoff,
                             boolean doGzip)
                      throws java.io.IOException
Create a new SplittingOutputStream.

Parameters:
conf - the Configuration to use to interface with HDFS
destDir - the directory where the files will go (should already exist).
filePrefix - the first part of the filename, which will be appended by a number. This file will be placed inside destDir.
cutoff - the approximate number of bytes to use per file
doGzip - if true, then output files will be gzipped and have a .gz suffix.
Throws:
java.io.IOException
Method Detail

wouldSplit

public boolean wouldSplit()
Returns:
true if allowSplit() would actually cause a split.

allowSplit

public void allowSplit()
                throws java.io.IOException
Defines a point in the stream when it is acceptable to split to a new file; e.g., the end of a record.

Throws:
java.io.IOException

close

public void close()
           throws java.io.IOException
Specified by:
close in interface java.io.Closeable
Overrides:
close in class java.io.OutputStream
Throws:
java.io.IOException

flush

public void flush()
           throws java.io.IOException
Specified by:
flush in interface java.io.Flushable
Overrides:
flush in class java.io.OutputStream
Throws:
java.io.IOException

write

public void write(byte[] b)
           throws java.io.IOException
Overrides:
write in class java.io.OutputStream
Throws:
java.io.IOException

write

public void write(byte[] b,
                  int off,
                  int len)
           throws java.io.IOException
Overrides:
write in class java.io.OutputStream
Throws:
java.io.IOException

write

public void write(int b)
           throws java.io.IOException
Specified by:
write in class java.io.OutputStream
Throws:
java.io.IOException


Copyright © 2010 Cloudera, Inc.