MRAsyncDiskService (Hadoop 0.20.2-cdh3u1 API)
Date: 2011-07-18

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.hadoop.util
Class MRAsyncDiskService

    java.lang.Object
      org.apache.hadoop.util.MRAsyncDiskService

    public class MRAsyncDiskService

extends Object

This class is a container of multiple thread pools, each for a volume, so that we can schedule async disk operations easily. Examples of async disk operations are deletion of files. We can move the files to a "toBeDeleted" folder before asychronously deleting it, to make sure the caller can run it faster. Users should not write files into the "toBeDeleted" folder, otherwise the files can be gone any time we restart the MRAsyncDiskService. This class also contains all operations that will be performed by the thread pools.

Field Summary
`static org.apache.commons.logging.Log` | `LOG`
`static String` | `TOBEDELETED`

Constructor Summary
`MRAsyncDiskService(FileSystem localFileSystem, String[] nonCanonicalVols)` Create a AsyncDiskServices with a set of volumes (specified by their root directories).
`MRAsyncDiskService(JobConf conf)` Initialize MRAsyncDiskService based on conf.

Method Summary
`boolean` | `awaitTermination(long milliseconds)` Wait for the termination of the thread pools.
`void` | `cleanupAllVolumes()` Move all files/directories inside volume into TOBEDELETED, and then delete them.
`boolean` | `moveAndDeleteAbsolutePath(String absolutePathName)` Move the path name to a temporary location and then delete it.
`boolean` | `moveAndDeleteFromEachVolume(String pathName)` Move the path name on each volume to a temporary location and then delete them.
`boolean` | `moveAndDeleteRelativePath(String volume, String pathName)` Move the path name on one volume to a temporary location and then delete them.
`void` | `shutdown()` Gracefully start the shut down of all ThreadPools.
`List<Runnable>` | `shutdownNow()` Shut down all ThreadPools immediately.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### LOG

    public static final org.apache.commons.logging.Log LOG

### TOBEDELETED

    public static final String TOBEDELETED

See Also:
  Constant Field Values

Constructor Detail

### MRAsyncDiskService

    public MRAsyncDiskService(FileSystem localFileSystem,
                              String[] nonCanonicalVols)
                       throws IOException

  Create a AsyncDiskServices with a set of volumes (specified by their root directories). The AsyncDiskServices uses one ThreadPool per volume to do the async disk operations.

  Parameters:
    `localFileSystem` - The localFileSystem used for deletions.
    `nonCanonicalVols` - The roots of the file system volumes, which may be absolte paths, or paths relative to the ${user.dir} system property ("cwd").
  Throws:
    `IOException`

### MRAsyncDiskService

    public MRAsyncDiskService(JobConf conf)
                       throws IOException

  Initialize MRAsyncDiskService based on conf.

  Parameters:
    `conf` - local file system and local dirs will be read from conf
  Throws:
    `IOException`

Method Detail

### shutdown

    public void shutdown()

  Gracefully start the shut down of all ThreadPools.

### shutdownNow

    public List<Runnable> shutdownNow()

  Shut down all ThreadPools immediately.

### awaitTermination

    public boolean awaitTermination(long milliseconds)
                             throws InterruptedException

  Wait for the termination of the thread pools.

  Parameters:
    `milliseconds` - The number of milliseconds to wait
  Returns:
    true if all thread pools are terminated within time limit
  Throws:
    `InterruptedException`

### moveAndDeleteRelativePath

    public boolean moveAndDeleteRelativePath(String volume,
                                             String pathName)
                                      throws IOException

  Move the path name on one volume to a temporary location and then delete them. This functions returns when the moves are done, but not necessarily all deletions are done. This is usually good enough because applications won't see the path name under the old name anyway after the move.

  Parameters:
    `volume` - The disk volume
    `pathName` - The path name relative to volume root.
  Returns:
    false if the file is not found
  Throws:
    `IOException` - If the move failed

### moveAndDeleteFromEachVolume

    public boolean moveAndDeleteFromEachVolume(String pathName)
                                        throws IOException

  Move the path name on each volume to a temporary location and then delete them. This functions returns when the moves are done, but not necessarily all deletions are done. This is usually good enough because applications won't see the path name under the old name anyway after the move.

  Parameters:
    `pathName` - The path name relative to each volume root
  Returns:
    false If any of the target pathName did not exist, note that the operation is still done on all volumes.
  Throws:
    `IOException` - If any of the move failed

### cleanupAllVolumes

    public void cleanupAllVolumes()
                           throws IOException

  Move all files/directories inside volume into TOBEDELETED, and then delete them. The TOBEDELETED directory itself is ignored.

  Throws:
    `IOException`

### moveAndDeleteAbsolutePath

    public boolean moveAndDeleteAbsolutePath(String absolutePathName)
                                      throws IOException

  Move the path name to a temporary location and then delete it. Note that if there is no volume that contains this path, the path will stay as it is, and the function will return false. This functions returns when the moves are done, but not necessarily all deletions are done. This is usually good enough because applications won't see the path name under the old name anyway after the move.

  Parameters:
    `absolutePathName` - The path name from root "/"
  Returns:
    false if we are unable to move the path name
  Throws:
    `IOException` - If the move failed

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2009 The Apache Software Foundation