Configuration (Hadoop 0.20.2-CDH3B4 API)
Date: 2011-02-21

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.hadoop.conf
Class Configuration

    java.lang.Object
      org.apache.hadoop.conf.Configuration

All Implemented Interfaces:
  Iterable<Map.Entry<String,String>>, Writable

Direct Known Subclasses:
  JobConf

    public class Configuration

extends Object
implements Iterable<Map.Entry<String,String>>, Writable

Provides access to configuration parameters.

#### Resources

Configurations are specified by resources. A resource contains a set of name/value pairs as XML data. Each resource is named by either a `String` or by a `Path`. If named by a `String`, then the classpath is examined for a file with that name. If named by a `Path`, then the local filesystem is examined directly, without referring to the classpath.

Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath:

1. `core-default.xml` : Read-only defaults for hadoop.
2. `core-site.xml`: Site-specific configuration for a given hadoop installation.

Applications may add additional resources, which are loaded subsequent to these resources in the order they are added.

#### Final Parameters

Configuration parameters may be declared final. Once a resource declares a value final, no subsequently-loaded resource can alter that value. For example, one might define a final parameter with: ``

      <property>
        <name>dfs.client.buffer.dir</name>
        <value>/tmp/hadoop/dfs/client</value>
        <final>true</final>
      </property>

Administrators typically define parameters as final in `core-site.xml` for values that user applications may not alter.

#### Variable Expansion

Value strings are first processed for variable expansion. The available properties are:

1. Other properties defined in this Configuration; and, if a name is undefined here,
2. Properties in `System.getProperties()`.

For example, if a configuration resource contains the following property definitions: ``

      <property>
        <name>basedir</name>
        <value>/user/${user.name}</value>
      </property>

      <property>
        <name>tempdir</name>
        <value>${basedir}/tmp</value>
      </property>

When `conf.get("tempdir")` is called, then `${basedir}` will be resolved to another property in this Configuration, while `${user.name}` would then ordinarily be resolved to the value of the System property with that name.

Nested Class Summary
`static class` | `Configuration.IntegerRanges` A class that represents a set of positive integer ranges.

Constructor Summary
`Configuration()` A new configuration.
`Configuration(boolean loadDefaults)` A new configuration where the behavior of reading from the default resources can be turned off.
`Configuration(Configuration other)` A new configuration with the same settings cloned from another.

Method Summary
`static void` | `addDefaultResource(String name)` Add a default resource.
`void` | `addResource(InputStream in)` Add a configuration resource.
`void` | `addResource(Path file)` Add a configuration resource.
`void` | `addResource(String name)` Add a configuration resource.
`void` | `addResource(URL url)` Add a configuration resource.
`void` | `clear()` Clears all keys from the configuration.
`static void` | `dumpConfiguration(Configuration config, Writer out)` Writes out all the parameters and their properties (final and resource) to the given `Writer` The format of the output would be { "properties" : [ {key1,value1,key1.isFinal,key1.resource}, {key2,value2, key2.isFinal,key2.resource}...
`String` | `get(String name)` Get the value of the `name` property, `null` if no such property exists.
`String` | `get(String name, String defaultValue)` Get the value of the `name` property.
`boolean` | `getBoolean(String name, boolean defaultValue)` Get the value of the `name` property as a `boolean`.
`Class<?>` | `getClass(String name, Class<?> defaultValue)` Get the value of the `name` property as a `Class`.
`
`<U> Class<? extends U>`
` | `getClass(String name, Class<? extends U> defaultValue, Class<U> xface)` Get the value of the `name` property as a `Class` implementing the interface specified by `xface`.
`Class<?>` | `getClassByName(String name)` Load a class by name.
`Class<?>[]` | `getClasses(String name, Class<?>... defaultValue)` Get the value of the `name` property as an array of `Class`.
`ClassLoader` | `getClassLoader()` Get the `ClassLoader` for this job.
`InputStream` | `getConfResourceAsInputStream(String name)` Get an input stream attached to the configuration resource with the given `name`.
`Reader` | `getConfResourceAsReader(String name)` Get a `Reader` attached to the configuration resource with the given `name`.
`
`<T extends Enum<T>> T`
` | `getEnum(String name, T defaultValue)` Return value matching this enumerated type.
`File` | `getFile(String dirsProp, String path)` Get a local file name under a directory named in dirsProp with the given path.
`float` | `getFloat(String name, float defaultValue)` Get the value of the `name` property as a `float`.
`
`<U> List<U>`
` | `getInstances(String name, Class<U> xface)` Get the value of the `name` property as a `List` of objects implementing the interface specified by `xface`.
`int` | `getInt(String name, int defaultValue)` Get the value of the `name` property as an `int`.
`Path` | `getLocalPath(String dirsProp, String path)` Get a local file under a directory named by dirsProp with the given path.
`long` | `getLong(String name, long defaultValue)` Get the value of the `name` property as a `long`.
`Pattern` | `getPattern(String name, Pattern defaultValue)` Get the value of the `name` property as a Pattern.
`Configuration.IntegerRanges` | `getRange(String name, String defaultValue)` Parse the given attribute as a set of integer ranges
`String` | `getRaw(String name)` Get the value of the `name` property, without doing variable expansion.
`URL` | `getResource(String name)` Get the `URL` for the named resource.
`Collection<String>` | `getStringCollection(String name)` Get the comma delimited values of the `name` property as a collection of `String`s.
`String[]` | `getStrings(String name)` Get the comma delimited values of the `name` property as an array of `String`s.
`String[]` | `getStrings(String name, String... defaultValue)` Get the comma delimited values of the `name` property as an array of `String`s.
`String` | `getTrimmed(String name)` Get the value of the `name` property as a trimmed `String`, `null` if no such property exists.
`Collection<String>` | `getTrimmedStringCollection(String name)` Get the comma delimited values of the `name` property as a collection of `String`s, trimmed of the leading and trailing whitespace.
`String[]` | `getTrimmedStrings(String name)` Get the comma delimited values of the `name` property as an array of `String`s, trimmed of the leading and trailing whitespace.
`String[]` | `getTrimmedStrings(String name, String... defaultValue)` Get the comma delimited values of the `name` property as an array of `String`s, trimmed of the leading and trailing whitespace.
`Map<String,String>` | `getValByRegex(String regex)` get keys matching the the regex
`Iterator<Map.Entry<String,String>>` | `iterator()` Get an `Iterator` to go through the list of `String` key-value pairs in the configuration.
`static void` | `main(String[] args)` For debugging.
`void` | `readFields(DataInput in)` Deserialize the fields of this object from `in`.
`void` | `reloadConfiguration()` Reload configuration from previously added resources.
`void` | `set(String name, String value)` Set the `value` of the `name` property.
`void` | `setBoolean(String name, boolean value)` Set the value of the `name` property to a `boolean`.
`void` | `setBooleanIfUnset(String name, boolean value)` Set the given property, if it is currently unset.
`void` | `setClass(String name, Class<?> theClass, Class<?> xface)` Set the value of the `name` property to the name of a `theClass` implementing the given interface `xface`.
`void` | `setClassLoader(ClassLoader classLoader)` Set the class loader that will be used to load the various objects.
`
`<T extends Enum<T>> void`
` | `setEnum(String name, T value)` Set the value of the `name` property to the given type.
`void` | `setFloat(String name, float value)` Set the value of the `name` property to a `float`.
`void` | `setIfUnset(String name, String value)` Sets a property if it is currently unset.
`void` | `setInt(String name, int value)` Set the value of the `name` property to an `int`.
`void` | `setLong(String name, long value)` Set the value of the `name` property to a `long`.
`void` | `setPattern(String name, Pattern pattern)` Set the given property to `Pattern`.
`void` | `setQuietMode(boolean quietmode)` Set the quietness-mode.
`void` | `setStrings(String name, String... values)` Set the array of string values for the `name` property as as comma delimited values.
`int` | `size()` Return the number of keys in the configuration.
`String` | `toString()`
`void` | `write(DataOutput out)` Serialize the fields of this object to `out`.
`void` | `writeXml(OutputStream out)` Write out the non-default properties in this configuration to the given `OutputStream`.
`void` | `writeXml(Writer out)` Write out the non-default properties in this configuration to the given `Writer`.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Constructor Detail

### Configuration

    public Configuration()

  A new configuration.

### Configuration

    public Configuration(boolean loadDefaults)

  A new configuration where the behavior of reading from the default resources can be turned off. If the parameter `loadDefaults` is false, the new instance will not load resources from the default files.

  Parameters:
    `loadDefaults` - specifies whether to load from the default files

### Configuration

    public Configuration(Configuration other)

  A new configuration with the same settings cloned from another.

  Parameters:
    `other` - the configuration from which to clone settings.

Method Detail

### addDefaultResource

    public static void addDefaultResource(String name)

  Add a default resource. Resources are loaded in the order of the resources added.

  Parameters:
    `name` - file name. File should be present in the classpath.

### addResource

    public void addResource(String name)

  Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

  Parameters:
    `name` - resource to be added, the classpath is examined for a file with that name.

### addResource

    public void addResource(URL url)

  Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

  Parameters:
    `url` - url of the resource to be added, the local filesystem is examined directly to find the resource, without referring to the classpath.

### addResource

    public void addResource(Path file)

  Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

  Parameters:
    `file` - file-path of resource to be added, the local filesystem is examined directly to find the resource, without referring to the classpath.

### addResource

    public void addResource(InputStream in)

  Add a configuration resource. The properties of this resource will override properties of previously added resources, unless they were marked final.

  Parameters:
    `in` - InputStream to deserialize the object from.

### reloadConfiguration

    public void reloadConfiguration()

  Reload configuration from previously added resources. This method will clear all the configuration read from the added resources, and final parameters. This will make the resources to be read again before accessing the values. Values that are added via set methods will overlay values read from the resources.

### get

    public String get(String name)

  Get the value of the `name` property, `null` if no such property exists. Values are processed for variable expansion before being returned.

  Parameters:
    `name` - the property name.
  Returns:
    the value of the `name` property, or null if no such property exists.

### getTrimmed

    public String getTrimmed(String name)

  Get the value of the `name` property as a trimmed `String`, `null` if no such property exists. If the key is deprecated, it returns the value of the first key which replaces the deprecated key and is not null Values are processed for variable expansion before being returned.

  Parameters:
    `name` - the property name.
  Returns:
    the value of the `name` or its replacing property, or null if no such property exists.

### getRaw

    public String getRaw(String name)

  Get the value of the `name` property, without doing variable expansion.

  Parameters:
    `name` - the property name.
  Returns:
    the value of the `name` property, or null if no such property exists.

### set

    public void set(String name,
                    String value)

  Set the `value` of the `name` property.

  Parameters:
    `name` - property name.
    `value` - property value.

### setIfUnset

    public void setIfUnset(String name,
                           String value)

  Sets a property if it is currently unset.

  Parameters:
    `name` - the property name
    `value` - the new value

### get

    public String get(String name,
                      String defaultValue)

  Get the value of the `name` property. If no such property exists, then `defaultValue` is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - default value.
  Returns:
    property value, or `defaultValue` if the property doesn't exist.

### getInt

    public int getInt(String name,
                      int defaultValue)

  Get the value of the `name` property as an `int`. If no such property exists, or if the specified value is not a valid `int`, then `defaultValue` is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - default value.
  Returns:
    property value as an `int`, or `defaultValue`.

### setInt

    public void setInt(String name,
                       int value)

  Set the value of the `name` property to an `int`.

  Parameters:
    `name` - property name.
    `value` - `int` value of the property.

### getLong

    public long getLong(String name,
                        long defaultValue)

  Get the value of the `name` property as a `long`. If no such property is specified, or if the specified value is not a valid `long`, then `defaultValue` is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - default value.
  Returns:
    property value as a `long`, or `defaultValue`.

### setLong

    public void setLong(String name,
                        long value)

  Set the value of the `name` property to a `long`.

  Parameters:
    `name` - property name.
    `value` - `long` value of the property.

### getFloat

    public float getFloat(String name,
                          float defaultValue)

  Get the value of the `name` property as a `float`. If no such property is specified, or if the specified value is not a valid `float`, then `defaultValue` is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - default value.
  Returns:
    property value as a `float`, or `defaultValue`.

### setFloat

    public void setFloat(String name,
                         float value)

  Set the value of the `name` property to a `float`.

  Parameters:
    `name` - property name.
    `value` - property value.

### getBoolean

    public boolean getBoolean(String name,
                              boolean defaultValue)

  Get the value of the `name` property as a `boolean`. If no such property is specified, or if the specified value is not a valid `boolean`, then `defaultValue` is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - default value.
  Returns:
    property value as a `boolean`, or `defaultValue`.

### setBoolean

    public void setBoolean(String name,
                           boolean value)

  Set the value of the `name` property to a `boolean`.

  Parameters:
    `name` - property name.
    `value` - `boolean` value of the property.

### setBooleanIfUnset

    public void setBooleanIfUnset(String name,
                                  boolean value)

  Set the given property, if it is currently unset.

  Parameters:
    `name` - property name
    `value` - new value

### getPattern

    public Pattern getPattern(String name,
                              Pattern defaultValue)

  Get the value of the `name` property as a Pattern. If no such property is specified, or if the specified value is not a valid `Pattern`, then `DefaultValue` is returned.

  Parameters:
    `name` - property name
    `defaultValue` - default value
  Returns:
    property value as a compiled Pattern, or defaultValue

### setPattern

    public void setPattern(String name,
                           Pattern pattern)

  Set the given property to `Pattern`. If the pattern is passed as null, sets the empty pattern which results in further calls to getPattern(...) returning the default value.

  Parameters:
    `name` - property name
    `pattern` - new value

### setEnum

    public <T extends Enum<T>> void setEnum(String name,
                                            T value)

  Set the value of the `name` property to the given type. This is equivalent to `set(<name>, value.toString())`.

  Parameters:
    `name` - property name
    `value` - new value

### getEnum

    public <T extends Enum<T>> T getEnum(String name,
                                         T defaultValue)

  Return value matching this enumerated type.

  Parameters:
    `name` - Property name
    `defaultValue` - Value returned if no mapping exists
  Throws:
    `IllegalArgumentException` - If mapping is illegal for the type provided

### getRange

    public Configuration.IntegerRanges getRange(String name,
                                                String defaultValue)

  Parse the given attribute as a set of integer ranges

  Parameters:
    `name` - the attribute name
    `defaultValue` - the default value if it is not set
  Returns:
    a new set of ranges from the configured value

### getStringCollection

    public Collection<String> getStringCollection(String name)

  Get the comma delimited values of the `name` property as a collection of `String`s. If no such property is specified then empty collection is returned.

  This is an optimized version of `getStrings(String)`

  Parameters:
    `name` - property name.
  Returns:
    property value as a collection of `String`s.

### getStrings

    public String[] getStrings(String name)

  Get the comma delimited values of the `name` property as an array of `String`s. If no such property is specified then `null` is returned.

  Parameters:
    `name` - property name.
  Returns:
    property value as an array of `String`s, or `null`.

### getStrings

    public String[] getStrings(String name,
                               String... defaultValue)

  Get the comma delimited values of the `name` property as an array of `String`s. If no such property is specified then default value is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - The default value
  Returns:
    property value as an array of `String`s, or default value.

### getTrimmedStringCollection

    public Collection<String> getTrimmedStringCollection(String name)

  Get the comma delimited values of the `name` property as a collection of `String`s, trimmed of the leading and trailing whitespace. If no such property is specified then empty `Collection` is returned.

  Parameters:
    `name` - property name.
  Returns:
    property value as a collection of `String`s, or empty `Collection`

### getTrimmedStrings

    public String[] getTrimmedStrings(String name)

  Get the comma delimited values of the `name` property as an array of `String`s, trimmed of the leading and trailing whitespace. If no such property is specified then an empty array is returned.

  Parameters:
    `name` - property name.
  Returns:
    property value as an array of trimmed `String`s, or empty array.

### getTrimmedStrings

    public String[] getTrimmedStrings(String name,
                                      String... defaultValue)

  Get the comma delimited values of the `name` property as an array of `String`s, trimmed of the leading and trailing whitespace. If no such property is specified then default value is returned.

  Parameters:
    `name` - property name.
    `defaultValue` - The default value
  Returns:
    property value as an array of trimmed `String`s, or default value.

### setStrings

    public void setStrings(String name,
                           String... values)

  Set the array of string values for the `name` property as as comma delimited values.

  Parameters:
    `name` - property name.
    `values` - The values

### getClassByName

    public Class<?> getClassByName(String name)
                            throws ClassNotFoundException

  Load a class by name.

  Parameters:
    `name` - the class name.
  Returns:
    the class object.
  Throws:
    `ClassNotFoundException` - if the class is not found.

### getClasses

    public Class<?>[] getClasses(String name,
                                 Class<?>... defaultValue)

  Get the value of the `name` property as an array of `Class`. The value of the property specifies a list of comma separated class names. If no such property is specified, then `defaultValue` is returned.

  Parameters:
    `name` - the property name.
    `defaultValue` - default value.
  Returns:
    property value as a `Class[]`, or `defaultValue`.

### getClass

    public Class<?> getClass(String name,
                             Class<?> defaultValue)

  Get the value of the `name` property as a `Class`. If no such property is specified, then `defaultValue` is returned.

  Parameters:
    `name` - the class name.
    `defaultValue` - default value.
  Returns:
    property value as a `Class`, or `defaultValue`.

### getClass

    public <U> Class<? extends U> getClass(String name,
                                           Class<? extends U> defaultValue,
                                           Class<U> xface)

  Get the value of the `name` property as a `Class` implementing the interface specified by `xface`. If no such property is specified, then `defaultValue` is returned. An exception is thrown if the returned class does not implement the named interface.

  Parameters:
    `name` - the class name.
    `defaultValue` - default value.
    `xface` - the interface implemented by the named class.
  Returns:
    property value as a `Class`, or `defaultValue`.

### getInstances

    public <U> List<U> getInstances(String name,
                                    Class<U> xface)

  Get the value of the `name` property as a `List` of objects implementing the interface specified by `xface`. An exception is thrown if any of the classes does not exist, or if it does not implement the named interface.

  Parameters:
    `name` - the property name.
    `xface` - the interface implemented by the classes named by `name`.
  Returns:
    a `List` of objects implementing `xface`.

### setClass

    public void setClass(String name,
                         Class<?> theClass,
                         Class<?> xface)

  Set the value of the `name` property to the name of a `theClass` implementing the given interface `xface`. An exception is thrown if `theClass` does not implement the interface `xface`.

  Parameters:
    `name` - property name.
    `theClass` - property value.
    `xface` - the interface implemented by the named class.

### getLocalPath

    public Path getLocalPath(String dirsProp,
                             String path)
                      throws IOException

  Get a local file under a directory named by dirsProp with the given path. If dirsProp contains multiple directories, then one is chosen based on path's hash code. If the selected directory does not exist, an attempt is made to create it.

  Parameters:
    `dirsProp` - directory in which to locate the file.
    `path` - file-path.
  Returns:
    local file under the directory with the given path.
  Throws:
    `IOException`

### getFile

    public File getFile(String dirsProp,
                        String path)
                 throws IOException

  Get a local file name under a directory named in dirsProp with the given path. If dirsProp contains multiple directories, then one is chosen based on path's hash code. If the selected directory does not exist, an attempt is made to create it.

  Parameters:
    `dirsProp` - directory in which to locate the file.
    `path` - file-path.
  Returns:
    local file under the directory with the given path.
  Throws:
    `IOException`

### getResource

    public URL getResource(String name)

  Get the `URL` for the named resource.

  Parameters:
    `name` - resource name.
  Returns:
    the url for the named resource.

### getConfResourceAsInputStream

    public InputStream getConfResourceAsInputStream(String name)

  Get an input stream attached to the configuration resource with the given `name`.

  Parameters:
    `name` - configuration resource name.
  Returns:
    an input stream attached to the resource.

### getConfResourceAsReader

    public Reader getConfResourceAsReader(String name)

  Get a `Reader` attached to the configuration resource with the given `name`.

  Parameters:
    `name` - configuration resource name.
  Returns:
    a reader attached to the resource.

### size

    public int size()

  Return the number of keys in the configuration.

  Returns:
    number of keys in the configuration.

### clear

    public void clear()

  Clears all keys from the configuration.

### iterator

    public Iterator<Map.Entry<String,String>> iterator()

  Get an `Iterator` to go through the list of `String` key-value pairs in the configuration.

  Specified by:
    `iterator` in interface `Iterable<Map.Entry<String,String>>`
  Returns:
    an iterator over the entries.

### writeXml

    public void writeXml(OutputStream out)
                  throws IOException

  Write out the non-default properties in this configuration to the given `OutputStream`.

  Parameters:
    `out` - the output stream to write to.
  Throws:
    `IOException`

### writeXml

    public void writeXml(Writer out)
                  throws IOException

  Write out the non-default properties in this configuration to the given `Writer`.

  Parameters:
    `out` - the writer to write to.
  Throws:
    `IOException`

### dumpConfiguration

    public static void dumpConfiguration(Configuration config,
                                         Writer out)
                                  throws IOException

  Writes out all the parameters and their properties (final and resource) to the given `Writer` The format of the output would be { "properties" : [ {key1,value1,key1.isFinal,key1.resource}, {key2,value2, key2.isFinal,key2.resource}... ] } It does not output the parameters of the configuration object which is loaded from an input stream.

  Parameters:
    `out` - the Writer to write to
  Throws:
    `IOException`

### getClassLoader

    public ClassLoader getClassLoader()

  Get the `ClassLoader` for this job.

  Returns:
    the correct class loader.

### setClassLoader

    public void setClassLoader(ClassLoader classLoader)

  Set the class loader that will be used to load the various objects.

  Parameters:
    `classLoader` - the new class loader.

### toString

    public String toString()

  Overrides:
    `toString` in class `Object`

### setQuietMode

    public void setQuietMode(boolean quietmode)

  Set the quietness-mode. In the quiet-mode, error and informational messages might not be logged.

  Parameters:
    `quietmode` - `true` to set quiet-mode on, `false` to turn it off.

### main

    public static void main(String[] args)
                     throws Exception

  For debugging. List non-default properties to the terminal and exit.

  Throws:
    `Exception`

### readFields

    public void readFields(DataInput in)
                    throws IOException

  Description copied from interface: `Writable`
  Deserialize the fields of this object from `in`.

  For efficiency, implementations should attempt to re-use storage in the existing object where possible.

  Specified by:
    `readFields` in interface `Writable`
  Parameters:
    `in` - `DataInput` to deseriablize this object from.
  Throws:
    `IOException`

### write

    public void write(DataOutput out)
               throws IOException

  Description copied from interface: `Writable`
  Serialize the fields of this object to `out`.

  Specified by:
    `write` in interface `Writable`
  Parameters:
    `out` - `DataOuput` to serialize this object into.
  Throws:
    `IOException`

### getValByRegex

    public Map<String,String> getValByRegex(String regex)

  get keys matching the the regex

  Parameters:
    `regex` -
  Returns:
    Map with matching keys

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2009 The Apache Software Foundation