org.apache.hadoop.hive.metastore
Class MetaStore
java.lang.Object
org.apache.hadoop.hive.metastore.MetaStore
public class MetaStore
- extends Object
MetaStore
A MetaStore on top of HDFS. The goal is to expose Tables/Schemas to users rather than flat files and
Wiki that describes their structure and contents.
The MetaStore is used in conjunction with org.apache.hadoop.contrib.hive.contrib.serde.SerDe and flat files for
storing other schema information in addition to the SerDe library for each table.
The store has the concept of a db. The db is assumed to be optionally prefixed to the tableName and followed by a dot.
e.g., falcon.view_photo, default.tmp_pete.
In the schema store on disk, these dbs are stored as SCHEMA_STORE_PATH/.db/ and then schemas under it are .dir/schema.
The schema store should have a symbolic link - ln -s SCHEMA_STORE_PATH SCHEMA_STORE_PATH/default.db .
All of a db's tables are stored in $ROOT_WAREHOUSE/dbname/tablename(s). The default db is special cased to
$ROOT_WAREHOUSE since dfs does not have symbolic links and since our data is already there.
Internally, almost everywhere, the code ignores the db name - that is other than the following conditions:
1. When looking up the schema file from the table name (s/\./.db/)
2. When deriving the table's intended location on DFS (s/^(.+?)\.(.+)$/$1/$2/)
3. When calling getFields(db.table.field1.field2). Here it peels off the prefix and checks if it's a db name
TODOs:
Think about making "db" an object and schema too.
Try to abstract away how we store dbs in the metastore and the warehouse. The latter is hard because the hive
runtime needs a way to lookup a table's schema from looking at the path to a specific partition that the map
is running on.
Method Summary |
static boolean |
dbExists(String dbName,
org.apache.hadoop.conf.Configuration conf)
|
static List<String> |
getDbs(org.apache.hadoop.conf.Configuration conf)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
RequiredSchemaKeys
protected static final String[] RequiredSchemaKeys
- Every schema must have a name, location and a serde
LogKey
public static final String LogKey
- See Also:
- Constant Field Values
DefaultDB
public static final String DefaultDB
- See Also:
- Constant Field Values
MetaStore
public MetaStore()
dbExists
public static boolean dbExists(String dbName,
org.apache.hadoop.conf.Configuration conf)
throws MetaException
- Throws:
MetaException
getDbs
public static List<String> getDbs(org.apache.hadoop.conf.Configuration conf)
throws MetaException
- Throws:
MetaException
Copyright © 2009 The Apache Software Foundation