com.cloudera.sqoop.manager
Class SqlManager

java.lang.Object
  extended by com.cloudera.sqoop.manager.ConnManager
      extended by com.cloudera.sqoop.manager.SqlManager
Direct Known Subclasses:
GenericJdbcManager

public abstract class SqlManager
extends ConnManager

ConnManager implementation for generic SQL-compliant database. This is an abstract class; it requires a database-specific ConnManager implementation to actually create the connection.


Field Summary
static org.apache.commons.logging.Log LOG
           
protected  SqoopOptions options
           
 
Constructor Summary
SqlManager(SqoopOptions opts)
          Constructs the SqlManager.
 
Method Summary
 void close()
          Perform any shutdown operations on the connection.
 void execAndPrint(java.lang.String s)
          Poor man's SQL query interface; used for debugging.
protected  java.sql.ResultSet execute(java.lang.String stmt, java.lang.Object... args)
          Executes an arbitrary SQL statement.
 void exportTable(ExportJobContext context)
          Export data stored in HDFS into a table in a database.
protected  void formatAndPrintResultSet(java.sql.ResultSet results, java.io.PrintWriter pw)
          Prints the contents of a ResultSet to the specified PrintWriter.
protected  java.lang.String getColNamesQuery(java.lang.String tableName)
           
protected  java.lang.String getColTypesQuery(java.lang.String tableName)
           
 java.lang.String[] getColumnNames(java.lang.String tableName)
          Return a list of column names in a table in the order returned by the db.
 java.util.Map<java.lang.String,java.lang.Integer> getColumnTypes(java.lang.String tableName)
          Return an unordered mapping from colname to sqltype for all columns in a table.
abstract  java.sql.Connection getConnection()
          Retrieve the actual connection from the outer ConnManager.
 java.lang.String getPrimaryKey(java.lang.String tableName)
          Return the name of the primary key for a table, or null if there is none.
protected  java.lang.String getSplitColumn(SqoopOptions opts, java.lang.String tableName)
          Determine what column to use to split the table.
 void importTable(ImportJobContext context)
          Default implementation of importTable() is to launch a MapReduce job via DataDrivenImportJob to read the table with DataDrivenDBInputFormat.
 java.lang.String[] listDatabases()
          Return a list of all databases on a server.
 java.lang.String[] listTables()
          Return a list of all tables in a database.
protected  java.sql.Connection makeConnection()
          Create a connection to the database; usually used only from within getConnection(), which enforces a singleton guarantee around the Connection object.
 java.sql.ResultSet readTable(java.lang.String tableName, java.lang.String[] columns)
          Execute a SQL statement to read the named set of columns from a table.
 void release()
          If a method of this ConnManager has returned a ResultSet to you, you are responsible for calling release() after you close the ResultSet object, to free internal resources.
 java.lang.String toHiveType(int sqlType)
          Resolve a database-specific type to Hive data type.
 java.lang.String toJavaType(int sqlType)
          Resolve a database-specific type to the Java type that should contain it.
 
Methods inherited from class com.cloudera.sqoop.manager.ConnManager
escapeColName, escapeTableName, getDriverClass
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

options

protected SqoopOptions options
Constructor Detail

SqlManager

public SqlManager(SqoopOptions opts)
Constructs the SqlManager.

Parameters:
opts - the SqoopOptions describing the user's requested action.
Method Detail

getColNamesQuery

protected java.lang.String getColNamesQuery(java.lang.String tableName)
Returns:
the SQL query to use in getColumnNames() in case this logic must be tuned per-database, but the main extraction loop is still inheritable.

getColumnNames

public java.lang.String[] getColumnNames(java.lang.String tableName)
Description copied from class: ConnManager
Return a list of column names in a table in the order returned by the db.

Specified by:
getColumnNames in class ConnManager

getColTypesQuery

protected java.lang.String getColTypesQuery(java.lang.String tableName)
Returns:
the SQL query to use in getColumnTypes() in case this logic must be tuned per-database, but the main extraction loop is still inheritable.

getColumnTypes

public java.util.Map<java.lang.String,java.lang.Integer> getColumnTypes(java.lang.String tableName)
Description copied from class: ConnManager
Return an unordered mapping from colname to sqltype for all columns in a table. The Integer type id is a constant from java.sql.Types

Specified by:
getColumnTypes in class ConnManager

readTable

public java.sql.ResultSet readTable(java.lang.String tableName,
                                    java.lang.String[] columns)
                             throws java.sql.SQLException
Description copied from class: ConnManager
Execute a SQL statement to read the named set of columns from a table. If columns is null, all columns from the table are read. This is a direct (non-parallelized) read of the table back to the current client. The client is responsible for calling ResultSet.close() when done with the returned ResultSet object, and for calling release() after that to free internal state.

Specified by:
readTable in class ConnManager
Throws:
java.sql.SQLException

listDatabases

public java.lang.String[] listDatabases()
Description copied from class: ConnManager
Return a list of all databases on a server.

Specified by:
listDatabases in class ConnManager

listTables

public java.lang.String[] listTables()
Description copied from class: ConnManager
Return a list of all tables in a database.

Specified by:
listTables in class ConnManager

getPrimaryKey

public java.lang.String getPrimaryKey(java.lang.String tableName)
Description copied from class: ConnManager
Return the name of the primary key for a table, or null if there is none.

Specified by:
getPrimaryKey in class ConnManager

getConnection

public abstract java.sql.Connection getConnection()
                                           throws java.sql.SQLException
Retrieve the actual connection from the outer ConnManager.

Specified by:
getConnection in class ConnManager
Returns:
the actual database connection.
Throws:
java.sql.SQLException

getSplitColumn

protected java.lang.String getSplitColumn(SqoopOptions opts,
                                          java.lang.String tableName)
Determine what column to use to split the table.

Parameters:
opts - the SqoopOptions controlling this import.
tableName - the table to import.
Returns:
the splitting column, if one is set or inferrable, or null otherwise.

importTable

public void importTable(ImportJobContext context)
                 throws java.io.IOException,
                        ImportException
Default implementation of importTable() is to launch a MapReduce job via DataDrivenImportJob to read the table with DataDrivenDBInputFormat.

Specified by:
importTable in class ConnManager
Throws:
java.io.IOException
ImportException

execute

protected java.sql.ResultSet execute(java.lang.String stmt,
                                     java.lang.Object... args)
                              throws java.sql.SQLException
Executes an arbitrary SQL statement.

Parameters:
stmt - The SQL statement to execute
Returns:
A ResultSet encapsulating the results or null on error
Throws:
java.sql.SQLException

toJavaType

public java.lang.String toJavaType(int sqlType)
Resolve a database-specific type to the Java type that should contain it.

Specified by:
toJavaType in class ConnManager
Parameters:
sqlType -
Returns:
the name of a Java type to hold the sql datatype, or null if none.

toHiveType

public java.lang.String toHiveType(int sqlType)
Resolve a database-specific type to Hive data type.

Specified by:
toHiveType in class ConnManager
Parameters:
sqlType - sql type
Returns:
hive type

close

public void close()
           throws java.sql.SQLException
Description copied from class: ConnManager
Perform any shutdown operations on the connection.

Specified by:
close in class ConnManager
Throws:
java.sql.SQLException

formatAndPrintResultSet

protected void formatAndPrintResultSet(java.sql.ResultSet results,
                                       java.io.PrintWriter pw)
Prints the contents of a ResultSet to the specified PrintWriter. The ResultSet is closed at the end of this method.

Parameters:
results - the ResultSet to print.
pw - the location to print the data to.

execAndPrint

public void execAndPrint(java.lang.String s)
Poor man's SQL query interface; used for debugging.

Specified by:
execAndPrint in class ConnManager
Parameters:
s - the SQL statement to execute.

makeConnection

protected java.sql.Connection makeConnection()
                                      throws java.sql.SQLException
Create a connection to the database; usually used only from within getConnection(), which enforces a singleton guarantee around the Connection object.

Throws:
java.sql.SQLException

exportTable

public void exportTable(ExportJobContext context)
                 throws java.io.IOException,
                        ExportException
Export data stored in HDFS into a table in a database.

Overrides:
exportTable in class ConnManager
Throws:
java.io.IOException
ExportException

release

public void release()
Description copied from class: ConnManager
If a method of this ConnManager has returned a ResultSet to you, you are responsible for calling release() after you close the ResultSet object, to free internal resources. ConnManager implementations do not guarantee the ability to have multiple returned ResultSets available concurrently. Requesting a new ResultSet from a ConnManager may cause other open ResulSets to close.

Specified by:
release in class ConnManager


Copyright © 2010 Cloudera, Inc.