commit ded54b29979a93e0f3ed773175cafd16b72511ba Author: Eli Collins Date: Thu May 20 17:15:13 2010 -0700 CLOUDERA-BUILD. Rename the fuse_dfs wrapper. Description: Rename the fuse_dfs wrapper to hadoop-fuse-dfs. Reason: Improvement Author: Alex Newman Ref: CDH-1103 commit 3e421937926d870a00809099352081f1bd82bab0 Author: Chad Metcalf Date: Wed May 19 15:38:14 2010 -0700 CLOUDERA-BUILD. Fixes for the fuse_dfs wrapper. Description: The wrapper uses bash syntax (i.e., +=) so we should use bash. We need to modprobe fuse explicitly on Ubuntu. Since this is installed by install_hadoop.sh we know HADOOP_HOME and should use it directly. Lastly, there is more robust JAVA_HOME checking in hadoop-config.sh so we should use that. Reason: Fuse currently broken on Ubuntu Author: Chad Metcalf Ref: CDH-1089 commit f9d68f074e5fad2bfe66912564f656b319625d51 Author: Eli Collins Date: Tue May 18 14:30:04 2010 -0700 HDFS-1161. Make DN minimum valid volumes configurable Description: This change adds a dfs.datanode.failed.volumes.tolerated parameter so that users can configure the number of volumes that are allowed to fail before a datanode stops offering service. By default any volume failure will cause a datanode to shutdown. Reason: Improvement Author: Eli Collins Ref: CDH-1081 commit 52a0074ebc6b409b3fccccd3089d66b56f2ead73 Author: Eli Collins Date: Mon May 17 19:49:44 2010 -0700 HDFS-1160. Improve some FSDataset warnings and comments. Description: Cleans up HDFS-547 warnings. Reason: Improvement Author: Eli Collins Ref: CDH-1080 commit 3f7ccf20e6ba4ef7caf1f2b46fe92bb644bdf3b8 Author: Eli Collins Date: Mon May 17 18:53:50 2010 -0700 HDFS-612. FSDataset should not use org.mortbay.log.Log. Description: Cleans up HDFS-547 logging. Reason: Improvement Author: Eli Collins Ref: CDH-1079 commit 4e5d23d3f7082e6606f5e187e4bce9ab98c33c93 Author: Eli Collins Date: Thu Apr 22 14:41:08 2010 -0700 HDFS-457. Better handling of volume failure in Data Node storage. Description: Current implementation shuts DataNode down completely when one of the configured volumes of the storage fails. This is rather wasteful behavior because it decreases utilization (good storage becomes unavailable) and imposes extra load on the system (replication of the blocks from the good volumes). These problems will become even more prominent when we move to mixed (heterogeneous) clusters with many more volumes per Data Node. Reason: Improvement Author: Eli Collins Ref: CDH-472 commit 3baa51943c748659e56a815d2643bd0188dd8c08 Author: Eli Collins Date: Sat May 15 13:39:08 2010 -0700 HADOOP-3659. Fix hadoop native to compile on Mac OS X. Description: This patch makes the autoconf script work on Mac OS X. LZO needs to be installed (including the optional shared libraries) for the compile to succeed. You'll want to regenerate the configure script using autoconf after applying this patch. Reason: Bug fix Author: Eli Collins Ref: CDH-825 commit 8f2fc6be652aea2ee7e7be4ce577e1cca5dd09b9 Author: Eli Collins Date: Sat May 15 12:55:06 2010 -0700 MAPREDUCE-1785. Add streaming config option for not emitting the key. Description: PipeMapper currently does not emit the key when using TextInputFormat. If you switch to input formats (eg LzoTextInputFormat) the key will be emitted. We should add an option so users can explicitly make streaming not emit the key so they can change input formats without breaking or having to modify their existing programs. Reason: Improvement Author: Eli Collins Ref: CDH-856 commit 93452c208373cd17250facecbb23ddddfadd3535 Author: Eli Collins Date: Thu May 13 20:07:02 2010 -0700 HDFS-1024. SecondaryNamenode fails to checkpoint because namenode fails with CancelledKeyException. Description: The secondary namenode fails to retrieve the entire fsimage from the Namenode. It fetches a part of the fsimage but believes that it has fetched the entire fsimage file and proceeds ahead with the checkpointing. Reason: Bug fix Author: Eli Collins Ref: CDH-891 commit 1e18e0b65a4a793289f3427fd805c18e34789ed4 Author: Todd Lipcon Date: Wed May 12 21:33:45 2010 -0700 HADOOP-6254. Slow reads cause s3n to fail with SocketTimeoutException Reason: Bug fix for users of s3n:// file system Author: Andrew Hitchcock Ref: CDH-1035 commit 11af68790ccb757de2a8ebcc921130edbee21b6d Author: Eli Collins Date: Wed May 12 12:05:26 2010 -0700 HADOOP-6667. RPC.waitForProxy should retry through NoRouteToHostException. Description: RPC.waitForProxy already loops through ConnectExceptions, but NoRouteToHostException is not a subclass of ConnectException. In the case that the NN is on a VIP, the No Route To Host error is reasonably common during a failover, so we should retry through it just the same as the other connection errors. Reason: Improvement Author: Eli Collins Ref: CDH-907 commit d3db49b4e5a4bef64615f9229a2784472529bf28 Author: Todd Lipcon Date: Fri Apr 16 14:54:59 2010 -0700 Ammend MAPREDUCE-551. Fix FairScheduler preemption when slow start is configured to non-default value Description: The Fair Scheduler assumed that slow start was always configured to the default, rather than looking at the actual configured value. This patch changes it to ask the JobInProgress whether reduces are schedulable rather than running its own calculations. This fix does not directly correspond to an Apache JIRA, since it affects only the version of preemption that is in CDH2. The version in CDH3 and in any Apache releases after 0.20 does not have this bug (was fixed by MAPREDUCE-706) Reason: Fixes incorrect preemption for jobs with slow start enabled. Author: Todd Lipcon Ref: CDH-948 commit 822204c6784893d548b8772f96224a5ca7bbd2c2 Author: newalex Date: Fri Apr 16 15:48:14 2010 -0700 CLOUDERA-BUILD. Add a fuse manpage Description: Adding a fuse_dfs manpage and adding a manpage to the build. Reason: New Feature Author: Alex Newman Ref: CDH-927 commit cff94989b10c8b6b6295d440bab41bd75838e11c Author: newalex Date: Thu Apr 8 10:35:19 2010 -0700 CLOUDERA-BUILD. Build script changes to build DEB packages Description: The required changes to the cloudera hadoop building scripts for pulling the fuse files out and cleaning up its mess v.v. DEBs. Reason: Building packages Author: Alex Newman Ref: CDH-929 commit 732c9a95dac4f135517f0c458beb5b2ce6013072 Author: newalex Date: Tue Apr 6 14:05:29 2010 -0700 CLOUDERA-BUILD. Added an RPM for fuse Description: The required changes to the cloudera hadoop building scripts for pulling the fuse files out and cleaning up its mess. Reason: Building packages Author: Alex Newman Ref: CDH-928 commit 377d10d403f3f1e44f806294dee3e544cedbdfa2 Author: Eli Collins Date: Tue Mar 30 15:17:50 2010 -0700 HADOOP-6522. Fix decoding of codepoint zero in UTF8. Description: TestUTF8 is actually flaky. It generates 10 random strings to run the test on. If you change this number to 100000 it fails every time. The problem is that the null character (codepoint zero) was correctly encoded but incorrectly decoded. I've attached a patch that fixes this and increases the size of the tests so that problems like this will likely be discovered sooner. Reason: Bugfix to UTF8 Author: Eli Collins Ref: CDH-718 commit c4d6c9d160e9fa52d72eb993ab719bc333695b8a Author: Aaron Kimball Date: Mon Mar 29 16:43:49 2010 -0700 MAPREDUCE-1536. DataDrivenDBInputFormat does not split date columns correctly. Description: The DateSplitter does not properly split a range of (min, max) dates. Reason: Bugfix to DateSplitter Author: Aaron Kimball Ref: CDH-813 commit 18b1c11a3de045d599d8fe27c3da33e0d45d9790 Author: Aaron Kimball Date: Mon Mar 29 16:11:22 2010 -0700 MAPREDUCE-1480. CombineFileRecordReader does not properly initialize child RecordReader Description: CombineFileRecordReader instantiates child RecordReader instances but never calls their initialize() method to give them the proper TaskAttemptContext. Reason: Bug in CombineFileInputFormat prevents proper use. Author: Aaron Kimball Ref: CDH-811 commit 4093cc0e74fd70eeb6a1a2d1b46dd48dc4f2fc76 Author: Aaron Kimball Date: Mon Mar 29 15:50:20 2010 -0700 MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple pools are configured Description: I have a map-reduce job that is using CombineFileInputFormat. It has configured 10000 pools and 30000 files. The time to create the splits takes more than an hour. The reaosn being that CombineFileInputFormat.getSplits() converts the same path from String to Path object multiple times, one for each instance of a pool. Similarly, it calls Path.toUri(0 multiple times. This code can be optimized. Reason: Improves CombineFileInputFormat performance (used by Sqoop); needed to apply MAPREDUCE-1480 cleanly Author: Dhruba Borthakur Ref: CDH-811 commit 0aa90ccd711ad26157d9b608b7958802bee5be63 Author: Aaron Kimball Date: Mon Mar 29 15:41:38 2010 -0700 MAPREDUCE-364. Change org.apache.hadoop.examples.MultiFileWordCount to use new mapreduce api. Description: Updates MultiFileWordCount example to use the new API in org.apache.hadoop.mapreduce instead of the deprecated API of org.apache.hadoop.mapred. This incorporates MAPREDUCE-367: Change org.apache.hadoop.mapred.lib.CombineFileInputFormat to use the new api. This solves duplicate issue MAPREDUCE-1112: Fix CombineFileInputFormat for hadoop 0.20 Reason: CombineFileInputFormat required for many clients of the new API, including Sqoop. Author: Amareshwari Sriramadasu Ref: CDH-811 commit c78aed9266941644c7704c3429e5fcb6466c13d9 Author: Chad Metcalf Date: Thu Mar 18 17:05:47 2010 -0700 HADOOP-6643. Set executable bit for python cloud scripts in the distribution Description: This needs to be set in the tar target. Reason: Required for the EC2 scripts. Author: Tom White Ref: CDH-821 commit 0a98764cc14ba860ceaedb3c3739069f709dca6f Author: Chad Metcalf Date: Mon Mar 1 15:28:19 2010 -0800 CLOUDERA-BUILD. hadoop-config runs before hadoop-env.sh conf/hadoop-env.sh says you can update JAVA_HOME there, but it gets sourced after hadoop-config.sh, which errors out if JAVA_HOME is not set. This patch changes the flow so hadoop-env is always sourced by hadoop-config after the --config flag is processed. This will allow JAVA_HOME to be set in hadoop-env and still allow for trying to find a valid JAVA_HOME. commit ad25b8aee50a0616c472e4d71d5b8e8770d214b0 Author: Todd Lipcon Date: Tue Mar 2 17:24:05 2010 -0800 Small tweaks and fixes to Cloudera styling: - Fixes trivial CSS bug for missing table cell borders in Chrome - Fixes footer to read "Distribution for Hadoop" instead of "Distribution of Hadoop" commit 9124aeee835a5b892760c3cbce151563374ca402 Author: Eli Collins Date: Mon Mar 1 17:36:41 2010 -0800 HDFS-761. Failure to process rename operation from edits log due to quota verification. When processing edits log, quota verification is not done and the used quota for directories is not updated. The update is done at the end of processing edits log. This rule is broken by change introduced in HDFS-677. This causes namenode from handling rename operation from edits log due to quota verification failure. Once this happens, namenode does not proceed edits log any further. This results in check point failure on backup node or secondary namenode. This also prevents namenode from coming up. commit 444f081e2fce75166c7a9735bf8842bc1b754056 Author: Todd Lipcon Date: Wed Feb 24 21:56:45 2010 -0800 Improve markup and add Cloudera styling to Web UIs This adds a favicon and a number of HTML/CSS improvements to make the pages more space-efficient and easy on the eyes. This may be an incompatible change for users who are scraping the HTML output of the web UIs. Those users are encouraged to access the data programmatically rather than through scraping. The non-Cloudera-specific improvements will be contributed upstream as HDFS-1013 and MAPREDUCE-1544. commit 3a843c84468c975d780093ec2f501837d5d915ac Author: Aaron Kimball Date: Tue Feb 23 16:39:04 2010 -0800 Disable failing asserts (see HDFS-696). Disabled asserts in HDFS that cause unit tests to fail. These will be re-enabled at a later date when the underlying cause is fixed upstream. In the meantime, these are disabled to keep our CI server returning only new failures. Issue HDFS-696 lists the failing tests and tracks their progress. commit 99aba75b4a4425c838c99ffbf27693f25a75ca4c Author: Aaron Kimball Date: Thu Feb 18 15:28:12 2010 -0800 MAPREDUCE-1093. Java assertion failures triggered by tests Removes failing asserts from the CDH build until they are fixed in trunk. Tracking MAPREDUCE-1506 to include a fix for this assertion failure. commit 24bdd4c1118891ce7e58bd7a67db4fe5578fbe14 Author: Aaron Kimball Date: Thu Feb 18 15:27:06 2010 -0800 MAPREDUCE-1092. Enable asserts for tests by default. commit 69bfc1662e8beff86ac46c00596cf1911b8766f1 Author: Eli Collins Date: Mon Feb 15 20:01:18 2010 -0800 HDFS-961: dfs_readdir incorrectly parses paths dfs_readdir in fuse-dfs does not work if the default fs.default.name port is used because it incorrectly parses the path component from the URI. commit 3181be14ebe7519de94dfb59eefdce845e76cad5 Author: Aaron Kimball Date: Fri Feb 12 12:01:30 2010 -0800 MAPREDUCE-1467. Add a --verbose flag to Sqoop. commit 584eef4a8ad869bbf23255238eb0e1cdcaf417ad Author: Todd Lipcon Date: Tue Feb 2 13:38:43 2010 -0800 HDFS-455, MAPREDUCE-1441, HADOOP-6534. Allow spaces in between comma-separated elements in directory list configurations. This fixes any configuration consisting of a comma-separated list of directories (eg dfs.data.dir, dfs.name.dir, fs.checkpoint.dir, mapred.local.dir, etc) so that the elements may also contain separating whitespace. Without this patch, setting mapred.local.dir to "/disk1, /disk2" would create a directory by the name " " in the user's home directory, or fail outright. The patch trims the directory names as they are fetched from the configuration. commit c33757342d74c189a0d2e256c213f8e0d134df9c Author: Todd Lipcon Date: Tue Feb 2 13:34:04 2010 -0800 HADOOP-2366. Support trimmed strings in Configuration. Contributed by Michele Catasta This provides support in Configuration to get comma-separated string lists in such a way that whitespace in between elements is ignored. This patch is required for later patches which fix mapred.local.dir, dfs.data.dir, etc to support spaces in between elements. Test plan: unit tested in TestStringUtils commit 8e662cb065be1c4bc61c55e6bff161e09c1d36f3 Author: Aaron Kimball Date: Fri Jan 29 14:45:49 2010 -0800 MAPREDUCE-1395. Sqoop does not check return value of Job.waitForCompletion() commit 3cb2197195085d3784381ff83cf3037cf4a01093 Author: Chad Metcalf Date: Thu Jan 28 17:28:17 2010 -0800 Set HADOOP_PID_DIR to something smarter then /tmp commit 2d805c0b1825855f0416f9f94ebb98a1a221a71c Author: Chad Metcalf Date: Thu Jan 28 17:23:13 2010 -0800 HADOOP-6453. Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH. commit 07985070f9c9835cfba36bd79f9edadc89d8cfc0 Author: Aaron Kimball Date: Tue Jan 26 22:18:18 2010 -0600 MAPREDUCE-1313. NPE in FieldFormatter if escape character is set and field is null commit e01c72fcd9ba23f5b4bf024d2d75a3f6225af4f4 Author: Aaron Kimball Date: Tue Jan 26 22:18:55 2010 -0600 MAPREDUCE-1394. Sqoop generates incorrect URIs in paths sent to Hive commit 4a31d76b50a69d3574570f6e36a070e8787493ea Author: Eli Collins Date: Tue Jan 26 19:10:11 2010 -0800 HADOOP-6460: Namenode runs of out of memory due to memory leak in ipc Server commit 9ef51e0a7619175c010b0892a1a0583316e27f42 Author: Eli Collins Date: Tue Jan 26 19:04:17 2010 -0800 HADOOP 6498: PC client bug may cause rpc call hang commit b960818cabda2d2c2f95c836b85d3c2c646faadc Author: Eli Collins Date: Tue Jan 26 15:18:38 2010 -0800 HADOOP-5687: Hadoop NameNode throws NPE if fs.default.name is the default value commit 9c9e3573ec73d919f0adb7e2521d865d57f21645 Author: Chad Metcalf Date: Fri Jan 22 17:23:14 2010 -0800 HADOOP-6506. Failing tests prevent the rest of test targets from execution. commit fbff89064e0896cb8090f4c966f4c81661b4591c Author: Chad Metcalf Date: Fri Jan 22 14:30:58 2010 -0800 HADOOP-6505: sed in build.xml fails. commit 8377b4238abcd646fbba58f2378c017102df44c6 Author: Todd Lipcon Date: Fri Jan 22 11:45:35 2010 -0800 HDFS-908 v2: Fix TestDistributedFileSystem hftp URI checking The previous patch fixed some systems but broke others. commit 5caa833e6de373b6fd38f94580cf884b8898d67b Author: Todd Lipcon Date: Thu Jan 21 18:49:17 2010 -0800 HADOOP-6503. Pull common ivy dependencies into contrib projects commit f78d19c3eee499c9e55fdf2a86d8bd975fe75f6d Author: Todd Lipcon Date: Mon Oct 26 12:31:41 2009 -0700 MAPREDUCE-1155. Fix streaming tests to properly throw exceptions, and pass. commit 06d7d9ab37c238a0183776c42f91c79216129d3a Author: Todd Lipcon Date: Thu Jan 21 15:23:16 2010 -0800 HADOOP-5647. TestJobHistory fails if /tmp/_logs is not writable to This fix is scoped to the test only, and reduces spurious test failures on shared QA machines. commit fe8d6dc0a80ba6aacf6c978223da0b4e9810b347 Author: Todd Lipcon Date: Thu Jan 21 13:46:04 2010 -0800 Fix javadoc warnings shown by test-patch, and update eclipse classpath to match current CDH. commit 90e1e9606080ff67adeef7c529fb1d49c41820df Author: Todd Lipcon Date: Thu Jan 21 12:42:46 2010 -0800 HDFS-127. Block read failures counter should be reset before new operations This is a second attempt at fixing HDFS-127. The behavior here is to reset the failure counter before any user-side read operation. Thus, retries will not risk resetting the failure counter, since the retry logic is all further inside the internals. This patch also contains two new unit tests: (a) a test in TestCrcCorruption which verifies against the regression seen in the first implementation of HDFS-127. (b) a test in TestDFSClientRetries which verifies that this new patch fixes the bug originally reported in the issue. commit 096b4baf02d6ee975d4cc9225d69e87c2054b509 Author: Todd Lipcon Date: Thu Jan 14 16:05:30 2010 -0800 Revert "HDFS-127: DFSClient block read failures cause open DFSInputStream to become unusable" This reverts commit d9e9589506f8856fcb607cdf0465bad22f48a267. This is being reverted as it causes infinite retries when there are no valid replicas. commit 609ddbf89b31327ecbcd5c1cc5c121120d9bba11 Author: Todd Lipcon Date: Thu Jan 14 17:18:40 2010 -0800 HDFS-877. Fix client-side block verification and add a unit test to prevent regressions. This is a simpler patch than the one on 0.21/0.22 since those fix a further regression since 0.20. commit 4f3ebc131bbbe17d441be7d8de9c0cc1bdb19aa5 Author: Todd Lipcon Date: Thu Jan 21 12:53:04 2010 -0800 Add mockito to 0.20 branch for easier unit testing of HDFS stability patches. commit ac59dd998f16734c07d8dc6fe38b3ddd427f44db Author: Todd Lipcon Date: Wed Jan 13 20:42:46 2010 -0800 Backport HDFS-630. In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block Contributed by Cosmin Lehene, modified by Cloudera to not break compatibility commit 17d88b28c6d5e531aa94c5933f703e5bcb4e4503 Author: Todd Lipcon Date: Thu Jan 21 17:49:42 2010 -0800 HDFS-793. Data node should receive the whole packet ack message before it constructs and sends its own ack message for the packet. Contributed by Hairong Kuang and ported to branch-0.20 in a compatible manner by Todd Lipcon. This patch is from HDFS-872 2010-01-16 12:09am. commit 97efc24a0e46f243d1903ca762ccc8f1971c9213 Author: Todd Lipcon Date: Tue Jan 19 18:58:06 2010 -0800 HDFS-908. Fix TestDistributedFileSystem on nodes with strange local DNS commit c8e1f700a050fd9e183de019246147051c12b9d2 Author: Todd Lipcon Date: Thu Jan 14 16:12:56 2010 -0800 HDFS-187. Fix TestStartup for the case when a SecondaryNameNode is running on test machine Contributed by Todd Lipcon commit 338fea8b2ae9d7c6ef2d0bbea7965ccd450e88f7 Author: Aaron Kimball Date: Thu Jan 21 11:45:04 2010 -0800 MAPREDUCE-433. TestReduceFetch failed. commit fb4940f784df1cbae7dec5c96ba0d2de3f58dbb9 Author: Eli Collins Date: Tue Jan 19 18:38:36 2010 -0800 HDFS-464: Memory leaks in libhdfs commit 1577b1f02b311379ca298e7b00c06a5acadd6ec0 Author: Aaron Kimball Date: Fri Jan 15 19:55:30 2010 -0800 Add test ivy configurations to additional projects. commit 4f34dcb5d900baf8da042ebe8a995962eb3456c9 Author: Aaron Kimball Date: Fri Jan 15 11:42:51 2010 -0800 Sqoop bin script now includes jars from contrib/sqoop/lib/ on classpath. commit d0d1a63026ea62529d1f60b23afefe8d4d4ae97d Author: Aaron Kimball Date: Fri Jan 15 11:32:01 2010 -0800 MAPREDUCE-1212. Mapreduce contrib project ivy dependencies are not included in binary target commit 50ecc4cb551a248ef5e8f9089fd7bbac2f07e2f2 Author: Tom White Date: Thu Jan 14 16:16:28 2010 -0800 Update cloud script version to cdh-0.20.1+169. commit fe979f434930647f0a52facad289e16197f958d3 Author: Tom White Date: Thu Jan 14 16:01:16 2010 -0800 HADOOP-6466. Add a ZooKeeper service to the cloud scripts. commit 115fbc409084e739fe673a63914e969a39be3165 Author: Tom White Date: Thu Jan 14 15:58:13 2010 -0800 HADOOP-6454. Create setup.py for EC2 cloud scripts commit c453ae7b178c8db5c075daf1522baa67c6c205ef Author: Tom White Date: Thu Jan 14 15:45:46 2010 -0800 HADOOP-6462. contrib/cloud failing, target compile does not exist commit 0e6535d506f9f29c5eb0c88707e429b262fe8d19 Author: Tom White Date: Thu Jan 14 15:04:40 2010 -0800 HADOOP-6444. Support additional security group option in hadoop-ec2 script commit 1f23e69da490a3f5dc52d3256c0aff535ce1874e Author: Tom White Date: Thu Jan 14 15:00:20 2010 -0800 HADOOP-6426. Create ant build for running EC2 unit tests commit 0f5e42fb3b12bb1aa2eb8821ded3bf1a8fd9ea19 Author: Tom White Date: Thu Jan 14 14:49:58 2010 -0800 HADOOP-6392. Run namenode and jobtracker on separate EC2 instances commit e8fa28a8be92c907c5242eb64c2326fca0e9226c Author: Aaron Kimball Date: Thu Jan 14 16:15:09 2010 -0800 HADOOP-6315. GzipCodec should not represent BuiltInZlibInflater as decompressorType commit 800a132016acce7fb158b6d2b123780a6a145770 Author: Eli Collins Date: Tue Jan 12 21:09:02 2010 -0800 HDFS-861: fuse-dfs does not support O_RDWR commit d1d6748ef6339e8288666419ba008b3569532103 Author: Eli Collins Date: Tue Jan 12 20:56:24 2010 -0800 HDFS-860: fuse-dfs truncate behavior causes issues with scp commit 4751df2bb4ecbde424502139a21477160db51931 Author: Eli Collins Date: Tue Jan 12 19:26:01 2010 -0800 HDFS-859: fuse-dfs utime behavior causes issues with tar commit a5ccd395399952bf913eb14ea1e0c4208cf66274 Author: Eli Collins Date: Tue Jan 12 19:05:54 2010 -0800 HDFS-858: Incorrect return codes for fuse-dfs commit 364f5174210cb3fdac24f1bbe7a0fd9d68d7be7e Author: Eli Collins Date: Tue Jan 12 18:39:39 2010 -0800 HDFS-857: Incorrect type for fuse-dfs capacity can cause df to return negative values on 32-bit machines commit c17dd16a34aaf0e9eea4f4809b7d6636e01bdfe5 Author: Eli Collins Date: Tue Jan 12 18:33:54 2010 -0800 HDFS-856: Hardcoded replication level for new files in fuse-dfs commit eab82a49029b8eaa6686820227fa2256d1d40f5f Author: Tom White Date: Thu Jan 14 14:41:24 2010 -0800 HADOOP-6108. Add support for EBS storage on EC2 commit 609890fdb9aea87df4f7ac6f1db0ce43e1292504 Author: Eli Collins Date: Tue Jan 12 14:42:30 2010 -0800 HADOOP-5625: Add I/O duration time in client trace commit 1747d7847952a798836625eb28da4d653c4f37ed Author: Eli Collins Date: Tue Jan 12 16:42:27 2010 -0800 HADOOP-4368: Superuser privileges required to do "df" commit 0db41dd2dea6156dd0e838404278a5a6e690ce69 Author: Eli Collins Date: Tue Jan 12 14:45:13 2010 -0800 HDFS-412: Hadoop JMX usage makes Nagios monitoring impossible (HADOOP-4482) commit 4264931e7a0cc58987dd08d206ee61a97f3db453 Author: Eli Collins Date: Tue Jan 12 11:11:36 2010 -0800 HADOOP-5222: Add offset in client trace commit 317491e189b28522078b7f8916781e908b526423 Author: Eli Collins Date: Thu Jan 7 09:38:17 2010 -0800 Solaris do-release-build fix (CDH-531) commit 61d3deffd72ce4f497e337120d93cf538a2283ba Author: Todd Lipcon Date: Wed Jan 6 17:52:39 2010 -0800 HDFS-758. Improve reporting of progress of decommissioning Contributed by Jitendra Nath Pandey commit 6723088dcf80cf327f703a449f7d2988abbba810 Author: Todd Lipcon Date: Wed Jan 6 17:31:35 2010 -0800 HDFS-185. Chown , chgrp , chmod operations should not be allowed when namenode is in safemode Contributed by Ravi Phulari commit d7f4ab56ccb1b19e49668c3462e3883e7466e352 Author: Aaron Kimball Date: Wed Jan 6 15:53:41 2010 -0800 MAPREDUCE-1310. CREATE TABLE statements for Hive do not correctly specify delimiters commit 9967ebe50a49310d2fa99e76efb5b8e0196093ae Author: Aaron Kimball Date: Wed Jan 6 15:49:32 2010 -0800 MAPREDUCE-1174. Sqoop improperly handles table/column names which are reserved sql words commit 898b684b40ba125eba7cb4ea6198c8b709dde593 Author: Aaron Kimball Date: Wed Jan 6 14:55:34 2010 -0800 MAPREDUCE-1235. java.io.IOException: Cannot convert value '0000-00-00 00:00:00' from column 6 to TIMESTAMP. commit 2e5876ccd877782d735d3c36b72d5737667ce3e6 Author: Aaron Kimball Date: Wed Jan 6 14:47:14 2010 -0800 MAPREDUCE-1146. Sqoop dependencies break Ecpilse build on Linux commit 59e0583d4c0fb812a513f027befd48d89c4db5e1 Author: Aaron Kimball Date: Wed Jan 6 14:45:53 2010 -0800 MAPREDUCE-1148. SQL identifiers are a superset of Java identifiers commit c6fdc457efa56331bcb67290216435b55495092d Author: Aaron Kimball Date: Wed Jan 6 14:43:53 2010 -0800 MAPREDUCE-1224. Calling "SELECT t.* from AS t" to get meta information is too expensive for big tables commit 8b87e2f97cf449e3f59264490b42125eca06d4c2 Author: Todd Lipcon Date: Wed Jan 6 14:20:33 2010 -0800 MAPREDUCE-1285. DistCp cannot handle -delete if destination is local filesystem Contributed by Peter Romianowski commit 1db814845412902b7b0ec41906a487283ce13b68 Author: Todd Lipcon Date: Wed Jan 6 13:22:56 2010 -0800 MAPREDUCE-764. TypedBytesInput's readRaw() does not preserve custom type codes. Contributed by Klaas Bosteels. Fixes a bug with dumbo interaction in streaming. commit f3fd1b2305b06cba1a2e84fa00dd98f8155bef31 Author: Todd Lipcon Date: Wed Jan 6 13:13:48 2010 -0800 HADOOP-6400. Log the exception when calls out for UnixUGI fail Contributed by Todd Lipcon commit 81e1719a093795b1d7e90c0ad39ac5d7f1911a97 Author: Todd Lipcon Date: Wed Jan 6 13:11:51 2010 -0800 MAPREDUCE-1293. AutoInputFormat doesn't work with non-default FileSystems Contributed by Andrew Hitchcock commit 9a08f8eedba882a955289d46b985d60cab3218f9 Author: Todd Lipcon Date: Tue Jan 5 15:02:49 2010 -0800 HDFS-686. NullPointerException is thrown while merging edit log and image. Contributed by Hairong Kuang. commit 194f8d98266b62d4e9df16da1548f260a2b1ad32 Author: Aaron Kimball Date: Wed Jan 6 12:12:47 2010 -0800 MAPREDUCE-1131. Using profilers other than hprof can cause JobClient to report job failure. commit 04f41708638d1a65f0f3b9a4c22183a26e7d7b1f Author: Aaron Kimball Date: Tue Jan 5 17:27:38 2010 -0800 MAPREDUCE-370. Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api. commit adb4cca3dafcd75322c34bf92f7c820d74646159 Author: Aaron Kimball Date: Tue Jan 5 13:43:56 2010 -0800 Backport MapReduceTestUtil to Hadoop 0.20 commit 6027b9b4a0cbc4eeb49f56e24f920ead0a39e46a Author: Todd Lipcon Date: Mon Dec 7 17:53:31 2009 -0800 HADOOP-5623. process provided status messages are overwritten every 10 seconds This also fixes MAPREDUCE-576 "writing to status reporter before consuming standard input causes task failure" Test plan: run the example job from MAPREDUCE-576 commit 4748ca669a9c0ba1b370dac884ea2dd5b9836c0c Author: Aaron Kimball Date: Thu Dec 31 11:57:09 2009 -0800 MAPREDUCE-1059. distcp can generate uneven map task assignments commit 5056139cb4d82953cd564e2cd8af2ec8cda78ec2 Author: Aaron Kimball Date: Thu Dec 31 11:25:58 2009 -0800 MAPREDUCE-1128. MRUnit Allows Iteration Twice commit e2ea5f9177c177f8f465098e5cf9b8a1dff2a48d Author: eli Date: Wed Dec 2 16:18:52 2009 -0800 HDFS-464. Memory leaks in libhdfs commit df3c9c774eb645dc4feb3ac52dd1620144c21cbe Author: eli Date: Wed Dec 2 19:04:20 2009 -0800 HDFS-423. Unbreak FUSE build and fuse_dfs_wrapper.sh commit 5c064dbecee62a594670d78ec40249eff33ef55d Author: eli Date: Wed Dec 2 17:59:43 2009 -0800 Make bin/hadoop-config.sh work with dev builds commit 3918d53c70ecc7fe88bf1432cad4bda8929cccc3 Author: eli Date: Mon Nov 30 13:56:40 2009 -0800 HDFS-790: c++ utils doesn't compile commit 8172e30d4c0ae2432b809bb364885223d9adde97 Author: eli Date: Mon Nov 30 13:38:21 2009 -0800 HDFS-727: bug setting block size hdfsOpenFile commit 8f541336ee323564f0ad47d00b235c1cdc37a99b Author: Eli Collins Date: Sat Nov 14 11:57:47 2009 -0800 HDFS-596. libhdfs does not free memory for mOwner and mGroup. commit 3894ed5bab17e1fd4661768149a4dd2f857b394c Author: Chad Metcalf Date: Tue Nov 10 17:05:00 2009 -0800 HDFS-677. Rename failure due to quota results in deletion of src directory commit 3567cc7fcd735740d0da0c513e835ae4e696d53b Author: Chad Metcalf Date: Tue Nov 10 17:00:03 2009 -0800 Fix OOM error in ReducerTask due to signed int overflow. commit 5d13e9a440f0389437c9be97d7daa68e7777bdb9 Author: Chad Metcalf Date: Tue Nov 10 16:59:35 2009 -0800 Revert "MAPREDUCE-967. TaskTrackers do not fully unjar job.jar" commit b9954debd1d9aae92fa1a070d152106350fbd5be Author: Chad Metcalf Date: Tue Nov 10 16:58:56 2009 -0800 HDFS-732. HDFS files are ending up truncated commit b97475058c5f90772ec69061887a533396cffc9f Author: Chad Metcalf Date: Tue Nov 10 16:58:23 2009 -0800 HADOOP-5640 puts this in the new test dir. It needs to be in the old one. commit e9a2f7439c55cf084e681500aca9cff8aa6c7c68 Author: Chad Metcalf Date: Tue Nov 10 16:56:34 2009 -0800 MAPREDUCE-1017. Compression and output splitting for Sqoop commit e05a77fc7ed81ad6a7060fcf0e7f6decb9b1f55a Author: Chad Metcalf Date: Tue Nov 10 16:55:41 2009 -0800 HADOOP-6312. Configuration sends too much data to log4j commit 5d84477c86661fe46372decaffac86ad61843628 Author: Chad Metcalf Date: Tue Nov 10 16:54:53 2009 -0800 HADOOP-5759. IllegalArgumentException when CombineFileInputFormat is used as job InputFormat commit 0d863ee05d559901d7c305611b2d20ff8514a933 Author: Todd Lipcon Date: Tue Nov 10 12:16:14 2009 -0800 HDFS-686. NullPointerException is thrown while merging edit log and image commit 418c903b6c350e1af9de96a722aa48e732e34048 Author: Todd Lipcon Date: Wed Oct 28 11:33:27 2009 -0700 MAPREDUCE-112. Add counters for reduce input, output records to the new API. Contributed by Jothi Padmanabhan commit 38c32cc6eeacc4bb2137b0b07b3f5ce2747e6981 Author: Todd Lipcon Date: Wed Oct 28 11:23:59 2009 -0700 MAPREDUCE-768. Add hadoop jobtracker -dumpConfiguration command This is modified from the original patch in that it does not dump QueueManager configuration. This is because we have not backported HADOOP-5396 commit 19654b90fa67491e5b4a6c51f67b5452d62b754f Author: Todd Lipcon Date: Wed Oct 28 11:10:41 2009 -0700 HADOOP-6184. Provide a configuration dump in json format. commit ac41af8a2c74e34b672f43b55a61ac2ebd76d1ff Author: Chad Metcalf Date: Mon Oct 26 11:39:07 2009 -0700 Duplicated effort. FULL_VERSION already set in package.mk Revert "Need to pass in FULL_VERSION" This reverts commit f8e6f5c017e79223934e3333ad96d84dbbc0fcbb. commit 31a79c62df9bcf785290c1000e1d61b9a0caf0f1 Author: Chad Metcalf Date: Mon Oct 26 10:56:44 2009 -0700 Copy the scoop man page to the expected version number. commit cb3092588975396fa5628608f0671a42abd380c0 Author: Chad Metcalf Date: Fri Oct 16 14:07:45 2009 -0700 Bump jdiff stable to 0.20.1 commit f8e6f5c017e79223934e3333ad96d84dbbc0fcbb Author: Chad Metcalf Date: Fri Oct 16 14:07:22 2009 -0700 Need to pass in FULL_VERSION commit 3d64e801bd9397dba27358f34e74ae5ded9122d1 Author: Aaron Kimball Date: Wed Oct 14 16:15:45 2009 -0700 MAPREDUCE-999. Improve Sqoop test speed and refactor tests commit df8cd1351848eef56455f3116f3970923e1426c3 Author: Todd Lipcon Date: Fri Oct 9 17:52:10 2009 -0700 MAPREDUCE-1089. Fix bug in Fair Scheduler where preemption would throw a NPE when facing tasks in "scheduled but not running" status commit 1a7b494d426f4a72329ab53d57dc67ecc76ca717 Author: Todd Lipcon Date: Wed Oct 7 15:33:37 2009 -0700 MAPREDUCE-551: Add preemption to the fair scheduler This fixes an error in the previous backport where the EagerTaskInitializationListener wasn't properly passed the TaskTrackerManager before starting. commit 9f74c6700adcc8b5749e1c6e7bca7fba70f24c86 Author: Todd Lipcon Date: Wed Oct 7 00:36:26 2009 -0700 MAPREDUCE-1070. Fix FairSchedulerServlet deadlock possibility. commit 35a5d7541e0bb46cdb35818a25f538ee4d496070 Author: Todd Lipcon Date: Sun Oct 4 09:54:59 2009 -0700 Fix hadoop wrapper to properly pass through multiword quoted arguments commit 9d8d1cb216d76b5ec72a1c0df195839441603df9 Author: Matt Massie Date: Fri Sep 25 15:02:56 2009 -0700 Sqoop documentation is now part of the generated tarball. Updated the install script to reflect that change. commit 220d1a70152b6526ded7b2843f88c63776bd6251 Author: Matt Massie Date: Fri Sep 25 13:55:19 2009 -0700 Generate the sqoop documentation and ensure that it's in the release tarball commit a29a2635b33e0b9ae03cdd089227d538ef2c8672 Author: Chad Metcalf Date: Thu Sep 24 16:07:18 2009 -0700 More changes to get debs building correctly commit b29653bb4757430033d5c97187df1903c1003d69 Author: Aaron Kimball Date: Thu Sep 24 14:54:48 2009 -0700 Reformatted Sqoop manpage asciidoc for CDH build process commit c955b2beda0ab1135c41bf26db302a94bd583fe9 Author: Todd Lipcon Date: Thu Sep 24 13:55:35 2009 -0700 Only rerun libtoolize if version 2.2 is installed commit 04a3f3d025b99184621910e2fc948ca3b200bfd5 Author: Todd Lipcon Date: Tue Sep 22 16:04:33 2009 -0700 HADOOP-6279. Add JVM memory usage to JvmMetrics commit 788443436f440255eb2d472a36d743511de630e5 Author: Todd Lipcon Date: Sun Sep 20 21:28:06 2009 -0700 HADOOP-6269. Fix synchronization on Configuration.class to avoid a possible deadlock commit a367eec549f4a76ff042ac0b3a97684cda038c63 Author: Todd Lipcon Date: Tue Sep 22 22:19:02 2009 -0700 Recreate c++ configure files during build if we have the right build dependencies commit a07c1fce55a0581bc9012e4b1bd5ed67c33bdf5e Author: Todd Lipcon Date: Tue Sep 22 20:06:40 2009 -0700 Revert "MAPREDUCE-551: Add preemption to the fair scheduler" This reverts commit 0e260191bd72e38361568b846b659933c5bcb6db. This patch as applied was found to be unstable and cause job submission failure. Will revisit and reapply after further testing. commit 66c65846c9017ccda6a154c755019b4bb972e3d8 Author: Chad Metcalf Date: Tue Sep 22 12:51:09 2009 -0700 Package sqoop docs w/o requiring asciidoc. commit 91b16081c4e375c1f897930036553a42c2f2e949 Author: Chad Metcalf Date: Tue Sep 22 11:23:43 2009 -0700 Revert "Package sqoop docs." This reverts commit 34a58c0e9727dd7af8e7f144709a1b234b63f142. commit 7bdbb68d5cd0dc411fbf6163817d3bd3b3589191 Author: Chad Metcalf Date: Wed Sep 16 15:54:13 2009 -0700 HADOOP-5891: If dfs.http.address is default, SecondaryNameNode can't find NameNode commit d9e9589506f8856fcb607cdf0465bad22f48a267 Author: Todd Lipcon Date: Mon Jul 20 13:07:43 2009 -0700 HDFS-127: DFSClient block read failures cause open DFSInputStream to become unusable HADOOP-4681: Also referenced This as-yet-uncommitted patch is recommended by HBase people. Applied patch "4681.patch" attached to the JIRA on 2008-11-18. commit 14fd272700ba3f3cf254c015f3eb0ecd795604e2 Author: Chad Metcalf Date: Thu Sep 10 14:44:12 2009 -0700 HADOOP-4655. New method FileSystem.newInstance() that always returns a newly allocated FileSystem object. (dhruba) git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@756152 13f79535-47bb-0310-9956-ffa450edef68 commit 9b82cd7ad13c852f5d5837e89ed0084f7353a78a Author: Todd Lipcon Date: Wed Sep 9 23:30:31 2009 -0700 MAPREDUCE-967. TaskTrackers do not fully unjar job.jar commit ca4d967a798d50e46a5749f607dfcef2cef89b9d Author: Chad Metcalf Date: Mon Sep 14 12:03:04 2009 -0700 MAPREDUCE-968: NPE in distcp encountered when placing _logs directory on S3FileSystem (Chad Metcalf for Aaron Kimball) commit f4248e0caad9ca6acf99058d6ace31ba5f3281d9 Author: Chad Metcalf Date: Tue Sep 15 22:21:42 2009 -0700 HADOOP-6133: ReflectionUtils performance regression commit 78be540d795d25b86973d6074c37873734c7ac6d Author: Chad Metcalf Date: Tue Sep 15 22:18:24 2009 -0700 HADOOP-6231: Allow caching of filesystem instances to be disabled on a per-instance basis commit 9849cf6a91393ad0e2228a4a329d1f76c3e581c0 Author: Chad Metcalf Date: Wed Sep 16 10:12:05 2009 -0700 HADOOP-6097: Multiple bugs w/ Hadoop archives commit 5c3e38f72cb742de4f81067d4239911c9f6f4fb5 Author: Chad Metcalf Date: Tue Sep 15 22:29:50 2009 -0700 HADOOP-5981: HADOOP-2838 doesnt work as expected commit 5a2c144f1a8f85c37f3a39a41d70de9f6b307a2c Author: Chad Metcalf Date: Tue Sep 15 22:09:57 2009 -0700 HADOOP-2838: Add HADOOP_LIBRARY_PATH config setting so Hadoop will include external directories for jni commit 0f6db246d4ecec3428b9c3b06e8e625ef273d471 Author: Chad Metcalf Date: Tue Sep 15 18:15:17 2009 -0700 MAPREDUCE-814: Move completed Job history files to HDFS commit 2d6bb7c55e33c09b4e89a47d0fae02d937d56e61 Author: Chad Metcalf Date: Tue Sep 15 17:59:39 2009 -0700 MAPREDUCE-693: Conf files not moved to done subdirectory after JT restart commit 1234649f462f677a0bb0454cf5a4caa5ac3e04c0 Author: Chad Metcalf Date: Tue Sep 15 17:54:11 2009 -0700 MAPREDUCE-683: TestJobTrackerRestart fails with Map task completion events ordering mismatch commit d2273ab392265bff99cf53ae1242f8b9548a4bc6 Author: Chad Metcalf Date: Tue Sep 15 17:52:19 2009 -0700 MAPREDUCE-416: Move the completed jobs' history files to a DONE subdirectory inside the configured history directory commit 9a4bdaa019e0ebebe399341e662b2f4f01fdd448 Author: Chad Metcalf Date: Tue Sep 15 17:45:49 2009 -0700 HADOOP-5733: Add map/reduce slot capacity and lost map/reduce slot capacity to JobTracker metrics commit 87782238f5d367b3d6403949e3d517eafb8b096b Author: Chad Metcalf Date: Tue Sep 15 17:44:04 2009 -0700 HADOOP-5738: Split waiting tasks field in JobTracker metrics to individual tasks commit 8a576eb8207d0de79d3bf69354dd3a9bda417d14 Author: Chad Metcalf Date: Tue Sep 15 17:26:37 2009 -0700 HADOOP-5442: The job history display needs to be paged commit ab167413303ac6e306083174e98fa62e471962d5 Author: Chad Metcalf Date: Tue Sep 15 17:18:54 2009 -0700 HADOOP-4842: Streaming combiner should allow command, not just JavaClass commit 402e34faf0202092b65f944df3ff3e104fdd9d39 Author: Philip Zeyliger Date: Tue Sep 15 01:19:54 2009 -0700 MAPREDUCE-987 (nee HDFS-621). Add ability to start MiniClusters from the command line commit 8284c1317062a166b3e54668f07fdf9d1ac7fd8c Author: Todd Lipcon Date: Tue Sep 15 01:19:54 2009 -0700 HADOOP-6267. Make build-contrib.xml reference relative paths using a dirname property This allows contrib projects to be built out-of-tree commit e55f8563ece9296640e82e7e4aaf779d74054474 Author: Todd Lipcon Date: Mon Sep 14 23:57:33 2009 -0700 HDFS-528: add ability for safemode to wait until datanodes have registered commit 01a80cf127504afa923eb262eeaae04a6fbc9e5e Author: Konstantin Shvachko Date: Wed Dec 31 07:05:11 2008 +0000 HADOOP-4936. Improvements to TestSafeMode. Contributed by Konstantin Shvachko. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@730331 13f79535-47bb-0310-9956-ffa450edef68 commit a0a976cee9f07f2af8cf182a72b196a2bf25973e Author: Tsz-wo Sze Date: Sat Apr 18 00:03:28 2009 +0000 HADOOP-5650. Fix safemode messages in the Namenode log. Contributed by Suresh Srinivas git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@766182 13f79535-47bb-0310-9956-ffa450edef68 commit 63a46ad91b2088fbc814a63a06d196d12654b334 Author: Philip Zeyliger Date: Tue Jul 28 15:28:18 2009 -0700 HADOOP-4675. Current Ganglia metrics implementation is incompatible with Ganglia 3.1 Patched using https://issues.apache.org/jira/secure/attachment/12407207/HADOOP-4675-v7.patch commit 59b78a17699646b273176b6cc6115620cd2e74a1 Author: Henry Robinson Date: Wed Jul 22 14:49:12 2009 +0100 HADOOP-6269. Fix missing synchronization around Configuration.defaultResources commit 3667d72554e3423f901c5da43558b1717a7c87df Author: Henry Robinson Date: Thu Jul 9 09:33:30 2009 +0100 Make some JT methods and classes public for use from within contrib plugins commit ffe137becc524954183b5f68b385f9aaf58927a7 Author: Todd Lipcon Date: Mon Sep 14 23:29:06 2009 -0700 MAPREDUCE-461 (incomplete on trunk). Add plugin support to JobTracker (Relies on HADOOP-5640) commit a11703e4bc485454840d17ade209f4bc851617d7 Author: Todd Lipcon Date: Wed Apr 15 17:12:19 2009 -0700 HADOOP-5640: Add dispatch mechanism for services to notify plugins of important events. commit 31fa6f8fc7b70354a18626159f5bc16890e26639 Author: Dhruba Borthakur Date: Fri Apr 3 05:44:52 2009 +0000 HADOOP-5257. HDFS servers may start and stop external components through a plugin interface. (Carlos Valiente via dhruba) git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@761530 13f79535-47bb-0310-9956-ffa450edef68 commit f3094dc85433914fa61e93718ea645aaf1eea4d8 Author: Aaron Kimball Date: Thu Sep 17 13:14:19 2009 -0700 MAPREDUCE-971: distcp does not always remove distcp.tmp.dir commit 34a58c0e9727dd7af8e7f144709a1b234b63f142 Author: Chad Metcalf Date: Tue Sep 15 16:24:19 2009 -0700 Package sqoop docs. commit 424cd9d505716efdba61cc4e91471a2718a910ad Author: Aaron Kimball Date: Tue Sep 15 15:13:23 2009 -0700 MAPREDUCE-923: Sqoop's ORM uses URLDecoder on a file, which replaces plus signs in a jar file name with spaces commit 68596d06c2821caf802e8debb59b173e0e896e29 Author: Aaron Kimball Date: Tue Sep 15 14:24:53 2009 -0700 Sqoop asciidoc syntax error commit ac0753f008e774d9271c5881329b5e18f37e7635 Author: Chad Metcalf Date: Mon Sep 14 20:11:00 2009 -0700 HADOOP-5450: Add support for application-specific typecodes to typed bytes commit b64d05788bd192e54d81e47d6c049d9e4eba4b75 Author: Chad Metcalf Date: Mon Sep 14 20:08:23 2009 -0700 HADOOP-1722: Make streaming to handle non-utf8 byte array commit 59dcf878187a3fa1cbc64dba7ddb49624861a6b4 Author: Aaron Kimball Date: Tue Sep 15 11:05:32 2009 -0700 More Sqoop documentation updates commit 28778e6442ac75d429ecf364e845dbfa65fc8f18 Author: Aaron Kimball Date: Mon Sep 14 19:54:21 2009 -0700 MAPREDUCE-840: DBInputFormat leaves open transaction (reapplied after HADOOP-4687) commit a7b776d5209cbf031ad9a39571f4a989601371e2 Author: Aaron Kimball Date: Mon Sep 14 18:05:09 2009 -0700 MAPREDUCE-906: Updated Sqoop documentation commit 9bd36193812f8b9fba44c500720259358af299b4 Author: Aaron Kimball Date: Mon Sep 14 18:02:16 2009 -0700 MAPREDUCE-907: Sqoop should use more intelligent splits commit 272ca62c4ebc90a58078fad0811aadeae503ef21 Author: Aaron Kimball Date: Mon Sep 14 18:00:52 2009 -0700 MAPREDUCE-885: More efficient SQL queries for DBInputFormat commit b0342f52ebe13d3f887c7a117d29b0ad5c007be7 Author: Aaron Kimball Date: Mon Sep 14 17:59:53 2009 -0700 MAPREDUCE-938: Postgresql support for Sqoop commit d550e91e79aeee4b75522f8244b40e77625458d5 Author: Aaron Kimball Date: Mon Sep 14 17:58:46 2009 -0700 MAPREDUCE-876: Sqoop import of large tables can time out commit e7032e9a71a76966332534d33316de9f3fec7839 Author: Aaron Kimball Date: Mon Sep 14 17:55:37 2009 -0700 MAPREDUCE-918: Test hsqldb server should be memory-only commit 1e4cb415dbe1631735e25479afa16456ff86d30f Author: Aaron Kimball Date: Mon Sep 14 17:54:37 2009 -0700 MAPREDUCE-875: Make DBRecordReader execute queries lazily commit fe2b40eba88164fb85e84180ebf92e6b8f85defd Author: Aaron Kimball Date: Mon Sep 14 17:54:04 2009 -0700 MAPREDUCE-825: JobClient completion poll interval of 5s causes slow tests in local mode commit c1d48d5d45d2262cd410d240b056ddc1a31a6880 Author: Aaron Kimball Date: Mon Sep 14 17:48:35 2009 -0700 MAPREDUCE-750: Extensible ConnManager factory API commit 5f1c3f457eac0ac3185340e02106a79bcb3c9b26 Author: Aaron Kimball Date: Mon Sep 14 17:47:31 2009 -0700 MAPREDUCE-749: Make Sqoop unit tests more Hudson-friendly commit 6f476cfac44d6539ecdfefd5d9c6e7805328487b Author: Aaron Kimball Date: Mon Sep 14 17:42:18 2009 -0700 MAPREDUCE-792: Fix unchecked warnings in DBInputFormat commit 1561cf1488502b4a8f845ec177db8ab4365ceff2 Author: Aaron Kimball Date: Mon Sep 14 17:40:22 2009 -0700 MAPREDUCE-716: org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle (Applied "trunk" version of the patch after incorporating HADOOP-4687's move of DBIF) commit 89e9d0db914fdcd3dd3459e986796361e53ed549 Author: Aaron Kimball Date: Mon Sep 14 17:39:13 2009 -0700 MAPREDUCE-910: MRUnit should support counters commit 7a34c6b3b662dcd529a864c0540ce111c5b8bd43 Author: Aaron Kimball Date: Mon Sep 14 17:34:47 2009 -0700 MAPREDUCE-798: MRUnit should be able to test a succession of MapReduce passes commit 97c5866d78b016e8db06a89baf77eb443ac5e063 Author: Aaron Kimball Date: Mon Sep 14 17:33:34 2009 -0700 MAPREDUCE-800: MRUnit should support the new API commit 68794a3e70ca2e88833a4e02ccb6469b5a66e118 Author: Aaron Kimball Date: Mon Sep 14 17:31:31 2009 -0700 MAPREDUCE-799: Some of MRUnit's self-tests were not being run commit 819b7876a020c451b5236ad6ad7d78a1ae5f38bf Author: Aaron Kimball Date: Mon Sep 14 17:30:33 2009 -0700 MAPREDUCE-797: MRUnit MapReduceDriver should support combiners commit 30c44373abe6c0f0ce83f601e9b409858165e825 Author: Aaron Kimball Date: Mon Sep 14 17:28:37 2009 -0700 Use 0.21-branch versions of DBInputFormat + shim from HADOOP-4687 commit 0e260191bd72e38361568b846b659933c5bcb6db Author: Aaron Kimball Date: Mon Sep 14 17:24:29 2009 -0700 MAPREDUCE-551: Add preemption to the fair scheduler commit 951c5910345474848c25b3d372b2c34480d728f2 Author: Aaron Kimball Date: Mon Sep 14 16:51:19 2009 -0700 HADOOP-5170: Set max map/reduce tasks on a per-job basis, either per-node or cluster-wide commit 4064b191c14aa98deca22155531db1830867f1c2 Author: Todd Lipcon Date: Wed Aug 26 11:38:09 2009 -0700 HADOOP-6166. Further improve the performance of the pure-Java CRC32 implementation. Contributed by Tsz Wo (Nicholas), SZE commit fd77206d56ba4fe544a48c6c2f31682b430bf4dd Author: Todd Lipcon Date: Wed Aug 26 11:36:23 2009 -0700 MAPREDUCE-782. Use PureJavaCrc32 in SpillRecord (Todd Lipcon via szetszwo) commit 8bf4c9f9c6b1543161b9db6be443e7a131c8d22d Author: Todd Lipcon Date: Wed Aug 26 11:34:51 2009 -0700 HDFS-496. Use PureJavaCrc32 in HDFS (Todd Lipcon via szetszwo) commit 99d8dfc7f401d8f9c2469b67e56240a33a0152a9 Author: Todd Lipcon Date: Wed Aug 26 11:32:55 2009 -0700 HADOOP-6148. Implement a fast, pure Java CRC32 calculator which outperforms java.util.zip.CRC32. Contributed by Todd Lipcon and Scott Carey commit a65ce17e5bfe3376d6c66cc72eba2adca7f14dee Author: Aaron Kimball Date: Thu Aug 13 16:20:41 2009 -0700 Updated Sqoop documentation for MAPREDUCE-816, MAPREDUCE-789 commit d17d51722cbefebf2ae917d4e48218ee87bee185 Author: Aaron Kimball Date: Thu Aug 13 15:44:36 2009 -0700 MAPREDUCE-789: Oracle support for Sqoop commit 4de0b29c1996d1bdd64cbb3e66ba38091ab519d4 Author: Aaron Kimball Date: Thu Aug 13 15:40:16 2009 -0700 MAPREDUCE-840: DBInputFormat leaves open transaction commit 9e87cf9cee2dc8c2f827df77f6ca186045f1f5c3 Author: Aaron Kimball Date: Thu Aug 13 15:38:21 2009 -0700 MAPREDUCE-816: Rename "local" mysql import to "direct" commit bb7781980a000dcda2efc8e863b93fe1f31d46f2 Author: Aaron Kimball Date: Thu Aug 13 15:37:25 2009 -0700 MAPREDUCE-716: org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle Applied MAPREDUCE-716.4.branch20.patch commit 00bccbbe823578bd483dc67bca22838bea79ddcd Author: Philip Zeyliger Date: Mon Jun 15 18:37:24 2009 -0700 Example hadoop-metrics.properties works with /metrics (HADOOP-5469). commit 27128c705078137d708949076421787cbd63b029 Author: Philip Zeyliger Date: Mon Jun 15 18:03:03 2009 -0700 HADOOP-5469. Add /metrics servlet to daemons, providing metrics over HTTP as either text or JSON. Contributed by Philip Zeyliger. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@762987 13f79535-47bb-0310-9956-ffa450edef68 commit 2302d90cb656757447da9671abe63a0f0164eb57 Author: Todd Lipcon Date: Mon Aug 3 12:26:26 2009 -0700 Added Sqoop documentation to installation script commit 3cc8a9ca1f70f598d84e38b5620db642d121659b Author: Matt Massie Date: Fri Jul 31 19:09:46 2009 -0700 Fix the hadoop/sqoop wrapper scripts commit 316a9417cba29d34f38ee71009dec219a3dc4b5f Author: Matt Massie Date: Fri Jul 31 18:45:00 2009 -0700 Fix a bug in the hadoop/sqoop wrapper generation commit eddd4b80f856fe92195cab9d5c532e2af9e9c482 Author: Matt Massie Date: Fri Jul 31 17:44:43 2009 -0700 Update the install hadoop script commit 02e575801414c8fdddbc9a6678bfa493a1b42e0b Author: Matt Massie Date: Fri Jul 31 17:39:50 2009 -0700 Rename the hadoop man page to be hadoop-0.20 commit 081a9ef0a427ad15f08f0bea1da734ecfbaef085 Author: Todd Lipcon Date: Mon Jul 20 18:10:24 2009 -0700 HADOOP-5745. Allow setting the default value of maxRunningJobs for all pools. Contributed by Dhruba Borthakur. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@771570 13f79535-47bb-0310-9956-ffa450edef68 commit d8ccfe192bb2482e6e3e7c21c2ffadd4c6895ee0 Author: Aaron Kimball Date: Sat Jul 25 16:52:32 2009 -0700 Update Sqoop documentation with user guide and manpage. commit 9e8ee41f9157c5603198f1bfec6efcd2b3abf27b Author: Aaron Kimball Date: Thu Jul 23 16:45:29 2009 -0700 Added MySQL Connector/J library for Sqoop. commit 3b81740964543fa0504e44dbbe04d0cc11fdfe56 Author: Aaron Kimball Date: Thu Jul 23 16:25:04 2009 -0700 MAPREDUCE-705: User-configurable quote and delimiter characters for Sqoop records and record reparsing commit 8a7e4f7fa90de2caa9023f27ae0dbf6d79e1915c Author: Aaron Kimball Date: Thu Jul 23 16:24:06 2009 -0700 MAPREDUCE-710: Sqoop should read and transmit passwords in a more secure manner commit 05fc9b0e6a934a0e8f569c670cb0d266a02b59a4 Author: Aaron Kimball Date: Thu Jul 23 16:23:01 2009 -0700 MAPREDUCE-713: Sqoop has some superfluous imports commit 35536eb4bdc90a0575d94359e6ba6554dca7ccbb Author: Aaron Kimball Date: Thu Jul 23 16:21:51 2009 -0700 MAPREDUCE-685: Sqoop will fail with OutOfMemory on large tables using mysql commit 56c76edabf47002cbc739a26618720b191a7a258 Author: Aaron Kimball Date: Thu Jul 23 16:20:39 2009 -0700 MAPREDUCE-674: Sqoop should allow a "where" clause to avoid having to export entire tables commit d92082012d7fa3a98101bc909fc32643ac3417ed Author: Aaron Kimball Date: Thu Jul 23 16:18:51 2009 -0700 MAPREDUCE-675: Sqoop should allow user-defined class and package names commit a71a875925ce7f4a355262c911a5913a5cb72843 Author: Aaron Kimball Date: Thu Jul 23 16:16:30 2009 -0700 MAPREDUCE-703: Sqoop requires dependency on hsqldb in ivy commit 0fe64cbc39a58f47b9facf613b9e11cd6f5d8ebe Author: Aaron Kimball Date: Thu Jul 23 16:12:20 2009 -0700 MAPREDUCE-692: Make Hudson run Sqoop unit tests commit 3d62ea0666884b7afeeeacd47ec9f5806c1698e6 Author: Aaron Kimball Date: Thu Jul 23 16:10:59 2009 -0700 HADOOP-5968: Sqoop should only print a warning about mysql import speed once commit b63a6286be32b54335c292ac97959f8036bf738b Author: Aaron Kimball Date: Thu Jul 23 16:10:21 2009 -0700 HADOOP-5967: Sqoop should only use a single map task commit dcfc5ac78db7e75193b2d82b2e9441dabd08cced Author: Aaron Kimball Date: Thu Jul 23 16:09:30 2009 -0700 HADOOP-5887: Sqoop should create tables in Hive metastore after importing to HDFS commit 3c65058ccc8e5ddc78c9438ddec7b268349571a4 Author: Todd Lipcon Date: Tue Jul 21 16:18:18 2009 -0700 Address issue where docs were not properly copied through to release tarball This was caused by some cleanup in build.xml early on in the CDH 0.20 branch commit b535db08cea3e642aec046d146e528767e1187f6 Author: Todd Lipcon Date: Tue Jul 21 15:42:23 2009 -0700 Decrease build time by only rebuilding the native code for each platform commit 4f9b46e691c8f9dca8256064131d1f0f8088df0e Author: Todd Lipcon Date: Tue Jul 21 14:36:22 2009 -0700 Run jdiff against vanilla Hadoop during Cloudera release build commit 5d77a82370260b78d6c9e5bfdb542bee13477301 Author: Aaron Kimball Date: Tue Jul 21 17:43:45 2009 -0700 MAPREDUCE-415: JobControl.Job always has an unassigned name commit 104f9abe67d1e35816129003413d46d5df1222f2 Author: Thomas White Date: Thu May 28 16:43:30 2009 +0000 HADOOP-5805. problem using top level s3 buckets as input/output directories. Contributed by Ian Nowland. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@779656 13f79535-47bb-0310-9956-ffa450edef68 commit 7108ee8e0ae2ea4cbca67a6226314556486980ff Author: Thomas White Date: Fri May 1 13:43:10 2009 +0000 HADOOP-5656. Counter for S3N Read Bytes does not work. Contributed by Ian Nowland. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@770681 13f79535-47bb-0310-9956-ffa450edef68 commit ba62b9ccb9c4914a76d6acb244722c21b554b733 Author: Todd Lipcon Date: Thu Jul 16 16:21:59 2009 -0700 HADOOP-5613. Change S3Exception to checked exception. Contributed by Andrew Hitchcock. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@770685 13f79535-47bb-0310-9956-ffa450edef68 commit 12e86e55d8ee04de28e9e07aa91701cc4a04e56b Author: Konstantin Shvachko Date: Thu Apr 2 20:08:21 2009 +0000 HADOOP-5528. Add a configurable hash partitioner operating on ranges of BinaryComparable keys. Contributed by Klaas Bosteels. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@761407 13f79535-47bb-0310-9956-ffa450edef68 commit 61636a356881670554cc3d1b9b9748a68347cad5 Author: Douglass Cutting Date: Mon Feb 23 22:30:34 2009 +0000 HADOOP-5240. Skip re-building javadoc when it is already up-to-date. Contributed by Aaron Kimball. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@747169 13f79535-47bb-0310-9956-ffa450edef68 commit 7d245d8f29151c8c802ba15c078acb3a7f56bbf0 Author: Thomas White Date: Wed May 27 11:00:11 2009 +0000 HADOOP-5175. Don't unpack libjars on classpath. Contributed by Todd Lipcon. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@779106 13f79535-47bb-0310-9956-ffa450edef68 commit 3a4cdcc07b94c1cb26949e626e52f95674d90a59 Author: Todd Lipcon Date: Thu Jul 16 15:15:18 2009 -0700 Fix scribe-log4j's ivy.xml to properly get log4j on the compile classpath commit 4874fe48d0f837a326e39958a817871d5b31be4c Author: Thomas White Date: Thu May 21 15:22:47 2009 +0000 HADOOP-4829. Allow FileSystem shutdown hook to be disabled. Contributed by Todd Lipcon. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@777152 13f79535-47bb-0310-9956-ffa450edef68 commit e39c01ccc1658be2ccbd17bfed5a5bf4b76951e1 Author: Devaraj Das Date: Thu Feb 5 05:35:09 2009 +0000 HADOOP-3327. Improves handling of READ_TIMEOUT during map output copying. Contributed by Amareshwari Sriramadasu. git-svn-id: https://svn.apache.org/repos/asf/hadoop/core/trunk@741009 13f79535-47bb-0310-9956-ffa450edef68 commit 4036ae240690d38dbeac52d7094fa078ff9b2160 Author: Aaron Kimball Date: Wed Jul 15 10:40:47 2009 -0700 MAPREDUCE-680. Reuse of Writable objects is improperly handled by MRUnit. commit efa77a3d1fce955acdca4978014de1eb950d1abd Author: Aaron Kimball Date: Wed Jul 15 10:40:15 2009 -0700 HADOOP-5518. MRUnit test library. commit 8da767bb6faefe26fcbad7d28b3ead8ede80a457 Author: Todd Lipcon Date: Tue Jun 2 20:55:17 2009 -0700 Add sqoop wrapper to bin/ commit 860bf603c83792808365b675a8386be1e0c6afea Author: Todd Lipcon Date: Tue Jun 2 20:06:18 2009 -0700 HADOOP-5844: Use mysqldump when connecting to local mysql instance in Sqoop Patch: mysqldump.patch 2009-5-14 commit 950f96b7e4e393e842cbc3648c2012e24e624893 Author: Todd Lipcon Date: Tue Jun 2 20:05:10 2009 -0700 HADOOP-5815: Sqoop: A database import tool for Hadoop Patch added: HADOOP-5815.2.patch 2009-05-13 version commit c6884be8bded19d36ffa01163473abdac877c66d Author: Todd Lipcon Date: Mon Jul 6 19:36:11 2009 -0700 MAPREDUCE-714: Fix JobConf.findContainingJars to work with jars with + in the name commit 4f40345e350694c1dc519fab5cf8f28ed17628fe Author: Todd Lipcon Date: Mon May 11 14:35:00 2009 -0700 Add libs commit 8284e87eea9ca99a9f0a1a43ba84ed6efd521d5e Author: Todd Lipcon Date: Mon May 11 14:32:38 2009 -0700 scribe_hadoop_trunk.patch Also, add empty ivy infrastructure for scribe-log4j commit 603998f595bab785738c1b220a419e606d88c3bf Author: Todd Lipcon Date: Wed Jul 15 15:30:14 2009 -0700 Use cloudera's versioning info from cloudera.hash in saveVersion.sh This should make the "hadoop version" output far more useful for determing exactly what code is running. The cloudera.hash property is set by cloudera/build.properties which is generated during the build process. commit 57b892f99b4c22169f13f12013ad679ad596f3ca Author: Todd Lipcon Date: Tue Jun 30 11:12:55 2009 -0700 Move saveVersion.sh in build.xml to ensure build This error is due to ant 1.7.1 not compiling package-info.java if the timestamp of the output class directory is newer than the package-info file itself. Since other compiles were happening after package-info.java was generated, the build dir was newer and compilation was being skipped. Fixes CLOUDERA-436 (hadoop version printing rUNKNOWN) Move cloudera hooks inside the package task of build.xml Fixes an issue where the fair scheduler jar was not built before the hooks were run, and therefore was not included in the target lib/ directory. commit e19297590d946e29188d0b242c119083508e036e Author: Todd Lipcon Date: Wed Jul 15 14:23:13 2009 -0700 New versions of cloudera scripts commit 7b2a0502fb15815ead220855b53a21ba41a82db3 Author: Todd Lipcon Date: Fri Jun 5 17:21:14 2009 -0700 Change paths to point to hadoop-0.20 where necessary commit db837e1302d76b505767715606484b446cac1e7d Author: Todd Lipcon Date: Fri Jun 5 16:47:56 2009 -0700 Add manpage to 20 repo commit f238378e945ca7d4c56cdd8e4b852a7d36599ddd Author: Todd Lipcon Date: Fri Jun 5 16:46:46 2009 -0700 Move install_hadoop.sh into hadoop repo commit 5f9f8cadd562459bb095c114389ea2c9e32fe77c Author: Todd Lipcon Date: Wed Jun 3 12:11:53 2009 -0700 Add example-confs directory for storing configuration of conf.pseudo commit d8aeb49e504965dc0296b385f2c340942ce159ab Author: Todd Lipcon Date: Tue Jun 2 21:12:45 2009 -0700 Replace hadoop-config.sh with Cloudera version commit d8b79d1efa26a48487d8d1e71215849068f8efdb Author: Todd Lipcon Date: Tue Jun 2 19:34:56 2009 -0700 Remove redundant code in build.xml between package and bin-package commit 398b2e9c22e3fa20fcb63d17af31d100e7f46d31 Author: Todd Lipcon Date: Mon May 11 14:32:42 2009 -0700 HADOOP-5611-Add-some-missing-includes-to-c-code-t.patch commit aea0e88dab50448ea764693beda8d4d01b9b7731 Author: Todd Lipcon Date: Tue Jun 2 17:57:09 2009 -0700 Hook build.xml to enable contrib modules commit 22e0fb039e00f7a7f714678c2562aadfe7326d0a Author: Todd Lipcon Date: Tue Jun 2 14:50:56 2009 -0700 HADOOP-5612: Add chmod rules to build.xml