Release Notes - Whirr - Version 0.5.0

Sub-task

  • [WHIRR-277] - Support multiple versions of ZooKeeper
  • [WHIRR-279] - Create ClusterSpec aware BlobStoreContext factory class
  • [WHIRR-280] - Create a blob cache that could be used for storing local files
  • [WHIRR-292] - Separate Cassandra install and configuration scripts into more generic functions
  • [WHIRR-296] - Separate Voldemort install and configuration scripts into more generic functions
  • [WHIRR-297] - Separate ZooKeeper and ElasticSearch install and configuration scripts into more generic functions

Bug

  • [WHIRR-172] - Log warning for unrecognized service names
  • [WHIRR-253] - ZooKeeper service should only authorize ingress to ZooKeeper instances
  • [WHIRR-268] - whirr hangs when the file '$HOME/.ssh/known_hosts' includes an obsolete identifier for a certain ip address host.
  • [WHIRR-271] - Classpath needs to be quoted in whirr script
  • [WHIRR-274] - Add wagon-ssh-external as a maven build extension
  • [WHIRR-298] - Use all cluster spec properties for hash and equality
  • [WHIRR-312] - Destroy instance removes all entries from the instances file except the one that is being terminated
  • [WHIRR-314] - HBase integration test can fail due to Thrift server race

Improvement

  • [WHIRR-61] - make more efficient use of ComputeServiceContext
  • [WHIRR-173] - Add ClusterAction for generic script execution
  • [WHIRR-216] - Improve error message if whirr.instance-templates left out of config
  • [WHIRR-222] - Support multiple versions of Hadoop
  • [WHIRR-236] - Update Configuration Guides with Recipe Info
  • [WHIRR-245] - Clearly demarcate the user and service provider APIs
  • [WHIRR-246] - Single place to store/load cluster state
  • [WHIRR-262] - Services should not have to do reverse DNS lookups
  • [WHIRR-269] - improve error msg "Key pair is encrypted"
  • [WHIRR-275] - Improve firewall API for services
  • [WHIRR-278] - Refactor ClusterSpec and extract InstanceTemplate class
  • [WHIRR-282] - Set number of Hadoop slots based on hardware
  • [WHIRR-283] - Whirr in 5 minutes
  • [WHIRR-284] - Runurl should only be installed when needed
  • [WHIRR-288] - Add blob store persistence for cluster state
  • [WHIRR-289] - Display role names in list-cluster command
  • [WHIRR-291] - Add "noop" role useful just for provisioning
  • [WHIRR-299] - Recipe for BYON provider
  • [WHIRR-300] - FAQ entry for noop role
  • [WHIRR-304] - Upgrade to jclouds 1.0-beta-9c
  • [WHIRR-310] - Improve Configuration Guide

New Feature

  • [WHIRR-191] - [CDH] Start other services based on CDH, not just HDFS and MR
  • [WHIRR-220] - Support local tarball upload
  • [WHIRR-237] - Add Voldemort as a service
  • [WHIRR-261] - Add ElasticSearch as a service
  • [WHIRR-285] - Add support for BYON

Task

Test

  • [WHIRR-287] - Script for running YCSB on HBase

Release Notes - Whirr - Version 0.4.0

Sub-task

  • [WHIRR-139] - upgrade to version 1 of the "enforcer" plugin

Bug

  • [WHIRR-170] - Instances should be started in the order specified in the template
  • [WHIRR-201] - [HBase] Integration test fails
  • [WHIRR-207] - Handle curl timeouts better
  • [WHIRR-217] - Log files should not be included in tarball or checked by RAT
  • [WHIRR-227] - CDH and Hadoop integration tests are failing
  • [WHIRR-232] - NPE for stopped instances on EC2
  • [WHIRR-234] - Resource functions/install_cdh_hadoop.sh not found when running from the CLI
  • [WHIRR-235] - fix whirr.provider in recipes/*
  • [WHIRR-241] - Update to use CDH3B4
  • [WHIRR-247] - Add license headers to service install and configure scripts
  • [WHIRR-250] - Ensure all libraries in binary distribution have associated licenses where stipulated
  • [WHIRR-251] - Handle Apache cryptography requirements for release
  • [WHIRR-263] - Default tarball not found for Cassandra (broken link in install_cassandra.sh)
  • [WHIRR-264] - JClouds is unable to do SSH on automatically selected images
  • [WHIRR-267] - Update NOTICE and LICENSE files to mention 3rd party products

Improvement

  • [WHIRR-55] - Users should be able to override an arbitrary Hadoop property before launch
  • [WHIRR-124] - Upgrade to jclouds 1.0-beta-9
  • [WHIRR-167] - Improve bootstrapping and configuration to be able to isolate and repair or evict failing nodes on EC2
  • [WHIRR-183] - ZooKeeper Data Directory Cleanup
  • [WHIRR-186] - [HBase] Add version support configurable in properties file
  • [WHIRR-193] - Recipe for a HBase Cluster
  • [WHIRR-195] - Display available roles instead of service names when running ./bin/whirr
  • [WHIRR-199] - Add aliases for short role names like nn, jt, tt, dn, zk
  • [WHIRR-215] - Add builder pattern to addRunUrl() call
  • [WHIRR-219] - Support dynamic addition of services to CLI
  • [WHIRR-231] - Update documentation after upgrade to jclouds 1.0-beta-9
  • [WHIRR-233] - Change test properties to be less provider bound and improve TemplateBuilder defaults
  • [WHIRR-242] - Update documentation for overriding locally supplied scripts
  • [WHIRR-244] - Add package-level javadoc
  • [WHIRR-254] - Document limitation that a role may only appear in one instance template group
  • [WHIRR-259] - Disable configuration list handling for Hadoop properties
  • [WHIRR-265] - Missing SVN EOL properties

New Feature

  • [WHIRR-158] - Allow users to log into clusters as themselves
  • [WHIRR-198] - support user-defined images
  • [WHIRR-225] - Support locally-supplied scripts
  • [WHIRR-226] - Add the ability to destroy a cluster instance

Task

  • [WHIRR-129] - Add Adrian Cole as a committer in the whirr pom.xml and site.
  • [WHIRR-248] - Update to jclouds-1.0-beta9b

Release Notes - Whirr - Version 0.3.0

Bug

  • [WHIRR-123] - Cassandra integration tests hang if whirr's scripts bucket is missing
  • [WHIRR-127] - binary assembly from WHIRR-100 is not generated as part of release process
  • [WHIRR-128] - In ec2 instances instead of public dns names a public ip address is resolved for the started master node which causes workers to not be able to connect back to the master
  • [WHIRR-137] - Allow use of an arbitrary AMI on EC2
  • [WHIRR-146] - Changing the mapred.child.java.opts value does not change the heap size from a default one.
  • [WHIRR-147] - Regression on launching clusters from EC2
  • [WHIRR-151] - Credentials not set correctly for Hadoop service configure step
  • [WHIRR-156] - Cli script doesn't launch post-modularization
  • [WHIRR-159] - Cassandra and ZooKeeper fail on Ubuntu on Rackspace
  • [WHIRR-162] - DnsUtilTest fails when offline or for slow connections
  • [WHIRR-164] - Tests fail if there is no ~/.ssh/id_rsa keypair
  • [WHIRR-165] - Hadoop integration tests fail due to WHIRR-160 changes
  • [WHIRR-175] - ZooKeeper service does not honor instance roles
  • [WHIRR-179] - [Hadoop] Guard $MOUNT/tmp mkdir call against existing directory
  • [WHIRR-180] - ListClusterCommand.run throws a NullPointerException for unrecognized service names
  • [WHIRR-185] - [ZooKeeper] Fix selection of instances for getHosts() call
  • [WHIRR-200] - Cassandra integration test hangs
  • [WHIRR-201] - HBase integration test fails
  • [WHIRR-204] - CDH Hadoop integration test fails on Rackspace
  • [WHIRR-205] - Override service.provider for integration tests
  • [WHIRR-209] - bin/whirr fails on a new release
  • [WHIRR-210] - Remove unneeded dependent libraries
  • [WHIRR-211] - Fix checkstyle errors for 0.3.0

Improvement

  • [WHIRR-87] - Parallelize Hadoop cluster creation
  • [WHIRR-115] - Distribution should include documentation
  • [WHIRR-140] - include KEYS file in release artifact.
  • [WHIRR-145] - Add Whirr recipes for common configurations
  • [WHIRR-150] - Allow retrieval of instance roles
  • [WHIRR-153] - Add documentation for WHIRR-87 (Parallelize Hadoop cluster creation)
  • [WHIRR-154] - cassandra: expose RPC port, as well
  • [WHIRR-155] - Support multiple versions of Cassandra
  • [WHIRR-157] - Remove service name property
  • [WHIRR-160] - Improve SSH key diagnostics
  • [WHIRR-161] - Check that both SSH keys belong to the same pair
  • [WHIRR-163] - Support environment variable interpolation in configuration properties
  • [WHIRR-166] - Improve docs regarding private keys
  • [WHIRR-174] - Fix ZooKeeper to allow stand-alone mode setups
  • [WHIRR-178] - [Hadoop] Guard useradd against existing user account
  • [WHIRR-181] - Add descriptions for CLI command options
  • [WHIRR-187] - [HBase] Change hbase.tmp.dir to be in line with Hadoop service
  • [WHIRR-190] - Create /tmp in HDFS for Pig
  • [WHIRR-194] - Update the list of supported services on the home page
  • [WHIRR-202] - Improve instance template syntax checking
  • [WHIRR-203] - General documentation improvements for 0.3.0
  • [WHIRR-206] - [HBase] Extract strings to a HBaseServiceConstants interface

New Feature

  • [WHIRR-25] - Add HBase service
  • [WHIRR-117] - Composable services
  • [WHIRR-176] - Set AWS credentials in the local site file for Hadoop S3 access

Task

  • [WHIRR-196] - Ensure integration tests pass on all supported providers

Test

  • [WHIRR-92] - Add a benchmark for Hadoop clusters

Release Notes - Whirr - Version 0.2.0

Bug

  • [WHIRR-91] - Add DISCLAIMER file to CLI JAR
  • [WHIRR-93] - Fail on checkstyle violation
  • [WHIRR-97] - Lucid is not stable on EC2
  • [WHIRR-101] - Hadoop on EC2 does not use the /mnt partition
  • [WHIRR-102] - unknown service NPEs cli, should print the bad service to console
  • [WHIRR-106] - improve logging in whirr cli
  • [WHIRR-107] - Test failing due to not matching Amazon Linux AMI on EC2
  • [WHIRR-108] - Fix checkstyle and rat violations
  • [WHIRR-113] - Hadoop cluster instances should all start in the same location
  • [WHIRR-114] - Support + character in version number
  • [WHIRR-122] - whirr site has two FAQ links
  • [WHIRR-126] - Deployment process does not deploy required test JARs

Improvement

  • [WHIRR-52] - Allow a Hadoop MapReduce job to be run against a Hadoop Service running on Rackspace Cloud Servers
  • [WHIRR-66] - Upgrade to jclouds 1.0-beta-7
  • [WHIRR-89] - Support maven 3 builds
  • [WHIRR-90] - Scripts should be versioned
  • [WHIRR-103] - add more to .gitignore
  • [WHIRR-104] - print available services in cli help string
  • [WHIRR-105] - Add version command to the CLI
  • [WHIRR-109] - Unit tests fail if there is no private key found at ~/.ssh/id_rsa
  • [WHIRR-110] - Create client-side Hadoop configuration file during cluster launch
  • [WHIRR-112] - Expand documentation

New Feature

  • [WHIRR-73] - Add a list command to the CLI
  • [WHIRR-100] - Create a binary distribution of Whirr

Release Notes - Whirr - Version 0.1.0

Sub-task

  • [WHIRR-40] - fill in getting started documentation - getting-started.confluence
  • [WHIRR-77] - Document and implement release process
  • [WHIRR-78] - Add KEYS file to distribution directory
  • [WHIRR-85] - Publish Maven artifacts to http://repository.apache.org
  • [WHIRR-86] - Update quick start documentation to work with release 0.1.0

Bug

  • [WHIRR-4] - hadoop-cloud push command invokes proxy creation
  • [WHIRR-37] - Don't require manual installation of Apache RAT to compile
  • [WHIRR-48] - Fix RAT warnings due to site files
  • [WHIRR-50] - Cassandra POM should depend on top-level
  • [WHIRR-65] - Workaround bug 331 in jclouds (Some EC2ComputeService operations fail for stopped instances)
  • [WHIRR-71] - Only allow access to clusters from defined networks
  • [WHIRR-79] - Hadoop service is broken
  • [WHIRR-82] - Integration tests should not run on "mvn install"
  • [WHIRR-84] - Log4j is missing from the CLI JAR

Improvement

  • [WHIRR-8] - Create ant build for running EC2 unit tests
  • [WHIRR-9] - Support additional security group option in hadoop-ec2 script
  • [WHIRR-10] - Create setup.py for EC2 cloud scripts
  • [WHIRR-20] - Generate RAT report
  • [WHIRR-21] - Enforce source code style
  • [WHIRR-22] - Separate unit and integration (system) tests
  • [WHIRR-23] - Upgrade to jclouds 1.0-beta-6
  • [WHIRR-26] - Allow script locations to be overridden
  • [WHIRR-31] - Add Whirr quick start to README.txt and website (once it's live)
  • [WHIRR-32] - Update POM to point to Apache RAT 0.8-SNAPSHOT
  • [WHIRR-34] - Open up ports 50010 (JobTracker) and 50070 (NameNode) for the Hadoop Service
  • [WHIRR-38] - Add core javadoc
  • [WHIRR-47] - Create a Service factory
  • [WHIRR-51] - Allow the Hadoop service to be run on Rackspace Cloud servers
  • [WHIRR-53] - Adopt the standard Java SPI interface
  • [WHIRR-54] - Implement service/cdh
  • [WHIRR-58] - introduce naming consistency for cloud service providers
  • [WHIRR-64] - Unify ClusterSpec and ServiceSpec
  • [WHIRR-70] - decouple keypairs from the files that hold them
  • [WHIRR-75] - Use Commons Configuration to manage cluster specs
  • [WHIRR-80] - Clean up POM dependencies

New Feature

  • [WHIRR-3] - Add support for EBS storage on EC2
  • [WHIRR-5] - Run namenode and jobtracker on separate EC2 instances
  • [WHIRR-6] - Write a Rackspace cloud provider
  • [WHIRR-7] - Add a ZooKeeper service to the cloud scripts
  • [WHIRR-27] - Add Cassandra service
  • [WHIRR-33] - Add a CLI

Task

  • [WHIRR-1] - Import initial source code from Hadoop contrib
  • [WHIRR-2] - Import initial Java source code
  • [WHIRR-19] - Create project website
  • [WHIRR-29] - Add target directories to svn ignore
  • [WHIRR-46] - Release version 0.1.0