Skip to content
This repository
Fetching contributors…

Octocat-spinner-32-eaf2f5

Cannot retrieve contributors at this time

file 866 lines (679 sloc) 31.571 kb
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865
===============
Version History
===============

.. contents::
    :local:

.. _version-0.95.4:

0.95.4
======

.. _v0954-news:


News
----
.. warning::

    This release fixes a show-stopping issue for users launching spot clusters
    on a default-VPC account without explicitly specifying a VPC/subnet. It is
    recommended that you upgrade as soon as possible.

Version 0.95.4 of StarCluster is a bug fix release. The sections below contain
the full list of changes. This version is *not* compatible with clusters
started with older versions of StarCluster. Please terminate all clusters
before upgrading to 0.95.4.

Features
--------
* sge: new ``slots_per_host`` setting for specifying a custom number of slots
  per execution host in the cluster (see :ref:`sge-plugin-sph`)

.. _v0954-fixes:

Bug Fixes
---------
* cluster: fix spot_requests property to correctly account for spot clusters
  that get automatically assigned to the default VPC by EC2. (#377)
* loadbalance: fetch and use the remote time zone from the master node rather
  than assuming the timezone is always set to UTC. (#385)
* sge: fix parallel environment slot allocation to respect both
  the ``master_is_exec_host`` and the new ``slots_per_host`` setting.

Enhancements
------------
* awsutils: show progress bar when waiting for spot requests/instances to
  propagate
* listspots: show detailed status and status message
* sge: remove redundant admin/submit host setup

.. _version-0.95.3:

0.95.3
======

.. _v0953-news:

News
----
Version 0.95.3 of StarCluster is a bug fix release. StarCluster no longer
assigns public IPs for non-default VPC clusters by default now because of the
following issues:

1. It opens up the VPC to the internet by default which is a security risk
2. It requires a special VPC configuration (internet gateway attached to the
   VPC and a route to the gateway with dest CIDR block 0.0.0.0/0 associated
   with the VPC subnet). Configuring this automatically (which does not happen
   currently) would be a security risk and without this configuration
   StarCluster cannot connect to the VPC nodes even though they've been
   assigned a public IP. See :ref:`connect-vpc` for more details.

Users can still enable public IPs for their non-default VPC clusters via the
``PUBLIC_IPS`` config setting or the new ``--public-ips`` option to the
``start`` command assuming they've properly configured their VPC to use public
IPs. If the VPC hasn't been properly configured for public IPs an error will be
raised during validation. The ``--no-public-ips`` option has been preserved so
that users can disable the config setting (``PUBLIC_IPS``) from the ``start``
command. Please see :ref:`using-vpc` and :ref:`connect-vpc` for more details.

Please note that this version is *not* compatible with clusters started with
older versions of StarCluster. Please terminate all clusters before upgrading
to 0.95.3.

Features
--------
* start: add -Q, -p, and --no-spot-master options
* cluster: update 'Creating cluster' log msg to indicate VPC

.. _v0953-fixes:

Bug Fixes
---------
* users: pass -n to echo to avoid trailing newlines (#375)
* cluster: do not use public IPs with VPC by default (#372)
* cluster: fix VPC public IPs validation

.. _version-0.95.2:

0.95.2
======

.. _v0952-news:

News
----
Version 0.95.2 of StarCluster is a bug fix release. Please note that this
version is *not* compatible with clusters started with older versions of
StarCluster. Please terminate all clusters before upgrading to 0.95.2.

.. _v0952-fixes:

Bug Fixes
---------
* cluster: sort chunked cluster tags before loading (#371)
* config: set public_ip setting type to boolean (#366)

.. _version-0.95.1:

0.95.1
======

.. _v0951-news:

News
----
Version 0.95.1 of StarCluster is a bug fix release. Please note that this
version is *not* compatible with clusters started with older versions of
StarCluster. Please terminate all clusters before upgrading to 0.95.1.

.. _v0951-fixes:

Bug Fixes
---------
* cluster: fix max length error when applying cluster tags (#348)
* start: fix default value for ``--no-public-ips`` option (#366)
* cluster: fix for custom security group permissions (#356)

.. _version-0.95:

0.95
====

.. _v095-news:

News
----
Version 0.95 of StarCluster is a feature release with many enhancements and bug
fixes. This release has support for Amazon's `Virtual Private Cloud (VPC)
<http://aws.amazon.com/vpc/>`_. See :ref:`using-vpc` for details on using
StarCluster with VPC. This release comes with new Ubuntu 13.04 AMIs for 32bit
and 64bit platforms as well as an HVM AMI that comes with drivers for GPU and
enhanced networking. The HVM AMI also comes with the CUDA toolkit, PyCuda,
MAGMA, and OGS GPU sensor. These new AMIs are available in all AWS regions.
Please use the **listpublic** command to find them::

    $ starcluster listpublic

The sections below list the full set of features, enhancements, and bug fixes
for this release. Please note that this version is *not* compatible with
clusters started with older versions of StarCluster. Please terminate all
clusters before upgrading to 0.95.

.. _v095-features:

Features
--------
* **Add Virtual Private Cloud (VPC) support**
* **Update base StarCluster AMI ids to 13.04**
* Add support for new C3 instance types
* Add support for new GRID GPU g2.2xlarge (non-GPGPU) instance type
* Add support for m3.{medium,large} instance types
* Add support for new I2 instance family
* start: add ``--subnet`` (``-N``) and ``--no-public-ips option`` (VPC-only)
* start: add ``--dns-prefix`` (``-P``) option to prefix hostnames with the
  cluster tag
* removenode: added ``--force`` (``-f``) option to removenode
* removenode: prompt for confirmation and add ``--confirm`` (``-c``) option
* sshmaster, sshnode: add ``--pseudo-tty`` (``-t``) option
* createkey: add ``--import-key`` (``-i``) option
* cluster: validate imported keypair fingerprints
* spothistory: add ``--vpc`` and ``--classic`` options
* spothistory: use classic or vpc prices based on default vpc
* hadoop: tune number of map/reduce tasks to number of CPUs
* hadoop: document all advanced settings
* listclusters, listinstances: show vpc and subnet ids

.. _v095-enhancements:

Enhancements
------------
* removenode: update usage to mimic addnode (ie added ``-n`` and ``-a``
  options similar to addnode) - old usage works but is deprecated
* Removed internal scp module in favor of @jbardin's scp module
* awsutils: wait for placement and security group deletion propagation in
  ec2.delete_group
* node: don't force NFSv3 - use OS default (NFSv4 on recent AMIs)
* ebsimage: reduce rsync output by switching -v to -q (s3->ebs)
* ebsimage: log the ebs volume formatting step
* volume: log the exception when volume creation fails
* loadbalance: set UTC tzinfo in all datetime objects
* utils: use UTC timezone for all datetime objects
* cluster: deprecated all get_node_by* calls in favor of cluster.get_node
* awsutils: raise exc in EasyEC2.wait_for_propagation after max_retries
* Use pytest to run tests

.. _v095-fixes:

Bug Fixes
---------
* node: fix stale NFS entries from force-terminated spot nodes
* node: remove 'user' setting from NFS mount options
* node: add work-around for bug in debian nfs-kernel-server init script in
  Ubuntu 13.04+
* node: use apt-get instead of apt in package_provider property
* node: run apt-get update before install in apt_install
* cluster: always use master's zone if not specified
* cluster: store availability_zone in core cluster settings tag
* cluster: raise error if MOUNT_PATH=/
* cluster: retry deleting placement group until successful
* cluster: check for master in remove_nodes prior to removing any nodes
* loadbalance: add nodes when below minimum
* loadbalance: raise an exception if min_nodes > max_nodes in loadbalancer
* loadbalance: fix crash in get_all_stats() when arr is empty
* ebsimage: set fs label = / on root vol when converting s3->ebs
* ebsimage: fix for bmap when converting s3->ebs
* ebsimage: only set 1 ephemeral drive when converting s3->ebs
* volume: only detach if vol != available in _delete_new_volume
* volume: remove duplicates from volume hosts list
* spothistory: improve accuracy by not using a possibly out-of-sync local time
  stamp for the end date as long as -e is not specified
* spothistory: set UTC tzinfo on all datetime objects
* awsutils: raise exception if keypair exists in create_keypair
* awsutils: use instance store bmap for 32bit m1.small
* awsutils: dont specify ephemeral drives w/ t1.micro
* cli: raise parser error if tag is incorrectly specified
* condor: fetch & create condor dirs from config
* condor: set SCHEDD_NAME on each host
* condor: improve dedicated scheduler config

.. _version-0.94.3:

0.94.3
======

.. _v0943-news:

News
----
Version 0.94.3 of StarCluster is a bug fix release. See the "Bug Fixes"
sections below for details. In general it's recommended to terminate all
clusters before upgrading to new versions of StarCluster although this is not
always strictly required.

.. _v0943-fixes:

Bug Fixes
---------
* Fixed 'InvalidSnapshot.NotFound' errors on cluster launch
* Fixed issue with Server.InternalError: Internal error on launch when using
  m1.small instance type with a 32bit AMI.
* Updated default AMIs in config template to 64bit AMIs

.. _version-0.94.2:

0.94.2
======

.. _v0942-news:

News
----
Version 0.94.2 of StarCluster is a bug fix release. See the "Bug Fixes"
sections below for details. In general it's recommended to terminate all
clusters before upgrading to new versions of StarCluster although this is not
always strictly required.

.. _v0942-fixes:

Bug Fixes
---------
* node: fixed regression where the instance's connection object was not being
  reused which was causing issues for folks using regions other than the default
  us-east-1.

.. _version-0.94.1:

0.94.1
======

.. _v0941-news:

News
----
Version 0.94.1 of StarCluster is a bug fix release. See the "Features" and
"Bug Fixes" sections below for details. In general it's recommended to
terminate all clusters before upgrading to new versions of StarCluster
although this is not always strictly required.

.. _v0941-features:

Features
--------
* restart: add ``--reboot-only`` option
* awsutils: force ``delete_on_termation`` on root volume when launching
  instances
* delay using optional imports in starcluster.utils until needed (reduces the
  time it takes to load StarCluster)

.. _v0941-fixes:

Bug Fixes
---------
* cluster: wait for all instance and spot requests to propagate (fixes
  'instance not found' errors and 'missing' spot requests when waiting for
  cluster to come up)
* cluster: terminate any instance that's been pending for more than 15 minutes
  (fixes infinite wait due to a forever pending node)
* s3image: fix block device map issue when host instance is EBS-backed
* awsutils: fix custom block device map for S3 instances/AMIs
* sshnode: fix keypair validation bug
* node: fix for duplicate entries in known_hosts file
* node: fix bug where credentials attributes were missing from node.ec2 object
* loadbalancer: always pass ``--utc`` when calling date command
* loadbalancer: set matplotlib backend to Agg - allows ``--plot`` option to
  work without a display
* config: convert ``EC2_CERT`` and ``EC2_PRIVATE_KEY`` in ``[aws]`` section to
  absolute path
* config: fix typo in config template (``PROTOCOL`` -> ``IP_PROTOCOL``)
* tmux plugin: fix race condition where ``new-window -t`` is called before
  ``new-session`` has finished
* spothistory: update help docs for ``--plot`` option

.. _version-0.94:

0.94
====

.. _v094-news:

News
----
Version 0.94 of StarCluster is a feature release that fixes *many* bugs. See
the "Features" and "Bug Fixes" sections below for details. This release is
not compatible with any active clusters that were created using any previous
version of StarCluster. Please terminate all existing clusters before
upgrading to 0.94.

.. _v094-features:

Features
--------
* Support for hs1.8xlarge, hi1.4xlarge, and m3 instance types
* Added support for userdata scripts (new cluster setting: userdata_scripts)
* Add *all* ephemeral drives for any instance type when launching instances
* Updated base AMIs to latest 12.04 versions
* OGS (formerly SGE): set default parallel environment allocation_rule to $fill_up
* Added authentication agent forwarding to sshmaster, sshnode, and sshinstance
* Support for loading config based on ``$STARCLUSTER_CONFIG`` environment
  variable
* New plugin to install python packages with pip (pypkginstaller plugin)
* New plugin that either copies or autogenerates a boto config for the cluster
  user
* Added ``--force`` option to terminate/stop commands
* Add tags and virtualization type to listinstance output
* Add ``--zone`` (``-z``) option to spothistory command

.. _v094-fixes:

Bug Fixes
---------
* sshutils: source /etc/profile by default
* awsutils: validate AWS SSL certificates by default
* awsutils: refetch group before calling authorize
* awsutils: use 3 decimal places in spot history prices
* cluster: fix thread leak when running plugins
* cluster: store cluster metadata in userdata/tags
* cluster: dont launch master as spot when ``CLUSTER_SIZE=1``
* cluster: reraise uncaught exceptions in run_plugins
* cluster: fix instance filtering for default VPC
* cluster: only clean-up placement groups in supported regions
* clustersetup: ebs volume mounting improvements
* clustersetup: raise exception if user's ``$HOME`` is owned by root
* improve ebs volume device and partition detection
* config: allow inline comments in the config file
* config: support ``[vol]`` shortcut in config
* config: document/warn where/when env overrides cfg
* threadpool: clear the exceptions queue in wait()
* node: fix Node.root_device_name for misconfigured AMIs
* node: fix ~/.ssh ownership in generate_key_for_user
* sge: Export ``$DRMAA_LIBRARY_PATH`` for Python DRMAA
* cli: use logging module to log/show exceptions
* cli: write header to sys.stderr
* logger: move console logs from stderr to stdout
* logger: output error messages to stderr
* logger: remove in-memory log handler (memory-leak fix)
* fix for DependencyViolation when deleting security group
* replace group-name filter with instance.group-name
* spothistory: show *current* spot price instead of oldest

**resizevol/createvol**:

* call e2fsck before resize2fs if needed
* cleanup vol on failure in resize()
* fix bug in _warn_about_volume_hosts
* fix bug in sfdisk repartitioning
* set default instance type to t1.micro for volume hosts
* fix bug when cleaning up failed volumes
* use force_spot_master when launching cluster

**ebsimage/s3image**:

* fetch root device from instance metadata
* only wait for snapshot if root device is in bmap
* show progress bar for CreateImage snapshot
* remove ssh host keys in clean_private_data
* use ec2.get_snapshot to fetch snapshots

**loadbalance**:

* collect all queue info from qstat
* fix logic in SGEStats._count_tasks
* fix node count in _eval_remove_node
* get slot calculations from all.q
* handle qacct error on fresh SGE install
* improve qstat parsing
* maintain a single instance of SGEStats
* move static settings to __init__
* remove bounds on interval and wait_time
* remove estimated time to completion calcs
* replace 'qlen > ts' condition in _eval_add_node
* require positive ints for numeric options
* take slots into account in _eval_add_node
* update sge tests for latest parser changes
* use # of nodes for max_nodes check (not exec hosts)
* don't wait for cluster to stabilize to add a node when there are no slots
* immediately add a node when jobs are in the queue and no slots exists

**IPython plugin**:

* Add section on engine restart + note on growing new engines
* ipcluster: raise exception if IPython isnt installed
* Better docstrings for the stop and restart plugins
* Better packer configuration support + updated doc
* Make it possible to configure a custom notebook folder
* Make it possible to restart engine, better logs, make msgpack setup explicit
* Open the controller ports for IPython 0.13.1
* Use the cluster thread pool to start the engines on the nodes
* Update the documentation according to tests done on 0.13.1

.. _version-0.93.3:

0.93.3
======

.. _v0933-news:

News
----
Version 0.93.3 of StarCluster is a patch release that fixes several bugs. See
the "Bug Fixes" section below for details.

* Bumped boto dep to 2.3.0
* Updated and improved default config template

.. _v0933-fixes:

Bug Fixes
---------
* sge: fix bug where master could never be an exec host even when toggling the
  ``master_is_exec_host`` plugin setting (issue 89)
* fix bug when setting custom SSH permission without ``cidr_ip`` (issue 91)
* start-cmd: fix bug where ``refresh_interval`` is always ``None`` if
  ``[global]`` is not defined in the config (issue 90)
* dont allow ``cluster_user=root`` when validating cluster settings (issue 92)
* fix another textwrap error with python 2.5

.. _version-0.93.2:

0.93.2
======

.. _v0932-news:

News
----
Version 0.93.2 of StarCluster fixes several bugs and adds a couple new
features. See the "Bug Fixes" section below for details.

* Amazon recently announced a new instance type ``m1.medium`` which is now
  supported. Amazon also enabled the m1.small and c1.medium instance types to
  be compatible with both 32bit *and* 64bit AMIs now. StarCluster now supports
  the new ``m1.medium`` instance type as well as the new changes to
  ``m1.small`` and ``c1.medium``.
* Switched from paramiko to actively maintained fork called `ssh` (1.7.13)
* loadbalancer: ``--kill-master`` option renamed to ``--kill-cluster``
* Bumped boto dep to 2.2.2
* Bumped jinja dep to 2.6

.. _v0932-features:

Features
--------
* SGE setup code has been moved to a plugin that is executed by default (**no
  configuration changes required**). This plugin supports an optional setting
  ``master_is_exec_host`` which controls whether or not the master node
  executes jobs. This is set to ``True`` by default. To change this you will
  need to set ``disable_queue=True`` in your cluster config and then manually
  define and add the SGE plugin in order to configure the new plugin's
  settings. See :ref:`sge-plugin` for more details.
* Added the following new options to ``addnode`` command: ``--image-id``,
  ``--instance-type``, ``--availability-zone``, ``--bid``. Thanks to Dan Yamins
  for his contributions. (issue 76)
* New ``CreateUsers`` plugin that creates multiple cluster users and configures
  passwordless SSH. Either number of users or a list of usernames can be
  specified. Also has options to tar and download all SSH keys for generated
  users. See :ref:`users-plugin` for more details.
* Added -X option to ``sshmaster`` and ``sshnode`` commands for forwarding X11.
  Can be used both when logging in interactively and when executing remote
  commands.

.. _v0932-fixes:

Bug Fixes
---------
* catch `Garbage packet received` exception and warn users about incompatible
  AMI
* ``createvolume``: support new /dev/xvd* device names
* fix textwrap error with python 2.5 (issue 85)
* ``sshmaster``, ``sshnode``: load /etc/profile by default when executing
  remote cmds
* ``terminate``: handle Ctrl-D gracefully when prompting for input (issue 80)
* ``terminate``: ignore keys when terminating manually
* validate key_location is readable (issue 77)
* relax validation constraints for running instances when using ``starcluster
  start -x`` (issue 78)
* fix cluster validation for `t1.micro` AMIs/instance types
* ``loadbalance``: fix _get_stats when no exec hosts
* ``loadbalance``: respect min_nodes when removing nodes (issue 75)
* ``loadbalance``: fix -K option to kill cluster when queue is empty (issue 86)
* ``loadbalance``: retry fetching stats 5x before failing (issue 65)
* skip NFS exports setup when there are no worker nodes
* sge: configure SGE profile on all nodes concurrently
* sge: use qconf's -\*attr commands to update cluster config (replaced a bunch
  of ugly code that parsed SGE's console output)
* add Changelog to MANIFEST.in in order to include Changelog in future release
  dists

.. _version-0.93.1:

0.93.1
======

.. _v0931-news:

News
----
.. warning:: It is recommended that you upgrade as soon as possible.

Version 0.93.1 of StarCluster is largely a patch release that fixes several
bugs. See the "Bug Fixes" section below for details.


Also, the ``put`` and ``get`` commands have been marked stable which means they
no longer require ``ENABLE_EXPERIMENTAL=True`` in the config in order to use
them.

.. _v0931-features:

Features
--------
* Added sys.argv to crash reports for better details when things go wrong
* Implemented on_add_node and on_remove_node for TMUX plugin to add/remove
  nodes from the TMUX control-center sessions for both ``root`` and
  ``CLUSTER_USER``
* Implemented on_add_node for the Ubuntu package installer plugin
  (``pkginstaller``) to ensure new nodes also install the required packages
* Implemented on_add_node for the MPICH2 plugin to ensure new nodes are
  configured for MPICH2 and also added to the hosts file. Moved the MPICH2
  hosts file to an NFS-shared location (``/home/mpich2.hosts``) for faster
  configuration
* Implemented on_add_node for the Xvfb plugin that installs and runs Xvfb on
  newly added nodes

.. _v0931-fixes:

Bug Fixes
---------
* Fixed major NFS bug in ``addnode`` command where ``/etc/exports`` was being
  wiped each time a new node was added. This caused the last-to-be-added node
  to be the only node correctly configured for NFS in the entire cluster.
* Updated all plugins which subclass DefaultClusterSetup to implement
  on_add_node and on_remove_node regardless of whether or not they actually use
  them to avoid the 'default' plugin's implementation being incorrectly called
  multiple times by plugins when adding/removing nodes using the ``addnode``
  and ``removenode`` commands
* Fixed string formatting bug in Condor plugin

.. _version-0.93:

0.93
====

.. _v093-news:

News
----
New StarCluster Ubuntu 11.10 EBS AMIs available including a new HVM AMI for use
with CC/CC2/CG instance types! Includes OpenGridScheduler (to replace SGE),
Condor, Hadoop/Dumbo, MPICH2, OpenMPI, IPython 0.12 (parallel+notebook), and
more. Run ``dpkg -l`` on the AMIs for the full list of packages.

The new StarCluster Ubuntu 11.10 EBS HVM AMI additionally comes with NVIDIA
driver, CUDA Toolkit/SDK, PyCuda, PyOpenCL, and Magma for GPU computing.

These new AMIs are available in all AWS regions. The new ``us-east-1`` 11.10
EBS AMIs are now the defaults in the StarCluster config template. Use the
``listpublic`` command to obtain a list of available StarCluster AMIs in other
regions::

    $ starcluster -r sa-east-1 listpublic

.. note::

    The **HVM** AMI is currently only available in the ``us-east-1`` region
    until other regions support HVM.

.. _v093-features:

Features
--------
* Added support for new CC2 instance types

* Added support for specifying a proxy host to use when connecting to AWS::

   [aws info]
   aws_proxy = your.proxy.com
   aws_proxy_port = 8080
   aws_proxy_user = yourproxyuser
   aws_proxy_pass = yourproxypass

  See :ref:`proxy-config` for more details

* Updated IPCluster plugin to support IPython 0.12 with :ref:`parallel
  <using-ipython-cluster>` and :ref:`notebook <ipython-notebook>`
  support (SSL+password auth). See :ref:`ipcluster-plugin` for more details

* Keypairs are now validated against EC2 fingerprint which notifies users ahead
  of time if the file they specified in KEY_LOCATION doesn't match the EC2
  keypair specified

* Support for Windows (tested on 32bit Windows 7). See :ref:`windows-install`
  for more details

New Plugins
^^^^^^^^^^^
* **Condor** - experimental support for creating a Condor pool on StarCluster.
  See :ref:`condor-plugin` for more details
* **Hadoop** - support for creating a Hadoop cluster with StarCluster
  See :ref:`hadoop-plugin` for more details
* **MPICH2** - configure cluster to use MPICH2 instead of OpenMPI
  See :ref:`mpich2-plugin` for more details
* **TMUX** - configures cluster SSH "dashboard" in tmux
  See :ref:`tmux-plugin` for more details
* **Package installer** - install Ubuntu packages on all nodes concurrently
  See :ref:`pkginstaller-plugin` for more details
* **Xvfb** - install and configure an Xvfb session on all nodes (``sets
  DISPLAY=:1``) See :ref:`xvfb-plugin` for more details
* **MySQL** - install and configure a MySQL cluster on StarCluster. See
  :ref:`mysql-plugin` for more details

.. _v093-fixes:

Bug Fixes
---------
* Fix bug in ``put`` command where in random cases the last few bytes of the
  file weren't being transferred
* Raise an error if placement group creation fails
* Fix bug in terminate command in the case that nodes are already
  ``shutting-down`` when the terminate command is executed
* Fix bug in s3image when destination S3 bucket already exists
* Fix known_hosts warnings when ssh'ing from node to node internally
* Fix trivial logging bug in createvolume command

.. _version-0.92.1:

0.92.1
======
:release-date: 11-05-2011

.. _v0921-features:

Features
--------
* Support for splitting the config into an arbitrary set of files::

    [global]
    include=~/.starcluster/awscreds, ~/.starcluster/myconf

  See :ref:`splitting-the-config` for more details

* createvolume: support naming/tagging newly created volumes::

    $ starcluster createvolume --name mynewvol 30 us-east-1d

  See :ref:`create-and-format-ebs-volumes` for more details

* listvolumes: add support for filtering by tags::

    $ starcluster listvolumes --name mynewvol
    $ starcluster listvolumes --tag mykey=myvalue

  See :ref:`managing-ebs-volumes` for more details

* sshmaster, sshnode, sshinstance: support for running remote
  commands from command line::

    $ starcluster sshmaster mycluster 'cat /etc/fstab'
    $ starcluster sshnode mycluster node001 'cat /etc/fstab'
    $ starcluster sshinstance i-99999999 'cat /etc/hosts'

  See :doc:`manual/runcommands` for more details

.. _v0921-fixes:

Bug Fixes
---------
The following bugs were fixed in this release:

**spothistory command**

* add package_data to sdist in order to include the necessary web media and
  templates needed for the ``--plot`` feature. The previous 0.92 version left
  these out and thus the ``--plot`` feature was broken. This should be fixed.

* fix bug when launching default browser on mac

**start command**

* fix bug in option completion when using the start command's
  ``--cluster-template`` option

**terminate command**

* fix bug in terminate cmd when region != us-east-1

**listkeypairs command**

* fix bug in list_keypairs when no keys exist

Improvements
------------
* listinstances: add 'state_reason' msg to output if available
* add system info, Python info, and package versions to crash-report
* listregions: sort regions by name
* improved bash/zsh completion support. completion will read from the correct
  config file, if possible, in the case that the global -c option is specified
  while completing.
* separate the timing of cluster setup into time spent on waiting for EC2
  instances to come up and time spent configuring the cluster after all
  instances are up and running. this is useful when profiling StarCluster's
  performance on large (100+ node) clusters.

.. _version-0.92:

0.92
====
:release-date: 10-17-2011

.. _v092-news:

News
----
.. note::

    Before upgrading please be aware that rc1 does not support clusters created
    with previous versions and will print a warning along with instructions on
    how to proceed if it finds old clusters. With that said, it's best not to
    upgrade until you've terminated any currently running clusters created with
    an old version. Also, the @sc-masters group is no longer needed and can be
    removed after upgrading by running "starcluster terminate masters"

You can find the new docs for 0.92rc1 here:

http://star.mit.edu/cluster/docs/0.92rc1/

NOTE: these docs are still very much a work in progress. If anyone is
interested in helping to improve the docs, please fork the project on github
and submit pull requests. Here's a guide to get you started:

http://star.mit.edu/cluster/docs/0.92rc1/contribute.html

Please upgrade at your leisure, test the new release candidate, and submit any
bug reports preferably on StarCluster's github issue tracker or this mailing
list.

.. _v092-features:

Features
--------
* Cluster Compute/GPU Instances- Added support for the new Cluster
  Compute/GPU instance types. Thanks to Fred Rotbart for his contributions

* EBS-Backed Clusters- Added support for starting/stopping EBS-backed
  clusters on EC2. The ``stop`` command now stops (instead of terminate)
  an EBS-backed cluster and terminates an S3-backed cluster. Added a
  terminate command that terminates *any* cluster

* Improved performance - using workerpool library
  (http://pypi.python.org/pypi/StarCluster) to configure nodes concurrently

* New ``ebsimage`` and ``s3image`` commands for easily creating new EBS-backed
  AMIs. Each command supports creating a new AMI from S3 and EBS-backed image
  hosts respectively. (NOTE: ``createimage`` has been renamed to s3image. you can
  still call ``createimage`` but this will go away in future releases)

* Add/Remove Nodes - added new ``addnode`` and ``removenode`` commands for
  adding/removing nodes to a cluster and removing existing nodes from a cluster

* Restart command - Added new restart command that reboots the cluster and
  reconfigures the cluster

* Create Keypairs - Added ability to add/list/remove keypairs

* Elastic Load Balancing - Support for shrinking/expanding clusters based on Sun
  Grid Engine queue statistics. This allow the user to start a single-node
  cluster (or larger) and scale the number of instances needed to meet the
  current SGE queue load. For example, a single-node cluster can be launched and
  as the queue load increases new EC2 instances are launched, added to the
  cluster, used for computation, and then removed when they're idle. This
  minimizes the cost of using EC2 for an unknown and on-demand workload. Thanks
  to Rajat Banerjee (rqbanerjee)

* Security Group Permissions - Added ability to specify permission settings to
  be applied automatically to a cluster's security group after it's been started

* Multiple Instance Types - Added support for specifying instance types on a
  per-node basis. Thanks to Dan Yamins for his contributions

* Unpartitioned Volumes- StarCluster now supports both partitioned and
  unpartitioned EBS volumes

* New Plugin Hooks - Plugins can now play a part when adding or removing a node as
  well as when restarting or shutting down the entire cluster by implementing the
  ``on_remove_node``, ``on_add_node``, ``on_shutdown``, and ``on_reboot``
  methods respectively

* Added a ``runplugin`` command and fixed the run_plugin() method in the
  development shell. You can now run a plugin in the dev shell like so::

    ~[1]> cm.run_plugin('myplugin', 'mycluster')

* Added support for easily switching regions (without config changes)
  using a global -z option. For example::

    $ starcluster -r eu-west-1 listclusters
    StarCluster - (http://star.mit.edu/cluster) (v. 0.92rc1)
    Software Tools for Academics and Researchers (STAR)
    Please submit bug reports to starcluster@mit.edu

    >>> No clusters found...

* Added new ``resizevolume`` command for easily resizing existing EBS volumes

* Added listregions/listzones commands
Something went wrong with that request. Please try again.