Assets 2

Feature

  • Add a check in 'stack report system' for named

  • Upgrade to Python 3.6.6

  • turn status field into a real feature

    Enhanced the message queue status reporting to be hostid based (not
    hostname) and added a ttl to all messages. The first part fixes issues
    when hosts are renamed and the status gets lost, now everything is
    keyed off the ID from the nodes table (permanent for lifetime of
    host). The second allows us the age out messages at different rates.

    Instrumented multiple parts of the installer to send more fine grained
    status messages. The typical workflow of a system install will see the
    following messages:

    message ttl
    install profile request 30
    install profile sent 30
    install profile 300
    install download 3600
    install stage=pre 3600
    install stage=pre-package 3600
    install download 3600
    install stage=post 3600
    install reboot 300
    install stage=boot-pre 3600
    install stage=boot-post 3600
    online 120

    The install download is seen both before the second stage image
    downloads and rpm tracker lookups. All install * messages should be
    thought of as the previous install and the extra information is for
    debugging and may change at any time.

    The online state was previously called up and just meant that the
    installer's first pass was done and the host was on the network. For
    SuSe it could be seens even though the second pass of the installer
    was active. To improve on this Stacki now tracks is sshd is running
    as a seperate state (piggy-back on the existing health message). The
    inspired a new command stack list host status which a state and
    ssh fields. The state field is the previous health message as
    described above. The ssh field is up IFF sshd is running, and None
    if it is not (TTL same as the state health).

    The new stack list host status command means:

    • stack list host status field is now deprecated and will be
      removed in the next release. Just still present to keep existing
      consumers from crashing but it will always says deprecated.

    • pluggins can be used to add move status (is service X runnning?)

    Other stuff:

        * update Redis to 4.0.11
      * change rmq to smq
      * health channel is now json payload (backwards compat with
          old text - but will deprecate)
      * stack-mq is now in the installer (SLES only)
      * stack-publish.py is gone replaced by updated smq-publish
      * fixed some leftover Python3 transition encode()/decode() bugs
      * remove old processors related to StackIQ Enterprise
      * changed `message` field in `Message` to `payload` (less confusing)
      * added psutil to foundation-python-packages
        * this requires refreshing version of all python code
        * hard code PyMySQL to 0.8.1 (newer version require newer pip)
    

    ToDo:

      * Instrument RedHat installer
      * Need a `install start` state (where to trigger it?)
      * fix test_enable_discover.py
    
  • Added the ability to have 'auto' in the IP address field in a host configuration spreadsheet and 'set host interface' command.

    Host IP address will be automatically assigned based on the network address, mask.

    JIRA: STACKI-419

  • remove the dump command

  • Enhance stack add pallet to recognize and add SUSE Enterprise Storage media.

  • Add 'load hostfile' test for duplicate interfaces

  • add cart accepts url and credentials on the command line

    BREAKING CHANGE:
    ALTER TABLE carts ADD url TEXT;

    list pallet expanded=true displays the url
    generalize the download function of add pallet and place it in pylib
    from stack.download import fetch
    file = fetch(url, username="", password="")
    username and password are optional

  • add ability to list supported switches

  • Split switch library into multiple files

  • Add vendor/make to output of 'stack list switch'

  • Add tests for 'stack add/remove host alias'

    JIRA: STACKI-286

  • add a url column to the rolls table

    BREAKING CHANGE:
    ALTER TABLE rolls ADD url TEXT;

    add an optional expanded parameter to list pallet to display the url
    accept username and password parameters for downloading pallets

  • [add,list] storage [controller,partition] with 'os' scope

    changed sql commands to the new format to prevent injection

Bugfix

  • Validation of host interface before entering them in etc/hosts file

  • stack report ansible remove padChar ------ from output

    Also moved ansible into foundation-python-packages, and cleaned
    up the code to use correct stacki-style argument parsing

  • when adding a new api group, if the group already exists, the command error that gets raised is malformed

  • Ensure stack report host only produces unique entries.

  • Update interface information with MAC addresses correctly

    When installing a backend node, all the interfaces are reported
    back by the installer. Any interfaces missing on the frontend,
    will be added to the database.

    The Bug is - when interfaces are specified in the database, with
    all the information except the MAC address, this call fails to
    update the MAC address.

    This commit fixes that.

    FIXES: JIRA STACKI-585

  • Set the frontend root password during frontend-install.py

  • minor tweak to make running set-up.sh more user-friendly

  • Plugins no longer run on a remove of an empty a:backend

  • Prevent syncing dhcpd with duplicate host interfaces

  • Fix database backup cronjob script

  • Major overhaul of the Dell x1052 ethernet switch configuration code.

    BREAKING CHANGE: Apply 'ALTER TABLE switchports DROP PRIMARY KEY;' to database.

  • Prevent loading a hostfile with entries containing an IP but no network

    This fixes JIRA: STACKI-551

  • In select(), we need to stringify the sql parameters to be able to join() them.

  • Prevent set host boot from throwing exception when given a valid-but-empty host specification.

  • add api user needs to return formatted output

    This fixes JIRA: STACKI-536

    When running stack add api user, the code would print
    the credentials of the user that were just created.
    This mean when this command was called through the ReST API,
    it would fail, since it would print the output on standard out,
    and not through stacki command line formatted output.

    This commit fixes the above problem.

    FEATURE: Calling "add api user" from the ReST API returns JSON

    The following JSON

  [
      {
          "hostname": "front.end.name",
          "key": "ran.dom.key",
          "username": "user.name"
      }
  ]

This can then be parsed by clients for future use.

  • fix typo in remove storage partition and prevent call/command list.firewall from printing other appliance rules as their own

  • 'sync host network' continues without backend's ssh available

  • Added more required input and error handling for "remove storage partition"

    Now requires device="*" to remove all scope=global partitioning
    Warns when using deprecated host input without scope=host
    Warns when using a non-existent scope

  • Removed extra "." on PQDN

Git

  • starting release 5.1rc6

@masonkatz masonkatz released this Jun 26, 2018 · 61 commits to master since this release

Assets 2

Feature

  • cleanup report attrfile

    print each entry on a single line
    ignore attrs with empty values; if loaded back these will erase attrs completely
    unload attrfile updated for python 3

  • Add "chassis power" commands to control and list the power settings of hosts.

    To set the power on a host, execute:

    stack set host power {host ...} command="on"|"off"|"reset"

    To list the power for a host, execute:

    stack list host power {host ...}

Bugfix

  • SLES nukedisks=false is now functional 💾

    JIRA: STACKI-307

Git

  • starting release 5.1rc5

@masonkatz masonkatz released this Jun 18, 2018 · 79 commits to master since this release

Assets 2

Feature

  • Add carts using URLs. Remove Unpack Carts

    Add carts using URL.
    Remove "unpack" cart command
    Clean up docstrings. Use authentication for downloading
    carts using URLS

    Fixes JIRA: STACKI-503

  • Reject common hostnames ('frontend' and 'backend')

    This fixes JIRA: STACKI-350

  • Add user settable host metadata

    This is similar to the AWS meta-data, the idea being a container for
    arbitrary data unique to a host. stack set host metadata is used
    to set this and the data itself is made available from a read-only
    host attribute called metadata.

    The same thing could have been accomplished using a standard attribute,
    but the intention is to have the semantics that this is unique to a host
    and not subject to the standard attribute inheritance rules.

    BREAKING CHANGE: schema addition

    $ mysql cluster
    MariaDB [cluster]> alter table nodes add column MetaData text default NULL;
    
  • Have the test-framework VMs use less RAM

    Change all the VMs to be created with 1GB of RAM instead of 2GB. That
    should still be enough for them to get their jobs done.

  • Generate Release Notes Automatically

    stack-releasenotes is installed on all nodes

    This is generated from the Git Log and assumes the new nice
    format for commits. Ugly old git logs are also included (for now)

Bugfix

  • Cart permissions need to be recursively applied.

    Cart permissions were only being applied at the top level, and
    to the files in the directories. None of the directory permissions
    changed. This commit fixes it so that directory permissions change
    as well

  • Give backends 2GB each during tests

    It turns out that the Centos 7 backends don't like to install into
    anything with less than 2GB of RAM.

  • Don't install the stacki-releasenotes package on SLES11

  • stack list os

    list os needs to trick the endOutput code to report only one column, all
    other commands report multiple columns. Recent changes in endOutput broke
    the trick. For output-format=json the was not broken, only for text output.

  • stack list host sorts hosts by rack/rank numerically then alphabetically

    Hosts will be sorted by rack numerically first, then alphabetically. That is, all hosts with
    numerical rack values will be listed first, then all hosts with non-numerical rack values will
    be listed last.

    Hosts with the same rack value, will be sorted numerically first, then alphabetically. Below
    is an example output of stack list host:

    HOST RACK RANK APPLIANCE
    stacki-2-1 0 0 frontend
    backend-0-0 1 station-11 backend
    ethernet-2-1 2 1 switch
    stacki-2-10 2 10 backend
    stacki-2-11 2 11 backend
    stacki-2-12 2 12 backend
    infiniband-2-18 2 18 switch
    infiniband-2-20 2 20 switch
    ethernet-2-43 2 43 switch
    backend-0-1 sector-42 2 backend
    backend-0-2 sector-42 3 backend
    backend-0-3 sector-42 4 backend
    backend-0-4 sector-42 station-8 backend
  • Ignore drop database warning

    This fixes JIRA: STACKI-493

  • SLES11 doesn't have the stack-releasenotes package (currently)

  • Better handling for old pip wheel files

    pip2src crashed on a package update because it of lazy parsing
    of the METADATA file.

  • stack-releasenotes missing from manifest

Git

  • starting release 5.1rc4

@masonkatz masonkatz released this May 18, 2018 · 107 commits to master since this release

Assets 2

Feature

  • Separate 'stack remove storage partition' into commands based on scope.

    Below commands have been introduced:
    stack remove appliance storage partition
    stack remove host storage partition
    stack remove os storage partition

    JIRA: STACKI-246

  • Add non-ethernet interfaces to database

    • Send back ipmi interfaces, and InfiniBand interfaces to frontend, during install.
    • On brand new machines, root user is not created or enabled in the BMC.
      This explicity creates, and enables the root user.
  • Add new exception ArgNotFound for invalid entities

    Don't return CommandError for bad lookups in *ArgumentProcessor, instead return ArgNotFound

  • Add test-framework to Stacki codebase

    This moves the stacki-test-framework project into the stacki
    codebase so that the tests can be easily kept in sync with
    changes to the code.

  • More boot argument for Console Installs

    • Add i40e driver support for all SLES 11 SP3 installs. This will load
      the Intel i40e driver into the kernel. If we happen to use Intel Purley
      nodes, then the driver gets used. Otherwise, it gets ignored.

    • Add nomodeset and textmode for all console installs
      We do not require X drivers for console installs. On Intel reference
      nodes, installs will stall if it tries to load the X VGA driver. So
      force it to not load the drivers.

  • Calculate, report and list the MD5 hashs for hosts.

    Calculate, report and list the MD5 hashes of a host's:
    - pallets
    - carts
    - profile (e.g., kickstart file, autoyast file, etc.)

    This will be reported by a host via the message queue.

    One can check if a host is 'synced' or 'notsynced' by executing:

      stack list host <hostname> hash=y
    

    The above command adds a 'HASH' column with the status.

  • add ability to specify 'channel' as a parameter for 'add host interface'

  • Make table output format more consistent.

    stack list commands with NULL output in the last column will now have -'s printed
    up to the width of that column's header as in other columns.

    This allows cell filling in the 'channel' column of 'stack list host interface',
    without cluttering the output with lots of -'s in 'stack list host' if 'comment' is long.

  • Add jmespath module

    This module is needed by Ansible's json_query filter.

Bugfix

  • stack load storage partition failed for host partitions

    This was a new bug since stacki-5.1rc2

  • Run udevsettle to wait for controller config to finish

    When configuring the controller, the command can exit, but the controller
    still hasn't completed configuration.
    udevsettle waits until the controller is done, before generating
    partitioning info

  • Remove 'stack run host test' since it requires salt

    This was old code left over from StackIQ Enterprise. It's great stuff
    but beyond the mission of ping and prompt. It may come back again
    one day.

    Removed this since it was barfing on the code coverage tests due to
    missing python modules.

  • WS flush cache and don't return stack trace on CommandError

    WS runs multiple threads with each long lived thread managing its own
    cache. A thread cannot invalidate another thread's cache, so always
    flush right before running the stack command (subcommands still use
    the cache).

    Previously we sent the stack trace on CommandErrors, now behave the
    same as stack.py and send the nice error message.

    Code cleanup based on flake8.

  • Allow correct python db api parameters

    Allow our sql methods (execute() and select()) to accept either our current
    style, which is vulnerable to sql injection, or the correct style which allows
    the database driver to correctly handle escaping parameters. This will let us
    migrate code to the safer syntax gradually.

    See github issue #198 for more details

  • Check for duplicate hosts correctly

    When using self.db.select function, make sure to not
    have "select" as the first word in the sql statement.
    This is implicit, and adding it will cause the code to
    fail.

  • make new appliances 'managed' in some situations

    Appliances are already set to kickstartable==True if node==backend
    This piggy backs on that behavior to make them also managed==True 📎

    Remove redundant appliance attributes.

  • Loading hostfile csv throws error on frontend renames 🔏

    CommandError(self, 'Renaming frontend is not supported!')
    JIRA: STACKI-272

  • Run pallet fails if frontend has more than one pallet of the same name

    If there are 2 pallets of the same name on the frontend, even if they happen to
    be different versions, or releases, and they belong to different boxes, "run pallet"
    will still fail on an SQL subquery.

    Make the SQL query cleaner, and context specific.

  • blank 'run host command=' no longer hangs

    🏃
    JIRA: STACKI-325

  • stack list host graph

    Bring "stack list host graph" to the 21st century

  • Refer to the correct information

    When printing nukecontroller information, refer
    to the list that contains nukecontroller information
    and not the nukedisks information.

  • cleanup entity names in example csv files

  • Make api response return JSON

    The api response was mistakingly running json.dumps against the text
    version of the command response, causing the output to be double
    encoded.

    This fixes JIRA: STACKI-432

Git

  • starting release 5.1rc3

  • Update README.md

  • Update README.md