@gareth-ferneyhough gareth-ferneyhough released this Jan 6, 2017 · 41 commits to master since this release


External contributions:

New features:


  • Use FLAGS.aws_user_name if set for Rhel (GH-1249)
  • Fio enhancements (GH-1246)
  • Use aes128-ctr if aes128-cbc isn't available (GH-1242)
  • Make aeropsike able to run with raw devices on static vms (GH-1243)
  • Add some metadata to netperf samples (GH-1236)
  • Remove unneeded Azure code and add availability sets (GH-1232)
  • Add cassandra read concurrency flag (GH-1233)
  • Add a metadata attribute 'ssd_count' to spark_service (GH-1229)
  • Add a metadata attribute 'spark_svc_cloud' to spark_service (GH-1226)
  • Add ability to specify boot_disk_size in AWS vm spec (GH-1231)
  • Add a metadata attribute 'spark_svc_cloud' (GH-1236)
  • Set a default zone for the spark test to allow for subnet creation (GH-1222)
  • Update the location of the spark examples jar on the emr cluster (GH-1222)
  • Persist package installation across reboots (GH-1185)
  • Allow mongodb_ycsb to specify readahead settings (GH-1256)

Bug fixes and maintenance updates:

  • Close Aersospike connection in 1 second (GH-1241)
  • Fix bug with having multiple Azure data disks (GH-1223)
  • Fix aws spark service (GH-1222)
  • Add some git ignore entries (GH-1222)
  • Fix gcs credential conflict (GH-1221)
  • Fix bug with run_processes (GH-1212)
  • Remove collector from RunBenchmarkTask args (GH-1210)
  • Add the minimum tox version to the "tox is not installed" message (GH-1201)
  • Move sysbench05plus path prefix logic into package (GH-1185)
  • Fix bug with specifying Azure image (GH-1204)
  • Fix two files that were failing boilerplate check (GH-1203)
  • Update README.md (GH-1200)


v1.9.0: Release 1.9.0

@asaksena asaksena released this Nov 18, 2016 · 134 commits to master since this release

Changes in this release.

New features:
- Add config file imports and allow for circular imports and add documentation (GH-1163)
- Add a way to run benchmarks in parallel (GH-1192)

- ContainerizedDebianMixin: Ping to a fixed docker image and remove the sudo hack (GH-1171)
- Auto detect  openjdk, libsnappy package versions (GH-1181)
- Add flags to govern subnet creation and Optionally create only 1 subnet (GH-1182)
- Install docker images as packages (GH-1184)
- Call FLAGS.get_help if it's available (GH-1187)
- Publish boot time samples for all tests (GH-1156)

Bugfixes and maintenance updates:
- Update requirements.txt to set version of contextlib2 (GH-1164)
- Fix bug with config flags (GH-1165)
- Updated the cloudsuite web-serving benchmark (thanks @nooshin-mirzadeh, GH-1166)
- Fix load command parsing (GH-1168)
- openstack: Open all TCP and UDP ports for the internal network (GH-1169)
- Fix aerospike metadata (GH-1170)
- Adjust aerospike default replication factor (GH-1172)
- Try replacing FlagValues._flags instead of FlagValues.FlagDict first (GH-1175)
- Fix ycsb aggregator (GH-1176)
- Fix bug during cleanup phase of object_storage_service benchmark (GH-1178)
- Netperf thinktime fixes and support think time in nanoseconds instead of microseconds (GH-1179, GH-1186, GH-1188)
- Fix sysbench05plus installation on ubuntu16.04 (GH-1183)
- Fix race condition in Aerospike benchmark (GH-1190)
- Clean up some pickling, unpickling issues (GH-1191)
- Calculate free ram with more robust /proc/meminfo instead of `free`(GH-1194)
- Fix multithread netperf (GH-1149)
- Multiple fixes and refactoring of the linux package management on vms (GH-1152)
- Fix dstat metadata in Analyze (GH-1154)
- Obey flag overrides for static vm specs (GH-1155)
- Fix links in README file for OpenStack and Cloudstack setup steps (thanks @shakhat, GH-1157)
- Fix issue with GetConfig using flags (GH-1160)
- Remove scratch disk from netperf and object_storage_service benchmark (GH-1147, GH-1148)
- Netperf changes to add num_streams metadata to samples (GH-1177)
- Remove collector from RunBenchmarkTask args (GH-1210)


v1.8.1: Bugfixes and maintanence updates:

@tedsta tedsta released this Oct 20, 2016 · 201 commits to master since this release

 - Fixed default machine type for Azure (GH-1161)


Release v1.8.0

@tedsta tedsta released this Oct 17, 2016 · 222 commits to master since this release

New features:
- Multithreaded netperf benchmark (GH-1141, GH-1144)
- Terasort benchmark using spark service infrastructure (GH-1104)
- Support 10 or more disks on Azure (GH-1102)
- Add sysctl and set_files flags (GH-1096)
- Support DigitalOcean block storage (GH-1092)
- Add cloudsuite_data_serving_rec_count and cloudsuite_data_serving_op_count parameters to cloudsuite_data_serving benchmark (GH-1091)
- Add support for setting GCE boot disk size and type (GH-1106)
- Add flag to skip reload on ycsb benchmark (GH-1108)
- Support for AWS dedicated hosts (GH-1109)
- Object storage multistream write-only scenario (GH-1123)
- Allow multiple configs for the same benchmark in the same file (GH-1126)
- Add flat that allows run retries (GH-1129)
- Add fio_parameters flag (GH-1131)
- Add config matrices (GH-1134)
- Add events at the beginning and end of benchmarks (GH-1135)
- Add extra_zones flag to pkb (GH-1137)
- Add flag matrix filters (GH-1138)
- Support allowing port ranges on Firewall.AllowPort (GH-1139)
- Add static vm tags (GH-1142)

Breaking changes:
- Because of GH-1112, region names have changed.

- Allow speccpu to run individual test (GH-1113)
- Use Azure resource management mode (GH-1112)
- GCP networks use subnets (GH-1087)
- Add top level flags key to configs (GH-1125)
- Precise resource usage for YCSB benchmarks (GH-1127)

Bugfixes and maintanence updates:
- Fix bugs with Terasort (GH-1136)
- Don't run container commands as sudo with ContainerizedDebianMixin (GH-1130)
- Automatically use amazon linux if os_type is rhel (GH-1124)
- Removed Panda as a dependency (GH-1118, GH-1119)
- Allow flag_util.ParseKeyValuePairs values to have colons in them (GH-1117)
- Add ycsb to data directory (GH-1116)
- Fix bug in mongodb_ycsb_benchmark cleanup (GH-1115)
- Fix bug where static VM disk specs were overriden (GH-1105)
- Use string formatting instead of addition to concatenate (GH-1103)
- Fixed regression with fio flag (GH-1097)
- Fix flags in static vm files (GH-1093)
- Fix wait_for_command.py: wait for status file to avoid race condition (GH-1090)


v1.7.0: New features:

@yuyantingzero yuyantingzero released this Aug 30, 2016 · 351 commits to master since this release

* Add ProfitBricks provider (thanks @tjb1019, GH-1005)
* Add YCSB JDBC benchmark (thanks @k1xme, GH-1049)
* Add support for st1 and sc1 disks on EBS (GH-1067)

Breaking changes:
* Change default gce_migrate_on_maintenance to True (GH-1069)

* Update OpenStack provider to use OpenStacks CLI instead of python
  (thanks @meteorfox, GH-942)
* Improve side-by-side tool (GH-1034)
* Specify flags to metadata (GH-1047)
* Allow Java version for Java benchmarks to be selected via command
  line (GH-1051)
* Add numjobs runtime option to fio_benchmark (GH-1066)
* Add netperf histogram support (GH-1072)
* Add multi-vm support for object_storage_service_benchmark (GH-1074)
* Add support for custom s3 endpoints for
* Add user-defined metadata for vms (GH-1080)

Bugfixes and maintenance updates:
* Refactor object_storage_service_benchmark (GH-1023)
* Support overriding ycsb_client_vms in benchmark spec (thanks @k1xme,
* Update YCSB to 0.9.0 and support latest logging format (thanks @k1xme,
* Fix zones flag (GH-1044)
* Update Bigtable to v2 API (thanks @tswast, GH-1045)
* Update maven version to 3.3.9 (GH-1046)
* Start background workload in prepare stage and stop in cleanup
  stage (GH-1048)
* Minor fixes in flag description (GH-1052)
* Only upload files needed for API testing script for
  object_storage_service_benchmark (GH-1053)
* Fix redis_ycsb benchmark to use multiple processes (GH-1062, GH-1068,
* Fix bug with custom machine types configs (GH-1077)
* Fix in documentation of ParseTimeCommandResult (thanks @skorgu,
* Replace Pandas dependency with Numpy (GH-1081)
* Turn off OpenBlas threading, improving HPCC on large vms (GH-1084)
* Cleanup object_storage_serivce benchmark (GH-1085)
* Cleanup aerospike_ycsb benchmarks (GH-1086)


v1.6.0: New features:

@hildrum hildrum released this Jul 5, 2016 · 486 commits to master since this release

New features:


  • Optionally publish VM hostnames in metadata (GH-1020)
  • Control which netperf benchmarks are run via flag (GH-1029)
  • Netperf benchmark reports min and max latency (GH-1013)
  • Add multichase to the google benchmark set (GH-1018)
  • Update Cloudsuite web serving benchmark (thanks @ivonindza, @GH-998)
  • Object storage updates (GH-987)
  • Repeat Run stage (GH-1032)

Bugfixes and maintenance updates:



@ehankland ehankland released this May 25, 2016 · 519 commits to master since this release


New features:

  • Add cloudsuite graph analytics benchmark (thanks @mdrumond; GH-986)
  • Add initial implementation of multichase benchmark (GH-977)
  • Add cloudsuite media streaming benchmark (thanks @ivonindza; GH-993)
  • Add analysis functions for processing multistream object storage benchmark data (GH-905)
  • Add CloudSuite Data-caching benchmark (thanks @neo-apz; GH-970)
  • Add ability to bundle PKB into a python-executable zip file (GH-971)
  • Add os_type for Juju Mixin (thanks @AdamIsrael; GH-764)
  • Add CloudSuite in-memory analytics benchmark (thanks @ivonindza; GH-934)

Breaking changes:

  • Replace the old CloudSuite Web Search benchmark with the docker version (thanks @javpicorel; GH-931)
  • Update Aerospike version (GH-978)


  • Add --fio_blocksize option (GH-943)
  • Update speccpu2006 benchmark to support iso file (GH-944)
  • Allow --run_stage to specify multiple stages (GH-935)
  • Add flag to control iperf timeout (GH-994)

Bugfixes and maintenance updates:

  • Fix cloud bigtable and hbase (GH-1001)
  • Install openssl for the object storage benchmark (GH-1000)
  • Fix bug in MongoDB YCSB benchmark (GH-999)
  • Change import_util.LoadModulesForPath to also load packages (GH-983)
  • Add 'ap-northeast-2' to the S3 regions table (GH-990)
  • Change check-lint.sh to invoke flake8 via tox (GH-979)
  • Change GetLastRunUri to handle the new temp directory structure (GH-985)
  • Support S3 Signature Version 4 (GH-984)
  • Add back support for python-gflags version 2. (GH-973)
  • Define units.Unit.ne (GH-976)
  • Reorder some PKB initialization steps (GH-965)
  • Move to gflags version 3.0.4 (GH-969)
  • Modify semantics of UnitsParser's convertible_to parameter (GH-968)
  • Add percent (%) as a recognized pint unit (GH-964)
  • Improve Azure CLI version check (GH-967, GH-958)
  • Don't request SSH verbose output when --log_level=debug (GH-962)
  • Move per-provider package requirement files (GH-961)
  • Move pint unit registry code into a separate module (GH-960)
  • Only check python package requirements if requirements.txt exists (GH-959)
  • Changed LoadProvider to accept un-lowered cloud provider name (GH-957)
  • Change bg tests to patch and verify call counts for each VM separately (GH-954)
  • Make BenchmarkSpec.vms order consistent across runs (GH-953)
  • Fix os type related issues and bugs (GH-950, GH-955, GH-952, GH-951, GH-949)
  • Call CheckPrerequisites earlier (GH-947)
  • Add percentiles option to PercentilesCalculator (GH-910)
  • Fix publisher bug when disk_size is None (GH-946)
  • Handle unset PYTHONPATH in tox.ini (GH-941)
  • Add a helper function to BenchmarkConfigSpec to redirect flags (GH-936)
  • Add support for interrupting child threads created by RunThreaded (GH-926)
  • Fix bugs and improve the Cloud Bigtable benchmark (GH-937, GH-933, GH-932)
  • Add a helpful error message if a UnitsParser parses a unitless value (GH-963)


v1.4.0: Release 1.4.0

@ehankland ehankland released this Apr 4, 2016 · 606 commits to master since this release

New features:

* openstack: Add support for optional floating ip pool (thanks @meteorfox, GH-861)
* openstack: Use Keystone session for handling authentication (thanks @meteorfox, GH-870)
* Support object storage classes in object storage service benchmarks (GH-895)
* Add Object Size Distributions in object storage service benchmarks (GH-888)
* Add MultiStreamThroughput benchmark to object storage benchmarks (GH-840)
* Adds a SPEC SFS 2014 benchmark which runs against GlusterFS (GH-876)
* gce_virtual_machine: Support user provided instance metadata (GH-859)

Improvements and fixes:

* openstack: Wait until VM deletion has been completed (thanks @meteorfox, GH-904)
* openstack: Fix floating IP allocation and deallocation (thanks @meteorfox, GH-862)
* rackspace: Fix missing flags bug (thanks @meteorfox, GH-903)
* Allow user to specify a flag when they run object storage benchmark on GCP, and default that flag to the latest known working version. (GH-925)
* Update mechanism used to get iperf server process id. (GH-921)
* Rename variables and improve documentation of BaseOsMixin.PullFile. (GH-923)
* Fix WindowsMixin._GetNumCpus when more than one Win32_processor exists. (GH-920)
* Add exception types to except clauses. (GH-893)
* Add Flag for List Consistency Iterations (GH-889)
* Add unit tests for scripts (GH-882)
* Add disk type examples to README.md (GH-871)
* Copy-edit the README (GH-877)
* Turn off selinux for mongodb. (thanks @akrzos, GH-867)
* Use temp files for Popen stdout and stderr in IssueCommand. (GH-878)


v1.3.0: External contributions:

@ehankland ehankland released this Mar 17, 2016 · 665 commits to master since this release

* Add RHEL based virtual machines to OpenStack provider. (thanks @akrzos;
  * Change rackspace provider information to use official CLI (thanks @meteorfox; GH-844)
  * Add rackspace requirements (thanks @meteorfox; GH-805)

New features:
  * Support flags in YAML format. (GH-857)
  * Check version of required packages at runtime (GH-834)
  * User-specified multiregion for GCS benchmarking (GH-845)
  * Support metadata for gcp instances (GH-859)

Bugfixes and maintences updates:
  * Change rackspace's AllowPort to enable UDP (thanks @meteorfox; GH-805)
  * Allow most recent verison gcs-oauth2-boto-plugin (GH-849)
  * Require Pint >= 0.7 (GH-850)
  * Update PIP (GH-842)
  * Fix windows log message (GH-832)
  * Properly Pickle Pint Quantities (GH-830)
  * os_type added to boot benchmark metadata (GH-826)
  * Better handle Azure timeouts (GH-825)
  * Better handling of AWS integration tests. (GH-869, GH-868)


v1.2.0: Release 1.2.0

@ehankland ehankland released this Jan 30, 2016 · 716 commits to master since this release

New features:
* Introduce Object Sizes (GH-808).
* Add ListDecoder for verifying a config option that expects list values

* Bump HBase to 1.0.3 (GH-822).
* Change MockFlags to be more like the real FlagValues (GH-812).
* Rename test_flag_util.py and add FlagDictSubstitution tests (GH-811).
* Create BenchmarkConfigSpec to aggregate benchmark input
  checking (GH-810).
* Remove flag proxy objects (GH-802).

Bugfixes and maintenance updates:
* Fix sample iperf sample (GH-801).
* FixIntDecoder and FloatDecoder behavior when min=0 or max=0 (GH-800).