v0.11.0

Description

Release v0.11.0 drops support for Docker Compose v1 in the Host Manager agent and adds support for Docker Compose v2. It also comes with improvements to the session.status object. Lastly, we have stopped publishing the ocs-utils image, which has not been properly maintained for a while now.

Compose v2 Update

OCS users relying on the Host Manager Agent likely already have Compose v2 installed, but should migrate to Compose v2 and remove v1. Using v1 may result in some errors preventing updates to services.

session.status Update

Details on this update can be found in #371. One added feature is the new "DEGRADED" OpCode value. See the updated documentation for how to use this to monitor the state of an agent.

Additionally, session.status is now placed into the "RUNNING" state automatically just before the agent operation is launched. Agent developers no longer need to call session.set_status('running') explicitly within their agents. Any instances of session.set_status('starting') should be removed from existing Agents, as they will cause an error related to the session status going backwards. session.publish_status has also been removed.

What's Changed

New Features

• Improvements to session.status by @mhasself in #371
• HostManager: Replace docker compose v1 with v2 by @mhasself in #383
• Update docs for Docker Compose v2 by @BrianJKoopman in #384

Bug Fixes

• Fix bug in how args are parsed from site-config by @jlashner in #378

Version Updates

• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #369
• Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 by @dependabot in #374
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #376
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #379

Misc.

• Relax pinned versions on docs requirements by @BrianJKoopman in #375
• Drop the ocs-utils container by @BrianJKoopman in #381

Full Changelog: v0.10.4...v0.11.0

v0.10.4

Description

Release v0.10.4 fixes a data duplication bug that can occur under certain circumstances in any agent that subscribes to a Feed. Within the core OCS agents this means the aggregator and InfluxDB publisher agents. The bug occurs when a network interruption disrupts the connection between the crossbar server and the subscribing agent, depending on the state of the crossbar server. Details are available in #366. Users are encouraged to update their agents.

What's Changed

• Bump actions/setup-python from 4 to 5 by @dependabot in #364
• Fix duplicate data frame writes in Aggregator agent by @BrianJKoopman in #366

Full Changelog: v0.10.3...v0.10.4

v0.10.3

Description

Release v0.10.3 focuses on the Host Manager Agent and introduces a configurable agent timeout for crossbar disconnects. Users are encouraged to upgrade.

Host Manager

There are many bug fixes and some new features for the Host Manager introduced by #353. For details see the "Centralized Management" page in the docs. One highlight in particular is the new "manage" options, detailed in "Advanced host config". These changes should also enable running a single Host Manager where two previously were run (one for agents on the host, and one for agents within Docker containers.)

Crossbar Timeout

The "crossbar-timeout" setting allows for agents to wait longer for the crossbar server to come back online before shutting down. They can wait indefinitely if this is set to 0. See "Crossbar Connection Timeout" for more details.

What's Changed

New Features

• HostManager overhaul by @mhasself in #353
• Allow user configurable agent timeout for crossbar disconnection by @BrianJKoopman in #337
• Add SITE_HOST envvar to ocs-agent-cli by @mhasself in #354

Bug Fixes

• systemd service file: add RestartSec=10s by @mhasself in #344
• HostManager overhaul by @mhasself in #353

Version Updates

• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #340
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #346
• Bump docker/login-action from 2 to 3 by @dependabot in #352
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #350
• Bump actions/checkout from 3 to 4 by @dependabot in #349
• Bump styfle/cancel-workflow-action from 0.11.0 to 0.12.0 by @dependabot in #355
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #358

Testing

• Replace pytest-docker-compose with pytest-docker plugin by @BrianJKoopman in #342

Full Changelog: v0.10.2...v0.10.3

v0.10.2

Description

Release v0.10.2 mostly contains bug fixes and quality of life improvements.

Address Root

Bug fix #327 effectively adds the functionality provided by the "address_root". This allows you to set the address root to a string other than 'observatory', however you must then adjust settings in your crossbar server configuration. See the updated System Configuration pages for the SCF and Crossbar Configuration for more details.

Note
Changing the address root on an existing system will have implications for viewing the data in Grafana and loading from .g3 file, as all feed names will change to include the new address root.

What's Changed

New Features

• Separate blocks for each registered operation by @jlashner in #312
• ocs-agent-cli renames itself in process list by @mhasself in #325

Bug Fixes

• Add importlib_metadata dependency for ocs-agent-cli by @BrianJKoopman in #323
• Revist address_root != "observatory" by @mhasself in #327
• Cast tuple param before type check by @BrianJKoopman in #332
• Support direct start calls with params to param decorated Tasks/Processes by @BrianJKoopman in #333

Testing

• Test when registry block changes by @BrianJKoopman in #314
• Test/development portability by @cnweaver in #319

Documentation

• Update session.data example in main registry process by @BrianJKoopman in #310
• Fix readthedocs builds by @BrianJKoopman in #328

Version Updates

• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #308
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #316
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #321

Misc.

• Replace master and develop branches with "main" branch by @BrianJKoopman in #309
• Fix the github workflow badge by @BrianJKoopman in #315
• Change warning text by @mhasself in #318

Full Changelog: v0.10.1...v0.10.2

v0.10.1

Description

Release v0.10.1 is mostly bug fixes and performance improvements.

Users are especially encouraged to upgrade the registry and influxdb publisher agents, as major performance improvements have been made which become necessary when the OCS network grows to a certain size. These improvements reduce I/O load on the system.

What's Changed

New Features

• Adds wait-time arg to registry by @jlashner in #296

Bug Fixes

• Check for bool when formatting line protocol by @BrianJKoopman in #297
• Combine payloads before writing to influxdb by @BrianJKoopman in #301
• Update ocs-local-support so it can launch HostManager again by @mhasself in #298
• Add boolean support to Aggregator by @BrianJKoopman in #299
• Remove 10 second fresh_time on registry feed by @BrianJKoopman in #303
• Sanitize session data by @mhasself in #304

Version Updates

• Bump actions/checkout from 2 to 3 by @dependabot in #292
• Bump actions/setup-python from 2 to 4 by @dependabot in #291
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #294

Misc.

• Setup pre-commit and run on all files by @BrianJKoopman in #288
• Configure dependabot by @BrianJKoopman in #289
• GitHub Actions workflow improvements by @BrianJKoopman in #293
• Deploy on v0* pre-release tags by @BrianJKoopman in #306

New Contributors

• @dependabot made their first contribution in #292
• @pre-commit-ci made their first contribution in #294

Full Changelog: v0.10.0...v0.10.1

v0.10.0

Description

There are several large changes in release v0.10.0. These may require user action, as described below.

ocs-web

ocs-web has been removed from this repository. It has been replaced by an improved Vue.js version kept in simonsobs/ocs-web.

Current users of ocs-web should upgrade to the new Vue.js based version. Setup instructions, including how to run with the provided Docker image, are located in the ocs-web repo's README file.

OCS Plugins

v0.10.0 adds the OCS Plugin system. This has moved all core OCS Agents to be within the ocs package. (They were previously in a separate agents/ directory.) This allows core Agent installation via pip from PyPI.

During this change all core ocs Agent Docker images have been merged into a single image. All core Agents can now be run from the simonsobs/ocs image. Configuration of the image can now largely be done by setting environment variables. For example, to run a fake data Agent a docker-compose service would look like:

fake-data1:
    image: simonsobs/ocs:v0.10.0
    hostname: ocs-docker
    environment:
      - INSTANCE_ID=fake-data1
    volumes:
      - ${OCS_CONFIG_DIR}:/config:ro

Please see each Agent's respective reference page for updated configuration file information.

Separate Agent images will not be built moving forward, so in order to update you must switch to using the new simonsobs/ocs:v0.10.0 image.

Task Abort

Task aborting has been a long planned feature of ocs. It is now implemented. Agent developers can now add the ability to abort tasks. You can read about how to do so in the newly re-written Agent Writing Guide. More specifically see "Aborting a Task".

OCS now tries to run the Process stop command in the same threading context as the Process itself (the same is true for Task aborts.) Existing Agents with their process start/stop commands in differing contexts can still be run, but OCS will print a warning. Agent developers are encouraged to put their start/stops in the same context. See the Agent guide above for details.

What's Changed

New Features

• Create OCS Plugin system by @BrianJKoopman in #284
• Task abort and other stop fix-ups by @mhasself in #253

Documentation Updates

• docs: Address quickstart guide issues by @BrianJKoopman in #276
• Rewrite Agent developer guide by @BrianJKoopman in #279
• Fix broken link to agent docs template by @BrianJKoopman in #283
• docs: Fix path to ocs_plugin_standard.py in agent guide by @BrianJKoopman in #286

Bug Fixes

• Fix flaky registry test by @BrianJKoopman in #278

Misc.

• Remove OCS web by @mhasself in #280

Full Changelog: v0.9.3...v0.10.0

v0.9.3

What's Changed

• Replace docker login with docker/login-action by @BrianJKoopman in #267
• Add agent_class to encoded feed by @jlashner in #265
• Passes environment variables to docker subprocesses by @jlashner in #268
• Add boolean support to feeds by @BrianJKoopman in #269
• Create repr for OCSClient by @BrianJKoopman in #270
• Enable data feed buffer in fake data agent by @BrianJKoopman in #271

Full Changelog: v0.9.2...v0.9.3

v0.9.2

What's Changed

• Use pip installed so3g in GH Action workflows by @BrianJKoopman in #256
• Hostremaster by @mhasself in #234
• Cherry-pick better agent handling and logs from host manager work by @BrianJKoopman in #261
• Bump minimum python version, fix link, shuffle shields by @BrianJKoopman in #262

Full Changelog: v0.9.1...v0.9.2

v0.9.1

Description

v0.9.1 is a relatively small release. The only new feature is the availability of the new testing module to help out with testing Agents.

Change Log

New Features

• Move useful integration testing utils to main library (#252 )

Bug Fixes

• Remove testpypi from twine upload step (#249)

Misc.

• Remove spt3g/so3g dependency from the Registry (#250)
• tests: test_params for the params decorator system (#242)

v0.9.0

Description

v0.9.0 introduces some new tools and features, such as the ocs-client-cli tool and param decorator. Test coverage is also greatly expanded. One change in the API that requires attention is the renaming of MatchedClient to OCSClient. Users should update their scripts where MatchedClient is used, as it will be removed in a later release. Full change log below.

Change Log

New Features

• Introduce new Client CLI tool (#218)
• Create new decorator for tasks/processes that accept parameters (#223)
• Expand test coverage with new unit and integration tests (#230 and #231)
• Rename MatchedClient to OCSClient (#236)

Documentation Updates

• Some bug fixes and a bunch of OpSession documentation work (#220)
• Establish Agent documentation guidelines (#221)
• Add sphinx extension that auto-links to source code (#225)
• Create rtd config file, split requirements, and pin versions (#235)
• Update Client Documentation (#236)

Bug Fixes

• Check if args are passed to MatchedClient (#206)
• Deal gracefully with crossbar not being found (#208)
• Shutdown AggregatorAgent on PermissionError (#213)
• Protect test imports (#217)
• Some bug fixes and a bunch of OpSession documentation work (#220)
• Only perform check on param if value is given (#228)
• Update feed message data check to handle bools (#229)
• Core updates, prelude for major ocsbow (#241)

Misc.

• Reorganize Dockerfile to leverage build cache (#212)
• Add tests for ocs_agent.py (#227)
• Core updates, prelude for major ocsbow (#241)
• Build and Deploy to PyPI (#243)
• Prepare for v0.9.0 release (#244)