CHANGES.rst

Change History

The project milestones describe the future plans.

1.6.11

released 2023-02-15

New Features

• Support to record all motor (or other ophyd-labeled devices) positions at start (or end) of run.
• SpecWriterCallback: write #O and #P lines from motor positions recorded at start of run.

Fixes

• NXWriter: scans with area detectors did not have image data. Fixed. Clients must wait for writer to finish.
• PVPositionerSoftDone: computation of 'done' signal and 'inposition' deconvoluted.
• Struck3820: changed (typo) do_readl_all to do_read_all
• Tests involving PVPositionerSoftDone call '.cb_readback()' method to upate 'done' signal.

Maintenance

• Added github/super-linter workflow, adds mypy, isort, and other tests.
• Apply isort code style.

Known Problems

• Observing random occurrence of unit test failures involving PVPositionerSoftDone.

1.6.10

released 2023-01-11

New Features

• Add apstools.utils.plotxy() utility function.
• Add tag-based documentation selection via PyData switcher.

Enhancements

• In listruns(), also search the hints for keys.

Fixes

• CI failed to build documentation when creating pip installation.
• ScalerMotorFlyer default fly_time_pad increased to 10 s.
• ScalerMotorFlyer now sets scaler to "OneShot" mode.
• ScalerMotorFlyer in-position comparison now based on motor's precision.
• Version number was not reported correctly in certain situations (issue #771).

Maintenance

• "Custom HDF5 File Name" example updated to latest advice.
• Pin Sphinx to <6 due to problems with PyData and Sphinx v6.
• Conda channel "nsls2forge" no longer needed.
• Remove nsls2forge channel from conda environment.

New Contributors

• @jwkim-anl (Jong Woo Kim, ANL) for the plotxy() idea.

1.6.9

released 2022-11-30

New Features

• Add ScalerMotorFlyer() device.
• Add functions to support reporting of logging messages.
• Add restorable_stage_sigs decorator.
• Add support for Python 3.11.
• Add utils.analysis_1D() & utils.analysis_2D() functions for peaks statistics.

Enhancements

• Add example notebook Fly Scans with EPICS motor and scaler.
• Add guide How to Search in Databroker.
• Add guide How to setup logging.

Maintenance

• Add convenience import:from apstools.devices import AD_EpicsFileNameMixin.
• Enable: from apstools.devices import AD_EpicsFileNameMixin.
• Resolve intermittent, random CI failures.
• Resolve problems reporting unit test coverage statistics.
• Unit tests now support Python version 3.8, 3.9, 3.10, & 3.11.

1.6.8

released 2022-10-16

Fixes

• [again] Make sure that YML (and other) files are packaged for pip and conda.

1.6.7

skipped

1.6.6

released 2022-10-13

New Features

• Add request_input() plan stub, per user request.
• Add fly scan for scaler v. continuous motor.

Maintenance

• Rename: devices.make_dict_device() is now devices.dict_device_factory().

Fixes

• Make sure that YML (and other) files are packaged for pip and conda.
• PVPositioner got stuck if target position was the same as current position.

Deprecations

• Removed all snapshot support.

1.6.5

released 2022-10-04

Maintenance

• Pip requirements updated.

1.6.4

released 2022-10-03

New Features and/or Enhancements

• Example of imaging with AreaDetector in Single mode writing HDF5 files.
• lineup(): user can choose which feature (max, min, cen, com), additional API changes.
• devices.make_dict_device() to make a recordable Device from a dictionary.
• Only publish documentation on demand by executing GitHub workflow.
• Switch HTML documentation to use PyData theme.

Maintenance

• Re-arranged the documentation. Some examples renamed. Content unchanged.
• Update the APS cycle dates file through 2023-04-30.

1.6.3

released 2022-08-15

New Features and/or Enhancements

• Add devices.CamMixin_V3_1_1
• Add devices.CamMixin_V34
• Add devices.SingleTrigger_V34
• Add EpicsScanIdSignal (scan_id from EPICS PV).
• Add run_blocking_function() plan to run blocking functions in the RunEngine.
• Published on conda-forge conda install -c conda-forge apstools.
• replay() can now take a run, [run], header, or [header]
• Switch HTML documentation to use furo theme.

Maintenance

• Combine install steps, workflows in CI
• Describe how to use the NXWriter callback.
• Refactor AD test with EPICS-controlled image file names.
• Refactor test_move_to_zero().
• Unit tests now support Python version 3.8, 3.9, & 3.10.

Deprecations

• bluesky_snapshot_viewer and underlying GUI code will be dropped by 2022-12-31.
• Drop support for Python 3.7 per NEP29.
• Stop publishing on channel -c aps-anl-tag (use -c conda-forge instead).

1.6.2

released 2022-07-06

Notice

• Confirmed: databroker finds HDF5 image files with custom names.
• Still not ready for databroker 2.0+.

New Features

• Documentation website: https://bcda-aps.github.io/apstools/
• New example: user-controlled HDF5 image file names.
• Using Jupyter notebooks directly (via nbsphinx extension).
• User-controlled HDF5, JPEG, TIFF image file names.

Fixes

• Fix timeout problem in utils.connect_pvlist().
• Fix unexpected key in datum kwargs.
• Fix utils.listdevice() mixing dot and underline name separators.

Maintenance

• Applied custom project badge for APS software License.
• Use micromamba in testing workflows.

Deprecations

• Drop LGTM.com static code analysis service.
• Drop RTD (readthedocs) documentation publishing service.

Contributors

• Harry Zhou

1.6.1

released 2022-01-26

Fixes

• Move enable Component out from synApps Record devices.
• Renew the unit tests for PVPositionerSoftDoneWithStop.

1.6.0

released 2022-01-20

Breaking Changes

• Moved apsbss support to new apsbss package (install with either pip or conda). See https://bcda-aps.github.io/apsbss/ for details.
• Can use Python 3.7 - 3.9. Cannot use Python 3.10 yet due to upstream limitation from databroker and intake packages.
• Moved command_list_as_table() from utils into plans/command_list.
• Removed BusyStatus from apstools.synApps.busy
• callbacks/: DocumentCollectorCallback, document_contents_callback, and SnapshotReport moved into callbacks/.
• devices/: Reorganized all devices, including synApps/, into devices/ subpackage.
• devices/: SynPseudoVoigt() moved from signals/ to devices/.
• plans/: Reorganized plans.py and _plans/ into plans/ subpackage.
• snapshot/: Moved snapshot application and related files to a subdirectory.
• utils/: Reorganized utils.py and _utils/ into utils/ subpackage.

New Features and/or Enhancements

• Add support for Eurotherm 2216e temperature controller
• Add support for Lakeshore 336 temperature controller
• Add support for Lakeshore 340 temperature controller
• Add support for synApps calc scalcout record.
• Add support for synApps calc sseq record.
• Add support for EPICS base sub record.
• Add support for synApps calc userAve database.
• Add support for synApps calc userStringSeq database.
• Add support for synApps calc userStringCalc database.
• Add support for synApps optics 2slit database.

Fixes

• Convert None to "null" when saving PeakStats to stream.

Maintenance

Now testing with Python versions 3.7 - 3.9. (Can't use with Py3.10 yet due to upstream requirements.)

Update notebooks:

• demo_specfile_example
• demo_tuneaxis

Remove notebooks:

• demo_specfile_databroker

Deprecations

• Applications
  • apstools_plan_catalog application and related support.
• Devices
  • ApsCycleComputedRO
  • move_energy() method in KohzuSeqCtl_Monochromator class
  • ProcessController
• Utilities
  • device_read2table
  • json_export
  • json_import
  • listdevice_1_5_2
  • listruns_v1_4
  • object_explorer

Contributors

• Gilberto Fabbris
• Jan Ilavsky
• Qingteng Zhang

1.5.4

released 2021-11-25

NOTE: The apsbss component will be moved out of apstools into its own package with the next release (1.6.0, ~Feb 2022) of apstools.

Notice

The Python version is limited to 3.7 due to aps-dm-api package. Expect this limitation to be relaxed, allowing any Python 3.7 and higher with the 1.6.0 release.

Fixes

• Added table of APS run cycle dates. Use that if aps-dm-api not available.
• Restricted python version to 3.7 due to upstream aps_dm_api package.
• Rename name uid to token to avoid LGTM security false alert.

Deprecations

This support was marked as deprecated in release 1.5.4:

• apstools.devices.ApsCycleComputedRO

1.5.3

released 2021-10-15

Notice

The apstools.beamtime module and related content (includes apsbss) will be moved to a new repository for release 1.6.0. This will remove the requirement that the APS data management tools (package aps-dm, which only works on the APS computing network) be included. With this change, users will be able to conda install apstools -c aps-anl-tag on computers outside of the APS computing network.

Breaking Changes

• apstools.utils.listdevice has a new API (old version renamed to listdevice_1_5_2)

New Features and/or Enhancements

• Kohzu monochromator energy, wavelength, and theta each are now a PVPositioner (subclass).
• Linkam temperature controller CI94
• Linkam temperature controller T96
• Stanford Research Systems 570 current preamplifier
• Stanford Research Systems PTC10 temperature controller
• XIA PF4 filter now supports multiple PF4 units.
• Generalize that amplifiers will have a gain Component attribute.
• Generalize that temperature controllers will have a temperature Component attribute that is a positioner (subclass of ophyd.PVPositioner).
• Enhanced positioners for EPICS Devices:
  • apstools.devices.PVPositionerSoftDone
  • apstools.devices.PVPositionerSoftDoneWithStop

Fixes

• Fixed bug in devices.ApsCycleComputedRO and devices.ApsCycleDM involving datetime.

Maintenance

• Moved all device support into individual modules under apstools._devices because apstools.devices module was getting too big. Will refactor all with release 1.6.0.
• Add unit tests for devices.ApsCycle* Devices.
• Add EPICS IOCs (ADSimDetector and synApps xxx) to continuous integration for use in unit testing.
• Unit tests now use pytest package.
• Suppress certain warnings during unit testing.

Deprecations

This support will be removed in release 1.6.0:

• apstools.beamtime module and related content (includes apsbss) will be moved to a new repository
• apstools.devices.ProcessController
• apstools.utils.device_read2table
• apstools.utils.listdevice_1_5_2
• apstools.utils.object_explorer

Contributors

• Fanny Rodolakis
• Gilberto Fabbris
• Jan Ilavsky
• Qingteng Zhang
• 4-ID-C Polar
• 8-ID-I XPCS
• 9-ID-C USAXS

1.5.2 (and previous)

See this table for release change histories, highlighted by version control reference (pull request or issue):

1.5.2

released 2021-09-29

• Drop Codacy (https://app.codacy.com/gh/BCDA-APS/apstools) as no longer needed.

• #540

  Add apstools.utils.listplans() function.

• #534

  Add apstools.utils.OverrideParameters class. Hoisted from APS USAXS instrument.

• #537

  Enhancements to apstools.utils.listruns():

  • Add search by list of scan_id or uid values.
  • Optimize search speed.

• #534

  Add apstools.plans.documentation_run() plan. Hoisted from APS USAXS instrument.

• #528

  Add kind= kwarg to synApps Devices.

• #539

  Break devices into submodule _devices.

1.5.1

released 2021-07-22

• #522

  Deprecate apstools.devices.ProcessController. Suggest ophyd.PVPositioner instead.

• #521

  Enhancement: new functions: getRunData(), getRunDataValue(), getStreamValues() & listRunKeys()

• #518

  Bug fixed: TypeError from summary() of CalcoutRecord

• #517

  Added support for python 3.9.

• #514

  Refactor 'SIGNAL.value' to 'SIGNAL.get()'

1.5.0

released 2021-04-02

• #504 comment

  Dropped support for python 3.6.

• #495

  Dropped diffractometer support code.

• #511

  & #497 Add utils.findbyname() and utils.findbypv() functions.

• #506

  spec2ophyd can now read SPEC config files from APS 17BM

• #504

  Overhaul of listruns() using pandas. Previous code renamed to listruns_v1_4().

• #503

  Unit tests with data now used msgpack-backed databroker.

• #495

  remove hklpy requirement since all diffractometer support code will be moved to [hklpy](https://github.com/bluesky/hklpy) package.

1.4.1

released: 2021-01-23

• add Area Detector configuration examples: Pilatus & Perkin-Elmer, both writing image to HDF5 file

• #488

  use first trigger_mode when priming AD plugin

• #487

  ensure spec2ophyd code is packaged

1.4.0

released: 2021-01-15

• #483

  Python code style must pass flake8 test.

• #482

  specwriter: Fix bug when plan_args structure includes a numpy ndarray.

• #474

  apstools.utils.listruns() now defaults to the current catalog in use.

  New functions:

  • apstools.utils.getDatabase
  • apstools.utils.getDefaultDatabase

• #472

  Respond to changes in upstream packages.

  • package requirements
  • auto-detection of command list format (Excel or text)
  • use openpyxl instead of xlrd and pandas to read Microsoft Excel .xlsx spreadsheet files

• #470

  Area Detector plugin preparation & detection.

  • apstools.devices.AD_plugin_primed()

    re-written completely

  • apstools.devices.AD_prime_plugin()

    replaced by apstools.devices.AD_prime_plugin2()

• #463

  Remove deprecated features.

  • apstools.suspenders.SuspendWhenChanged()
  • apstools.utils.plot_prune_fifo()
  • apstools.utils.show_ophyd_symbols()
  • apstools.synapps.asyn.AsynRecord.binary_output_maxlength()
  • apstools.devices.AD_warmed_up()

• #451

  Undulator and Kohzu monochromator functionalities

  • apstools.devices.ApsUndulator()

    Adds some Signal components (such as setting kind kwarg) that are helpful in moving the undulator

1.3.9

released 2020-11-30

• #459

  apsbss: list ESAFs & proposals from other cycles

• #457

  apstools.utils.rss_mem(): show memory used by this process

1.3.8

released: 2020-10-23

• #449

  diffractometer wh() shows extra positioners

• #446

  utils: device_read2table() renamed to listdevice()

• #445

  synApps: add Device for iocStats

• #437

  diffractometer add pa() report

• #426

  diffractometer add simulated diffractometers

• #425

  BUG fixed: listruns() when no stop document

• #423

  BUG fixed: apsbss IOC starter script

1.3.7

released: 2020-09-18

• #422

  additional AD support from APS USAXS

• #421

  wait for undulator when start_button pushed

• #418

  apsbss: only update APS run cycle name after current cycle ends

1.3.6

released 2020-09-04

• #416

  apsbss: allow iso8601 time strings to have option for fractional seconds

• #415

  Get APS cycle name from official source

1.3.5

released 2020-08-25

• #406

  replace plot_prune_fifo() with trim_plot() and trim_plot_by_name()

• #405

  add Y1 & Z2 read-only signal to Kohzu Monochromator device

• #403

  deprecate SuspendWhenChanged()

1.3.4

released 2020-08-14

• #400

  resolve warnings and example documentation inconsistency

• #399

  parse iso8601 date for py36

• #398

  DiffractometerMixin: add wh() method

• #396

  provide spec2ophyd application

• #394

  add utils.copy_filtered_catalog()

• #392

  RTD make parameter lists clearer

• #390

  improve formatting of parameter list in RTD

• #388

  add utils.quantify_md_key_use()

• #385

  spec2ophyd: make entry point

1.3.3

released 2020-07-22

• #384

  apsbss: print, not log from commands

• #382

  spec2ophyd analyses

1.3.2

released 2020-07-20

• #380

  apsbss: fix object references

1.3.1

released 2020-07-18

• #378

  apsbss_ioc.sh: add checkup (keep-alive feature for the IOC)

• #376

  apsbss: example beam line-specific shell scripts

• #375

  apsbss: add PVs for numbers of users

• #374

  apsbss_ophyd: addDeviceDataAsStream() from USAXS

• #373

  account for time zone when testing datetime-based file name

• #371

  update & simplify the travis-ci setup

• #369

  spec2ophyd: handle NONE in SPEC counters

• #368

  spec2ophyd: config file as command-line argument

• #367

  apsbss: move ophyd import from main

• #364

  apsbss: add PVs for ioc_host and ioc_user

• #363

  Handle when mailInFlag not provided

• #360

  prefer logging to print

1.3.0

release expected by 2020-07-15

• add NeXus writer callback
• add apsbss : APS experiment metadata support

• #351

  apsbss: put raw info into PV

• #350

  apsbss: clarify meaning of reported dates

• #349

  apsbss: add "next" subcommand

• #347

  some apbss files not published

• #346

  publish fails to push conda packages

• #344

  listruns() uses databroker v2 API

• #343

  review and update requirements

• #342

  summarize runs in databroker by plan_name and frequency

• #341

  tools to summarize activity

• #340

  update copyright year

• #339

  resolve Codacy code review issues

• #338

  unit tests are leaving directories undeleted

• #337

  Document new filewriter callbacks

• #336

  add NeXus file writer from USAXS

• #335

  update requirements

• #334

  support APS proposal & ESAF systems to provide useful metadata

• #333

  access APS proposal and ESAF information

• #332

  listruns(): use databroker v2 API

• #329

  add NeXus writer base class from USAXS

1.2.6

released 2020-06-26

• #331

  listruns succeeds even when number of existing runs is less than requested

• #330

  BUG: listruns: less than 20 runs in catalog

• #328

  epid: add final_value (.VAL field)

• #327

  epid: remove clock_ticks (.CT field)

• #326

  BUG: epid failed to connect to .CT field

• #325

  BUG: epid final_value signal not found

• #324

  BUG: epid controlled_value signal name

1.2.5

released 2020-06-05

• #322

  add py38 to travis config

• #320

  multi-pass tune should use FWHM for next scan

• #318

  AxisTunerMixin is now subclass of DeviceMixinBase

• #317

  BUG: USAXS can't tune motors

• #316

  BUG: Error in asyn object definition

• #315

  BUG: AttributeError from db.hs

1.2.3

released 2020-05-07

• #314

  fix ImportError about SignalRO

• #313

  update packaging requirements

1.2.2

released 2020-05-06

• DEPRECATION #306

  apstools.plans.show_ophyd_symbols() will be removed by 2020-07-01. Use apstools.plans.listobjects() instead.

• #311

  adapt to databroker v1

• #310

  enhance listruns() search capabilities

• #308

  manage diffractometer constraints

• #307

  add diffractometer emhancements

• #306

  rename show_ophyd_objects() as listobjects()

• #305

  add utils.safe_ophyd_name()

• #299

  set_lim() does not set low limit

1.2.1

released 2020-02-18 - bug fix

• #297

  fix import error

1.2.0

released 2020-02-18 - remove deprecated functions

• #293

  remove run_blocker_in_plan()

• #292

  remove list_recent_scans()

• #291

  remove unix_cmd()

• #288

  add object_explorer() (from APS 8-ID-I)

1.1.19

released 2020-02-15

• #285

  add EpicsMotorResolutionMixin

• #284

  adjust ophyd.EpicsMotor when motor limits changed from other EPICS client

• #283

  print_RE_md() now returns a pyRestTable.Table object

1.1.18

released 2020-02-09

• PyPI would not accept the 1.1.17 version: filename has already been used
• see release notes for 1.1.17

1.1.17

released 2020-02-09 - hot fixes

• #277

  replace .value with .get()

• #276

  update ophyd metadata after motor set_lim()

• #274

  APS user operations could be in mode 1 OR 2

1.1.16

released 2019-12-05

• #269

  bug: shutter does not move when expected

• #268

  add redefine_motor_position() plan

• #267

  remove lineup() plan for now

• #266

  bug fix for #265

• #265

  refactor of #264

• #264

  Limit number of traces shown on a plot - use a FIFO

• #263

  device_read2table() should print unless optioned False

• #262

  add lineup() plan (from APS 8-ID-I XPCS)

1.1.15

released 2019-11-21 : bug fixes, adds asyn record support

• #259

  resolve AssertionError from setup_lorentzian_swait

• #258

  swait record does not units, some other fields

• #255

  plans: resolve indentation error

• #254

  add computed APS cycle as signal

• #252

  synApps: add asyn record support

1.1.14

released 2019-09-03 : bug fixes, more synApps support

• #246

  synApps: shorten name from synApps_ophyd

• #245

  swait & calcout: change from EpicsMotor to any EpicsSignal

• #240

  swait: refactor swait record & userCalc support

• #239

  transform: add support for transform record

• #238

  calcout: add support for calcout record & userCalcOuts

• #237

  epid: add support for epid record

• #234

  utils: replicate the unix() command

• #230

  signals: resolve TypeError

1.1.13

released 2019-08-15 : enhancements, bug fix, rename

• #226

  writer: unit tests for empty #O0 & P0 control lines

• #224

  rename: list_recent_scans --> listscans

• #222

  writer: add empty #O0 and #P0 lines

• #220

  ProcessController: bug fix - raised TypeError

1.1.12

released 2019-08-05 : bug fixes & updates

• #219

  ProcessController: bug fixes

• #218

  replay(): sort chronological by default

• #216

  replay(): fails when not list

1.1.11

released 2019-07-31 : updates & new utility

• #214

  new: apstools.utils.APS_utils.replay()

• #213

  list_recent_scans show exit_status

• #212

  list_recent_scans show reconstructed scan command

1.1.10

released 2019-07-30 : updates & bug fix

• #211

  devices calls to superclass __init__()

• #209

  devices call to superclass __init__()

• #207

  show_ophyd_symbols also shows labels

• #206

  new: apstools.utils.APS_utils.list_recent_scans()

• #205

  show_ophyd_symbols uses ipython shell's namespace

• #202

  add labels attribute to enable wa and ct magic commands

1.1.9

released 2019-07-28 : updates & bug fix

• #203

  `SpecWriterCallback`: #N is number of data columns

• #199

  spec2ophyd handle CNTPAR:read_misc_1

1.1.8

released 2019-07-25 : updates

• #196

  spec2ophyd handle MOTPAR:read_misc_1

• #194

  new show_ophyd_symbols shows table of global ophyd Signal and Device instances

• #193

  spec2ophyd ignore None items in SPEC config file

• #192

  spec2ophyd handles VM_EPICS_PV in SPEC config file

• #191

  spec2ophyd handles PSE_MAC_MOT in SPEC config file

• #190

  spec2ophyd handles MOTPAR in SPEC config file

1.1.7

released 2019-07-04

• DEPRECATION

  apstools.plans.run_blocker_in_plan() will be removed by 2019-12-31. Do not write blocking code in bluesky plans.

• Dropped python 3.5 from supported versions

• #175

  move plans.run_in_thread() to utils.run_in_thread()

• #168

  new spec2ophyd migrates SPEC config file to ophyd setup

• #166

  `device_read2table(): format `device.read() results in a pyRestTable.Table

• #161

  `addDeviceDataAsStream()`: add Device as named document stream event

• #159

  convert xlrd.XLRDError into apstools.utils.ExcelReadError

• #158

  new run_command_file() runs a command list from text file or Excel spreadsheet

1.1.6

released 2019-05-26

• #156

  add ProcessController Device

• #153

  print dictionary contents as table

• #151

  EpicsMotor support for enable/disable

• #148

  more LGTM recommendations

• #146

  LGTM code review recommendations

• #143

  filewriter fails to raise IOError

• #141

  ValueError during tune()

1.1.5

released 2019-05-14

• #135

  add refresh button to snapshot GUI

1.1.4

released 2019-05-14

• #140

  event-model needs at least v1.8.0

• #139

  ValueError in ~apstools.plans.TuneAxis.tune._scan

1.1.3

released 2019-05-10

• adds packaging dependence on event-model

• #137

  adds utils.json_export() and utils.json_import()

1.1.1

released 2019-05-09

• adds packaging dependence on spec2nexus

• #136

  get json document stream(s)

• #134

  add build on travis-ci with py3.7

• #130

  fix conda recipe and pip dependencies (thanks to Maksim Rakitin!)

• #128

  SpecWriterCallback.newfile() problem with scan_id = 0

• #127

  fixed: KeyError from SPEC filewriter

• #126

  add uid to metadata

• #125

  SPEC filewriter scan numbering when "new" data file exists

• #124

  fixed: utils.trim_string_for_EPICS() trimmed string too long

• #100

  fixed: SPEC file data columns in wrong places

1.1.0

released 2019.04.16

• change release numbering to Semantic Versioning (remove all previous tags and releases)
• batch scans using Excel spreadsheets
• bluesky_snapshot_viewer and bluesky_snapshot
• conda package available