Adding a batch of plans to the queue based on uploaded spreadsheet

[dmgav]

Experimental feature that allows to submit a batch of plans to the queue by uploading spreadsheet to HTML server:

http --form POST http://localhost:60610/queue/upload/spreadsheet spreadsheet@<path-to-spreadsheet>

where <path-to-spreadsheet> is a path to the Excel spreadsheet. Now only custom Excel spreadsheet for submitting sequences of xafs plan to BMM beamline are supported. A beamline may use several types of spreadsheets for different types of experimental work. The API allows to pass spreasheet type as optional data_type parameter::

http --form POST http://localhost:60610/queue/upload/spreadsheet spreadsheet@<path-to-spreadsheet> data_type=<spreadsheet-type>

Spreadsheet type (data_type) is a user defined string, which is recognized by the beamline-specific custom function, which is called for processing the spreadsheet. There is no standard for beamline-specific spreadsheet types.

By default, spreadsheets are processed by built-in default function, which generates a list of plans based on the contents of spreadsheet (this function is not implemented yet). Since many users would prefer to use custom spreadsheets and write their own code that converts the spreadsheets to lists of plans, the server implements mechanism that allows to substitute default processing function. The custom function must be named spreadsheet_to_plan_list with matching signature and be importable from an installed python module (from <some_module> import spreadsheet_to_plan_list should import the function). The name of the module is passed to the server by setting environment variable BLUESKY_HTTPSERVER_CUSTOM_MODULE to the name of the module before starting the server. For example, the server may be started with custom module for BMM beamline as follows:

export BLUESKY_HTTPSERVER_CUSTOM_MODULE=bluesky-httpserver-bmm 
uvicorn bluesky_queueserver.server.server:app --host localhost --port 60610

('-' are allowed in module name: they are automatically replaced by '_'). It is assumed that the same module will be used to keep all code that is specific for a given beamline. If the server is started with the custom module, the users will still be able to access the default processing function: if the custom function is returning None (in response to certain value of data_type or missing datatype), then the server will call the default function. Custom function is expected to raise an exception with error message if it fails to process spreadsheet due to error.

Operation of the server was tested using custom processing module for BMM beamline NSLS-II-BMM/bluesky-httpserver-bmm#1 using the following simulated startup script:

# Contain plan headers for testing of queue execution for BMM beamline
# Plans: mv, xafs, change_edge, shb_close_plan, set_slot

from ophyd.sim import hw
import ophyd

from bluesky.plans import count, scan
from bluesky.plan_stubs import mv

from bluesky_queueserver.manager.profile_tools import set_user_ns

det1, det2, motor = hw().det1, hw().det2, hw().motor

# Those are devices used with 'mv' plans
xafs_x = motor
xafs_y = motor
slits3_hsize = motor
xafs_det = motor

@set_user_ns
def xafs(inifile=None, *, user_ns, **kwargs):
    yield from count([det1, det2], num=5, delay=1)


@set_user_ns
def change_edge(el, focus=False, edge='K', energy=None, slits=True, target=300., xrd=False, bender=True, *, user_ns):
    yield from scan([det1, det2], motor, -1, 1, 10)


def shb_close_plan():
    yield from count([det1, det2])


def set_slot(self, n):
    yield from count([det1])

Addresses the issue #132