-
Notifications
You must be signed in to change notification settings - Fork 0
Guidance on Carrier Class Creation
This guide is intended for anyone looking to create a new odin-control instance to run on a LOKI-based system (that is, for a project using the stfc-aeg/loki toolflow as its basis that will be running on LOKI hardware).
References to the actual creation of images for LOKI-based systems, altering the firmware design, and actually installing this adapter into the build are out of scope. Creating new carrier base classes is also out of scope. See Related.
It is assumed that the top-level project has already been created elsewhere, and that this 'control' portion of the project is either within the overall application repository (in a toplevel ./control directory) or a standalone repository that will be installed into the image with Yocto (see Yocto Layer for Odin Control for further details).
This part of the project will as a minimum include:
- Python source for any application-specific odin-control adapters
- Configuration file(s) for odin-control
And optionally:
- Sequences for the odin-sequencer
#TODO Explain general structure of a project, what ties into what, how ABCs have been used (some of this may go in Maintainer LOKI Carrier Class Documentation. #TODO generally advertise what functionality is available in the base code already (in less detail than below summary)
#TODO Talk about function visibility- anything without an underscore should be considered 'resonable callable' by someone running sequences from the UI. Everything else should be hidden. I have a repeated pattern of external functions, internal loops, and internal direct read functions. See the thread creation section.
These general steps assume that you are currently in the control root directory, either in a control repository for your project, or a subdirectory in a general project.
Create a directory for python adapter source code
mkdir src
mkdir src/carriertouch src/carrier/carrier.py
touch src/carrier/__init__.pyThe bulk of control functionality for your project will be provided with a carrier class (meant to represent the whole system aside from the ASIC).
Since there are so many common elements, the LOKI core project contains a LokiCarrier_1v0 class, which (by making liberal use of abstract base class functionality) can be extended to meet varied applications. Source code for this class can be found here.
Much of the parameter tree is also standardised and created by the base class, so see Extending the Parameter Tree.
Create your own derived carrier class with the following basic content:
from loki.adapter import LokiCarrier_1v0, DeviceHandler
class LokiCarrier_BabyD (LokiCarrier_1v0):
def __init__(self, **kwargs):
self._logger = logging.getLogger('LOKI-based Carrier')
# Must currently be set to ~something~ due to bug
self._default_clock_config = 'ZL30266_LOKI_Nosync_500MHz_218MHz.mfg'
kwargs.setdefault('clkgen_base_dir', './clkgen/')
# Potentially instantiate your ASIC here (RegisterController)
self._logger.info('ASIC instance creation complete')
# MUST call the superclass init LAST
super(LokiCarrier_BabyD, self).__init__(**kwargs)
self._logger.info('LOKI super init complete')
def _gen_app_paramtree(self):
# This custom parameter tree function must be overridden, even if it is empty for now
custom_pt = {
}
return custom_ptCreate your adapter class as normal, but it is suggested that you read and make use of the functionality provided by the base class, documented in detail below.
#TODO General suggestions, not requirements: just explain how the register controller works
Create the adapter source file. The exact contents of this is beyond scope, but it should at least instantiate your new carrier class. See the BabyD version for an example.
touch src/carrier/adapter.pyNote that it will also need to provide context for functions within the carrier and ASIC classes to be accessible from the sequencer.
An example of this (where the ASIC is instantiated inside the carrier) from BabyD looks like this (added to the ApiAdapter's initialize function:
self.adapters['odin_sequencer'].add_context('carrier', self.carrier)
self.adapters['odin_sequencer'].add_context('asic', self.carrier._asic)The Python module now needs a setup file. This is a standardised file and out of scope. However, some considerations:
-
install_requiresshould includeodin-control - The repository URL should point to your own repository
- Use the following to specify the location of project source:
setup(
...
packages=find_packages('src'),
package_dir={'': 'src'},
)
An odin-control instance will require a config file to tell the server what to do.
It can be created wherever you like, but for consistency I would create it here (from the control directory):
mkdir config
touch config/config.cfgThe contents should look something like this:
[server]
debug_mode = 1
http_port = 8888
http_addr = 127.0.0.1
static_path = static
adapters = detector, odin_sequencer
[tornado]
logging = info
[adapter.detector]
module = carrier.adapter.CarrierAdapter
clkgen_base_dir = ./clkgen/
[adapter.odin_sequencer]
module = odin_sequencer.adapter.CommandSequenceManagerAdapter
sequence_location = sequencesTo include this adapter in the Yocto build, it needs its own recipe in the application Yocto user layer. See Yocto Layer for Odin Control.
#TODO Generally introduce what is already available in the base carrier, and mention that many of these things lead to a standardised parameter tree.
After instantiation, the LOKI base class will create several threads of its own for various tasks, owned by one ThreadPoolExecutor.
This is for three main reasons:
- Preventing long-duration tasks from hanging the main thread: If all tasks ran in one thread, anything that took a long time (seconds) -like the ASIC bring-up process involving setting up various PCB peripherals- would hog the main thread, preventing other processing from occurring, and additionally freezing up the user interface. This is jarring for the user, and would bring into question the reliability of any control loop- any events that needed a relatively quick response would also not run.
- Caching state to decouple external requests from hardware access rate: The
- Separation of Error Handling: While errors should be handled properly by a caller, running tasks in separate threads means that if something does trigger an uncaught error, the system will not go down completely. For example, if a thread meant to update readings from a humidity sensor fails, the task meant to track and report high voltage state will continue. Additionally, tracking the origin of errors is easier when you know the thread it was raised from.
The following threads currently run under the LOKI base adapter, to be supplemented by the developer writing an application specific carrier:
- ams - reads Zynq internal temperatures and caches them for readback
- env - regularly reads sensor values and caches them for readback - typically extended by the application code to include sensors from external boards
- gpio - regularly reads GPIO pin states and caches them for readback
- perf - reads Zynq performance information and caches for readback
- watchdog - monitors existing running threads and reports their current state, error status, and whether they have kicked the watchdog recently enough (if applicable)
In the LOKI carrier, threads are added with a friendly name, which is then used when reporting thread status etc.
To add a thread to a carrier:
self.add_thread(<friendly name>, <function to run>, <optional args>)A target function can take no arguments, named and/or unnamed arguments as desired, supplied as arguments at the end of this call.
Note that this can only be called after the initialisation of the carrier has completed, or at least, once the init of the base carrier superclass has been called in the subclass init.
Therefore to make this easier, it is typical to add all threads to a custom _start_io_loops function, which is called automatically.
However, this must first call the superclass function.
See the typical implementation below.
The LOKI Watchdog allows you to catch hanging threads by enforcing a minimum time between 'kicks' received from that thread. If the thread fails to send a kick after the duration since the last has elapsed, it is assumed to have hanged. As a result of this:
- The error will be recorded in the log
- The state of the thread in the parameter tree will indicate that the watchdog has tripped
- An optional callback will be called so that the user can handle the error- perhaps shutting the system down if this is a critical process.
To add a kick duration to an already added thread:
self.watchdog_add_thread(<friendly name>, <timeout secs>, <callback function (optional)>)Some useful functions:
-
watchdog_kick(self, thread_name=None)Kick the watchdog for a given thread name, or if one is not supplied, kick for the thread calling the function. -
watchdog_remove_thread(self, threadname)Remove a named thread from the watchdog completely -
watchdog_pause_thread(self, threadname=None)Stop monitoring a thread - could be used to ignore a particularly long-running operation. If not given athreadname, will use the thread that is calling the function. Not yet implemented -
watchdog_resume_thread(self, threadname=None)Re-start monitoring a thread. Not yet implemented
As mentioned, it is typical to set up the threads in a _start_io_loops function, that also calls the superclass function first.
This is an example from HEXITEC-MHz:
def _start_io_loops(self, options):
# override IO loop start to add loops for this adapter
super(LokiCarrier_HMHz, self)._start_io_loops(options)
self.add_thread('enable_state_machine', self._mhz_enable_state_machine_loop)
self.watchdog_add_thread('enable_state_machine', 10, lambda: logging.error('!!!! Enable State Machine Loop watchdog triggered !!!!'))
self.add_thread('adc_update', self._mhz_adc_update_loop, update_period_s=5)
self.watchdog_add_thread('adc_update', 10, lambda: logging.error('!!!! ADC Loop watchdog triggered !!!!'))
self.add_thread('firefly_channel_loop', self._mhz_firefly_channel_loop)
self.watchdog_add_thread('firefly_channel_loop', 10, lambda: logging.error('!!!! Firefly Loop watchdog triggered !!!!'))These threads all use lambda to define what happens when the watchdog detects a hanging thread.
For the adc_update thread, an additional argument named update_period_s is passed through to the target function.
For threads that have been added in this way, the following parameter tree entry format will appear at <adapter base>/carrier_info/loopstatus/:
"env": {
"running": true,
"done": false,
"exception": "N/A",
"wd_state": "OK"
},-
runningwill returntrueif the task is still running, even if it is hanging - The
donestatus indicates that the function has returned or thrown an error (more typical given most of these threads run continuously). -
exceptionwill return"N/A"if there is no exception, and the error from Python otherwise. -
wd_statewill return"OK"if the watchdog kick has been received in time,"N/A"if the thread is not being monitored, and"Triggered"if the thread missed a kick.
On exit, the LOKI base class will set self.TERMINATE_THREADS to True.
Threads should usually loop while this is False:
def example_thread_function(self):
## SETUP
while not self.TERMINATE_THREADS:
self.watchdog_kick()
## DO STUFF
time.sleep(1)In addition, there is a carrier.cleanup() function that should be called by any adapter implementing a LOKI carrier derived class.
You may extend this function in your carrier to perform any cleanup operations (such as placing the detector in a safe state), but you must use super to call the superclass method.
Code in the adapter (cleanup() is called automatically by odin-control):
def cleanup():
self.carrier_instance.cleanup()And the carrier function:
def cleanup(self):
# The cleanup function is called by odin-control on exit, for example if reloaded in debug mode
# by a file edit.
# This will terminate all threads, after setting main enable to false. This should also set SYNC
# low first, ensuring that the ASIC completes its last packet before going down.
self._exit_nicely()
super(LokiCarrier_BabyD, self).cleanup()LOKI makes GPIO available to Python through the use of libgpiod (or at least, the Python bindings for it).
The process for requesting access to these pins in various modes is relatively complicated, so is handled internally by reading configuration from the carrier kwargs supplied during initialisation.
This means that:
- You don't have to deal with the line requests directly- just provide some basic information
- Many control lines that are standardised are taken care of automatically by the base carrier (such as the
application_enableline for example), but can be overridden by the application code - Pin definitions can also be overridden with settings added to the config file, on a live system (great for debugging!)
When 'registered' pins are given a friendly internal name, which can then be used when getting or setting the pin state.
For example, if a pin for firefly_en has been set up already as an output, to set the pin value:
self.set_pin_value('firefly_en', True)There are also:
-
get_pin_value(self, friendly_name)to return the currently set value (output pins) or current input reading (input pins) -
is_pin_active_high(self, friendly_name)returnsTrueif the pin will go high voltage when setTrue -
is_pin_input(self, friendly_name)returnsTrueif the pin has been set up as an input pin -
get_pin(self, friendly_name)returns the gpiod pin directly, in case of more advanced usage -
get_pin_names(self)returns the friendly names of all pins set up
| Pin Friendly Name | Default GPIOD Pin (can be overridden) | Direction | Description |
|---|---|---|---|
app_present |
APP nPRESENT |
Input | Will read as True if the application board is present (application defined) |
bkpln_present |
BACKPLANE_nPRESENT |
Input | Will read as True if the backplane board is present (application defined) |
app_en |
APPLICATION nRST |
Output | If set True should enable the application |
per_en |
PERIPHERAL nRST |
Output | If set True should enable peripheral devices (regulators for example) |
These pins use special gpiod pins pre-allocated for certain functions, see [[#gpiod Pin Names]] |
Setting up pins is achieved by setting kwargs, which are then read by the pin handler.
These are generally in the format pin_config_<setting>_<pin friendly name>.
To set up an output pin on EMIO22 with the friendly name firefly_en:
# In the carrier __init__()
kwargs.setdefault('pin_config_id_firefly_en', 'EMIO22')
kwargs.setdefault('pin_config_active_low_firefly_en', False)
kwargs.setdefault('pin_config_is_input_firefly_en', False)
kwargs.setdefault('pin_config_default_value_firefly_en', 0) # Active high so disabled by default| Setting Name | Required? | Description |
|---|---|---|
pin_config_id_<pin friendly name> |
Yes | This is what sets the association between friendly name and gpiod line ID as seen by the libgpiod system |
pin_config_active_low_<pin friendly name> |
No | If set True will set the output to 0v when a True values is written to the pin. If the pin is an input, a high voltage in will be read back as False (default: False). |
pin_config_is_input_<pin friendly name> |
Yes | If set True, pin will be an input. Otherwise, an output |
pin_config_default_value_<pin friendly name> |
Yes (for outputs only) |
True/False. This is the value that will be set on an output pin immediately after initialisation, respecting the active high/low logic. For example, if a pin is active low and this value is set to True, the pin will start off at 0v. |
The use of setdefault makes it possible to override these values in the configuration file using the same names.
Refer to above for settings.
This is the same, but with pin_config_is_input_<> set True, and not requiring the setting of a default value:
# MIC284 interrupt pin
kwargs.setdefault('pin_config_id_tint', 'EMIO27')
kwargs.setdefault('pin_config_active_low_tint', True)
kwargs.setdefault('pin_config_is_input_tint', True)If one of the pins reserved by the base carrier (app_en for example) is configured by default with undesireable settings, it is possible to override them in the application carrier class init.
For example, the app_en signal is requested in the base class as active low.
If for your application you need an active high enable pin, you could alter the setting:
# In the application carrier self.__init__()
kwargs.update({'pin_config_active_low_per_en': False})If you'd just like to override a pin setting as a temporary test, consider doing so in the configuration file. For the same example as above under the adapter's section:
pin_config_active_low_per_en = TrueThis could also potentially be useful for debugging pin control; if you supply a different value for
pin_config_id_<pin name>, everything will behave as normal except the named signal will appear on a different pin.
When requesting pins, the names given are for the gpiod line name.
If you have access to the LOKI system terminal, these can be listed with the gpioinfo command:
petalinux-custom:~$ gpioinfo
gpiochip0 - 174 lines:
line 0: "MIO0" unused input active-high
line 1: "MIO1" unused input active-high
...
line 89: "CLKGEN AC2" unused input active-high
line 90: "LTC_NRST" "LOKI 1v0 HEXITEC-MHz" output active-low [used]
line 91: "LED0" "LOKI 1v0 HEXITEC-MHz" output active-high [used]
line 92: "LED1" "LOKI 1v0 HEXITEC-MHz" output active-high [used]
...
line 107: "EMIO29" "LOKI 1v0 HEXITEC-MHz" output active-high [used]
line 108: "EMIO30" "LOKI 1v0 HEXITEC-MHz" output active-high [used]
line 109: "EMIO31" unused input active-high
line 110: "EMIO32" unused input active-high
...There are three general types of line available:
-
MIOn: Multiplexed IO: These are pins directly tied to the processing system for certain peripheral interfaces. They are on fixed pins, and none have been routed out to LOKI's external connectors, so they are irrelevant. -
EMIOn: Un-allocated Extended Multiplexed IO: These are general purpose control lines available for application use. Exactly where these are routed to the main connector / elsewhere will depend on the application-specific constraints file. See Zynq Control - Guidance on Altering Hardware Design. -
<Other Names>: Standardised LOKI Control Lines: Other named lines are those that have been assigned a specific purpose within LOKI systems. Typically these will have already been configured by the LOKI base adapter, but can be overridden if desired. Again, exact mapping of these to physical pins is more or less fixed, but can be overridden in application specific constraints.
This is out of scope for this document, but for the sake of example, the firefly_en signal shown above as an output pin setup example is for HEXITEC-MHz, where the constraint is specified as follows:
# Firefly Reset (shared) (EMIO22)
# Rebind EMIO 22 to the desired package pin
# HMHz FF_RESET# -> SEAF C19 -> LOKI HP_G4_L9_P -> J4:62 -> B64_L23_P -> AH2
set_property PACKAGE_PIN AH2 [get_ports {GPIO_APP_21_31[1]}]Here, the package pin for EMIO22 (the ID used in Python and gpiod) is routed to package pin AH2, which eventually makes its way to the LOKI main connector pin C19, which for this application is routed to the FireFly enable signal.
Some of the standardised pins above are for 'user defined' IO: LEDs and buttons. However, you don't need to (shouldn't) interface with these pins directly- there are helper functions.
The LOKI v1 carrier has four LEDs and two user buttons (friendly names displayed): !loki_buttons_leds.jpg
These can be set with leds_set_led(self, friendly_name, value) while current state is returned with leds_get_led(self, friendly_name).
The LEDs are also exposed to the parameter tree, both read and write under user_interaction/leds/<friendly name>.
Button state is read with buttons_get_button(self, friendly_name).
The buttons also appear in the parameter tree, read-only at user_interaction/buttons/<friendly name>.
The gpiod bindings use the term 'chip' to refer to each separate GPIO bus.
In the majority of LOKI systems, there is only one bus: the default one provided as part of the ZynqMP IP called /dev/gpiochip0.
However, if more (AXI) GPIO buses have been added to the system in the hardware design, this may no longer be the case.
If this is true:
- You must specify the chip number of the base bus using the kwarg
zynqmp_base_gpio_chip_num, which will be picked up by the pin handler. It is suggested that you do this in the carrier__init__()before thesuper()call. - If lines are being identified by unique names added in the device tree, this will continue to work as normal assuming that the names are unique including all buses present
- If lines are being identified by line numbers, you will need to specify a chip number too. This is done in the same manner as other pin options with
<pinname>_chipnum. By default, the chip will be assumed to be the base chip for the ZynqMP platform. Make sure this is specified correctly (see top point).
The LOKI carrier implements an 'environmental monitoring' loop intended to gather sensor readings for the general environment of the application system. This is typically for temperature and humidity readings.
Once configured, the look will periodically request readings from all sensors, add them to a standardised cache, and report them in a standard format to the UI via the parameter tree. Due to this, any sensors added will automatically appear in the LOKI-specific environment monitoring aspects of the UI without the developer having to do anything additional.
Sensors are sub-categorised by 'type', which by default is either 'temperature' or 'humidity'.
flowchart LR
subgraph Direct Read Functions
lokisens[LOKI Sensors]
appsens[APP Sensors]
end
subgraph Env Thread
thr(Thread)
lokisens <-.->|5s| thr
appsens <-.->|5s| thr
end
subgraph Cache
temp[(temperature)]
hum[(humidity)]
othr[(other...)]
thr <-->|5s| temp
thr <-->|5s| hum
thr <-->|5s| othr
end
subgraph UI
PT[Parameter Tree]
seq[Sequences]
ui[UI]
temp <==>|rapid| PT
hum <==>|rapid| PT
othr <==>|rapid| PT
temp <==>|rapid| seq
hum <==>|rapid| seq
othr <==>|rapid| seq
PT <==>|rapid| ui
end
Rate limiting is handled by the thread, which provides external functions for reading cached values from the carrier context as well through the parameter tree. Therefore, adding a sensor is as simple as providing a function to directly read the value.
There are two parts to adding a sensor:
- Adding to the configuration to
_env_sensor_info - Creating / adding to an
_env_get_sensorfunction that will be called by the thread
As an example, we will add a temperature sensor that has an imaginary function to get temperature already implemented in the application carrier class called example_get_temp(self).
Each sensor must have an entry in a class variable self._env_sensor_info that tells the system about the sensor name, type, and gives a short description.
Sensors are allowed the same name if they have different types (for example, you might have the same physical chip able to return a temperature and humidity reading).
This is simply a list, but will already have entries for the LOKI system sensors, so add to it with extend:
self._env_sensor_info.extend([
('EXAMPLE', 'temperature', {"description": "Example Temperature Reading", "units": "C"}),
])This list can contain multiple additions at once
Note: It is suggested that sensors names make sense in the context of the application if possible, so rather than being named after a device like 'MIC284', they should be named 'Power Board' so that it is obvious what it represents. The actual device is not relevant.
The second part is adding the function call to read the given sensor.
This is done by re-implementing the _env_get_sensor(self, name, sensor_type) function.
This will be called at regular intervals by the thread, where name and sensor_type are strings supplied above for the given sensor.
The function should return the reading directly for whatever sensor is relevant, but should also try the superclass if the sensor / type combination is not supported in your class, as it may be internal.
For example:
def _env_get_sensor(self, name, sensor_type):
try:
# First try the superclass
return super(LOKICarrier_MHz, self)._env_get_sensor(name, sensor_type)
except NotImplementedError:
# Handle custom application sensors
if name == 'EXAMPLE'
if sensor_type == 'temperature':
return self.example_get_temp()
# If not implemented in superclass or here must be incorrect
raise NotImplementedError('Sensor does not exist')#TODO
The LOKI v1 carrier uses a ZL30266 clock generator for external clock generation. However, the API for clock generation is not device specific; the IC is configured automatically at initialisation, making available some functions for the adapter to use:
-
clkgen_get_config_avail(self)Returns the available configuration names as a list -
clkgen_set_config(self, config)Sets a named configuration, from the list returned above
These are also exposed automatically through the parameter tree, and a standard LOKI React component exists to create a drop-down of available configurations and allow selection without any implementation in the application carrier (besides specifying configuration file location).
The directory in which the system will look for valid configuration files (.mfg for ZL30266) must be set with argument clkgen_base_dir.
This can be done in the config file or in the application carrier initialisation.
If this is done with kwargs.setdefault('clkgen_base_dir'), it can be overridden in the configuration file.
The LOKI carrier provides a standardised set of functions for accessing numbered DAC outputs starting from 0, no matter the numbering on whatever IC is used to generate them. On the LOKI carrier, these are numbered 0-7.
Configuration is not necessary in the application carrier implementation, and parameter tree entries are already added.
Functions provided:
-
dac_set_output(self, output_num, value): Set a value with the DAC on a given output number -
dac_get_output(self, output_num): Get the DAC output value on a given output number. For the MAX5306 specifically this simply returns the last sent value, as there is no native readback.
Although the DAC output setting already appears in the parameter tree, on some occasions I have re-wrapped calls to the external functions under different names in the parameter tree to provide more information.
For example, HEXITEC-MHz uses the DAC for VCAL and adds additional checks to safe output range while still making use of the existing generic DAC access functions:
def set_vcal_in(self, value):
# Use the DAC functionality provided by the generic carrier to set VCAL on DAC
# DAC output 0 (SEAF:B50).
# Check that we are in range for HEXITEC-MHz (nothing greater than 1.2)
if value >= self._vcal_in_limit:
raise Exception('VCAL cannot be set to {}v; it exceeds the limit {}v. To change the limit, set "vcal_in_limit" in the config file.'.format(
value, self._vcal_in_limit))
else:
self.dac_set_output(0, value) # Output number is LOKI count, not MAX5306 countIn addition, anyone writing sequences does not need to know that the DAC output number associated with VCAL is 0.
#TODO
#TODO introduction: of course people will want to add functionality that isn't present in the existing code. There are some things that could make this easier.
The main function for creating the parameter tree is already in the LOKI base class, since much of the parameter tree is now standardised between systems.
However, there is a top-level branch added called application for application-specific additions.
These additions can be made by returning a custom dictionary from an application-implemented _gen_application_paramtree(self) function.
The contents of this dictionary will be added as a parameter tree under application in the full tree.
#TODO mostly achieved through device handlers. The ASIC has its own one however.
#TODO #TODO Using the device handler class #TODO Accessing I2C and SPI devices, what information (bus names) are needed
- ( #TODO update) Zynq Control - Guidance on Carrier Class Creation
- #TODO link to guide on Yocto Recipes for Odin Instances
- #TODO link to guide on pin mappings and IO
- #TOOD link to guide on creating / modifying carrier base classes