Develop model for single-file input modules #525
Comments
|
One issue I foresee is how to properly determine the options to display for each Input on the Data page. I may have to list all available variables (e.g. 'resolution', 'sensitivity', etc.) in |
|
I had been wondering about that. This dust sensor has two modes. We are using the default power-up mode right now. I'm not sure how to present a second mode that would only run the sample fan just before and during a sample in the Q&A mode without it being a separate sensor variant. The logic to manage the device changes dramatically. |
|
My current progress includes adding the following dictionary to 2 test modules (test_humidity_01.py and test_temperature_01.py): Mycodo/mycodo/inputs/custom_inputs/test_humidity_01.py Lines 15 to 55 in dcb76f6 Then, I created a parser (parse_inputs.py) that iterates through the modules in the directory and compiles a dictionary of all inputs and their information. The output from running the parser is this: Now I need to begin modifying the various parts of the system that rely on reading this information from various non-central sources: config_devices_units.py (1, 2), utils_input.py, controller_input.py, and Flask templates for input options. |
|
Since this parser takes quite a while to complete (2+ seconds), this should only need to be executed once and stored by the running daemon, then accessed from the daemon thereafter. There will need to be a system for adding and removing modules after the dictionary is created, or merely deleting and recreating the dictionary (pausing any potential dictionary reads while this process executes). |
|
Some nice progress:
That's all for today. |
|
Interestingly the time it takes to scan the input modules when the code is executed in the daemon is much faster than when it's executed in my test script. |
|
I realized needing to query the daemon from the frontend could be an issue if the daemon is not running. Back to make some fixes. |
|
I was able to successfully refactor the input addition code (actually adding a new input from the Data page), though for this to be complete, I need to now add the new input module dictionary to all of the current inputs and switch over to parsing the actual input directory (not just the dummy directory currently set up), for the input choices (found at various parts of Mycodo, such as the available measurements when adding a new Graph) to properly be compiled. |
|
So far I have working: sorted list of available inputs that can be added, generating input choices for inputs that have been added (for selecting measurements to display on Graphs, for instance), adding inputs (properly modify the SQL database). Things are moving along well. It's nice to finally be at a point where I see the few built-in modules beside the test modules, all being detected and rendered using only information contained within each module's file.
|
|
Within the last commit I found it helpful to begin creating a central translation dictionary that contains the phrases used in tooltips, which will make finding and editing specific phrases much easier. Mycodo/mycodo/config_translations.py Lines 9 to 16 in 7d3c343 |
|
I'm really liking how this is coming along. Thus far, it's working beautifully, and it's not a lot of extra information added to each input module. For instance, this is all that's needed to be added to the Atlas PT-1000 Temperature sensor module to make it completely independent and able to be loaded from the single file: Mycodo/mycodo/inputs/atlas_pt1000.py Lines 9 to 22 in 1c9fb63 |
|
I'm fairly sure I have nearly all refactoring complete. I fixed all pytests and got TravisCI to succeed, and I just added a new configuration page that allows importing and deleting of custom input modules. I still need to add a few more sanity checks for the incoming files, but so far it's working beautifully.
|
|
I made a minimal example of basic input module, at https://github.com/kizniche/Mycodo/blob/single_file_input_modules/mycodo/inputs/examples/minimal_humidity_temperature.py The links referenced at the beginning are to these other example files with a full list of the available options and descriptions: # coding=utf-8
import logging
from mycodo.inputs.base_input import AbstractInput
from mycodo.inputs.sensorutils import convert_units
from mycodo.inputs.sensorutils import calculate_dewpoint
# Input information
# See the inputs directory for examples of working modules.
# The following link provides the full list of options with descriptions:
# https://github.com/kizniche/Mycodo/blob/single_file_input_modules/mycodo/inputs/examples/example_all_options_temperature.py
INPUT_INFORMATION = {
'unique_name_input': 'TEST_00',
'input_manufacturer': 'Company X',
'common_name_input': 'Input 00',
'common_name_measurements': 'Humidity/Temperature',
'unique_name_measurements': ['dewpoint', 'humidity', 'temperature'], # List of strings
'dependencies_pypi': ['random'],
'interfaces': ['I2C'],
'i2c_location': ['0x5c'],
'i2c_address_editable': False,
'options_enabled': ['period', 'convert_unit', 'pre_output'],
'options_disabled': ['interface', 'i2c_location']
}
class InputModule(AbstractInput):
""" Input support class """
def __init__(self, input_dev, testing=False):
super(InputModule, self).__init__()
self.logger = logging.getLogger("mycodo.inputs.{name_lower}".format(
name_lower=INPUT_INFORMATION['unique_name_input'].lower()))
self._dewpoint = None
self._humidity = None
self._temperature = None
if not testing:
self.logger = logging.getLogger(
"mycodo.inputs.{name_lower}_{id}".format(
name_lower=INPUT_INFORMATION['unique_name_input'].lower(),
id=input_dev.id))
self.convert_to_unit = input_dev.convert_to_unit
self.interface = input_dev.interface
# Load dependent modules
import random
self.random = random
def __repr__(self):
""" Representation of object """
return "<{cls}(dewpoint={dewpoint})(humidity={humidity})(temperature={temperature})>".format(
cls=type(self).__name__,
dewpoint="{0:.2f}".format(self._dewpoint),
humidity="{0:.2f}".format(self._humidity),
temperature="{0:.2f}".format(self._humidity))
def __str__(self):
""" Return measurement information """
return "Dewpoint: {dewpoint}, Humidity: {humidity}, Temperature: {temperature}".format(
dewpoint="{0:.2f}".format(self._dewpoint),
humidity="{0:.2f}".format(self._humidity),
temperature="{0:.2f}".format(self._temperature))
def __iter__(self):
""" iterates through readings """
return self
def next(self):
""" Get next reading """
if self.read():
raise StopIteration
return dict(dewpoint=float('{0:.2f}'.format(self._dewpoint)),
humidity=float('{0:.2f}'.format(self._humidity)),
temperature=float('{0:.2f}'.format(self._temperature)))
@property
def dewpoint(self):
""" dewpoint """
if self._dewpoint is None:
self.read()
return self._dewpoint
@property
def humidity(self):
""" temperature """
if self._humidity is None:
self.read()
return self._humidity
@property
def temperature(self):
""" temperature """
if self._temperature is None:
self.read()
return self._temperature
def get_measurement(self):
""" Measures temperature and humidity """
# Resetting these values ensures old measurements aren't mistaken for new measurements
self._dewpoint = None
self._humidity = None
self._temperature = None
dewpoint = None
humidity = None
temperature = None
# Actual input measurement code
try:
humidity = self.random.randint(0, 100)
temperature = self.random.randint(0, 50)
dewpoint = calculate_dewpoint(temperature, humidity)
except Exception as msg:
self.logger.error("Exception: {}".format(msg))
# Unit conversions
# A conversion may be specified for each measurement
# Humidity is returned as %, but may be converted to another unit (e.g. decimal)
humidity = convert_units(
'humidity', '%', self.convert_to_unit,
humidity)
# Temperature is returned as C, but may be converted to another unit (e.g. K, F)
temperature = convert_units(
'temperature', 'C', self.convert_to_unit,
temperature)
# Dewpoint is returned as C, but may be converted to another unit (e.g. K, F)
dewpoint = convert_units(
'dewpoint', 'C', self.convert_to_unit,
dewpoint)
return humidity, temperature, dewpoint
def read(self):
"""
Takes a reading and updates the self._dewpoint, self._temperature, and
self._humidity values
:returns: None on success or 1 on error
"""
try:
# These measurements must be in the same order as the returned tuple from get_measurement()
self._humidity, self._temperature, self._dewpoint = self.get_measurement()
if self._temperature is not None:
return # success - no errors
except Exception as e:
self.logger.exception(
"{cls} raised an exception when taking a reading: "
"{err}".format(cls=type(self).__name__, err=e))
return 1 |
|
I wanted to be able to add options that could be created on the fly from any Input module, so that if I wanted to create a module with options that don't already exist in Mycodo (or add more options to a module), I could use the INPUT_INFORMATION dictionary to define them. This would allow new options to be created without having to modify the Mycodo system files, which truly makes a standalone Input Module. I was able to do this, and as a proof of concept, I added a checkbox option to enable the fan of the ZH03B sensor to be turned off while it wasn't recording a measurement. The new INPUT_INFORMATION looks like this for this module: # Input information
INPUT_INFORMATION = {
'input_name_unique': 'WINSEN_ZH03B',
'input_manufacturer': 'Winsen',
'input_name': 'ZH03B',
'measurements_name': 'Particulates',
'measurements_list': ['particulate_matter_1_0', 'particulate_matter_2_5', 'particulate_matter_10_0'],
'options_enabled': ['uart_location', 'uart_baud_rate', 'custom_options', 'period', 'convert_unit', 'pre_output'],
'options_disabled': ['interface'],
'dependencies_module': [
('pip-pypi', 'binascii', 'binascii')
],
'interfaces': ['UART'],
'uart_location': '/dev/ttyAMA0',
'uart_baud_rate': 9600,
'custom_options': [
{
'id': 'modulate_fan',
'type': 'checkbox',
'default_value': True,
'name': lazy_gettext('Fan Off After Measure'),
'phrase': lazy_gettext('Turn the fan on only during the measurement')
},
{
'id': 'another_option',
'type': 'textbox',
'default_value': 'my_text_value',
'name': lazy_gettext('Another Custom Option'),
'phrase': lazy_gettext('Another custom option description (this is translatable)')
}
]
}And the result on the Data page looks like this:
I'll be improving the code to include the ability to create dropdown menus and potentially other form types, but I just wanted to document this new feature in the appropriate thread and to share my excitement for a powerful feature that was relatively easy to code. |
|
Woohoo! I keep finding new interesting intelligent sensors to integrate with Mycodo and this is pretty cool in my world view. |
|
While improving the code, I came across a neat way to allow constraints for each custom input. Simply set the dictionary value of 'constraints_pass' to the constraints_pass() function, then create whatever checks you desire in that function. Below is a basic example, but it allows for much more user control over the input module options. def constraints_pass_fan_seconds(value):
"""
Check if the user input is acceptable
:param value: float
:return: tuple: (bool, list of strings)
"""
errors = []
all_passed = True
# Ensure value is positive
if value <= 0:
all_passed = False
errors.append("Must be a positive value")
return all_passed, errors
# Input information
INPUT_INFORMATION = {
'input_name_unique': 'WINSEN_ZH03B',
'input_manufacturer': 'Winsen',
'input_name': 'ZH03B',
'measurements_name': 'Particulates',
'measurements_list': ['particulate_matter_1_0', 'particulate_matter_2_5', 'particulate_matter_10_0'],
'options_enabled': ['uart_location', 'uart_baud_rate', 'custom_options', 'period', 'convert_unit', 'pre_output'],
'options_disabled': ['interface'],
'dependencies_module': [
('pip-pypi', 'binascii', 'binascii')
],
'interfaces': ['UART'],
'uart_location': '/dev/ttyAMA0',
'uart_baud_rate': 9600,
'custom_options': [
{
'id': 'fan_modulate',
'type': 'bool',
'default_value': True,
'name': lazy_gettext('Fan Off After Measure'),
'phrase': lazy_gettext('Turn the fan on only during the measurement')
},
{
'id': 'fan_seconds',
'type': 'float',
'default_value': 5.0,
'constraints_pass': constraints_pass_fan_seconds,
'name': lazy_gettext('Fan On Duration'),
'phrase': lazy_gettext('How long to turn the fan on (seconds) before acquiring measurements')
},
]
} |
…hecks (with error response capability) (#525)
Allow a new input to be added to Mycodo with a single file.
This issue is to track the progress of this potential feature. A new branch will be created for this, single_file_input_modules.
Example commit of the minimal amount of internal code editing required to add a new input module (with new measurements/units that don't already exist): 133a2dc
The text was updated successfully, but these errors were encountered: