Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

custom PM sensor #13

Open
misko903 opened this issue Dec 15, 2020 · 20 comments
Open

custom PM sensor #13

misko903 opened this issue Dec 15, 2020 · 20 comments

Comments

@misko903
Copy link

misko903 commented Dec 15, 2020 •
edited
Loading

Hello Jonathan,

I have installed your plugin on WeeWX 4.2.0 with extended database - pm10_0 and pm2_5 values are stored in main weewx.sdb database.
I dont have weewx-purpleair sensor, but custom sensor, which fills out the database through FilePile. So I modified the weewx.conf to:

# Options for extension 'aqi'
[AqiService]
    [[standard]]
        data_binding = aqi_binding
        standard = user.aqi.us.NowCast
    [[air_sensor]]
        data_binding = wx_binding
        usUnits = usUnits
        dateTime = dateTime
#        temp = extraTemp1
#        pressure = pressure
        pm2_5 = pm2_5
        pm10_0 = pm10_0

but I got these errors:

Dec 15 23:07:06 WeeWX /weewxd: AqiService: ValueError AQI calculation for pm2_5 on 1608069960 failed: NowCast AQI could not be calculated for the observations. Too many missing. Missing 2, which meets or exceeding the limit of 1
Dec 15 23:07:06 WeeWX /weewxd: AqiService: ValueError AQI calculation for pm10_0 on 1608069960 failed: NowCast AQI could not be calculated for the observations. Too many missing. Missing 2, which meets or exceeding the limit of 1
Dec 15 23:07:06 WeeWX /weewxd: AqiService: not storing record for dateTime 1608069960

what I have done wrong?

Thanks,
Michal, Slovakia
@Distortionist
Copy link

Did you ever figure this out? I just found this trying to get my pm2_5 readings to be calculated to air quality index of canada or US. I am in over my head, I am trying to do what you are doing, just use the existing values stored in weewx.sdb like the pm2_5, temp, pressure and humidity. I have not installed this extension, I am still trying to figure out if it will work for me and how easy it is to get going. The instructions assume I know more than I do haha.

@misko903
Copy link
Author

This issue looks like it is missing two parameters for AQI calculation. So I have uncommented

#        temp = extraTemp1
#        pressure = pressure

But then weewx crashes:

Dec 28 03:50:11 WeeWX weewx[32674] INFO user.interceptor: shutting down server thread
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__: Caught unrecoverable exception:
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  unsupported operand type(s) for -: 'NoneType' and 'float'
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  Traceback (most recent call last):
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 210, in run
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      self.dispatchEvent(weewx.Event(weewx.CHECK_LOOP, packet=packet))
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 241, in dispatchEvent
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      callback(event)
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 630, in check_loop
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      raise BreakLoop
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  weewx.engine.BreakLoop
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  During handling of the above exception, another exception occurred:
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  Traceback (most recent call last):
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 646, in post_loop
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      self._catchup(self.engine.console.genArchiveRecords)
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 690, in _catchup
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      for record in generator(lastgood_ts):
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/drivers/__init__.py", line 30, in genArchiveRecords
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      raise NotImplementedError("Method 'genArchiveRecords' not implemented")
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  NotImplementedError: Method 'genArchiveRecords' not implemented
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  During handling of the above exception, another exception occurred:
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  Traceback (most recent call last):
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewxd", line 154, in main
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      engine.run()
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 217, in run
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      self.dispatchEvent(weewx.Event(weewx.POST_LOOP))
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 241, in dispatchEvent
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      callback(event)
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 648, in post_loop
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      self._software_catchup()
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 710, in _software_catchup
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      origin='software'))
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/engine.py", line 241, in dispatchEvent
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      callback(event)
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/user/aqi/service.py", line 311, in new_archive_record
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      temp_kelvin = weewx.units.CtoK(weewx.units.FtoC(row['outTemp']))
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****    File "/usr/share/weewx/weewx/units.py", line 43, in FtoC
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****      return (x - 32.0) * 5.0 / 9.0
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL __main__:     ****  Exiting.

Q: what is doing this in my weewx.conf AQI Stanza:
usUnits = usUnits
in my weewx i am using METRIC, or METRICWX
for example [FilePile], which is collecting data from my SDS sensor:
unit_system = METRICWX
but in [StdConvert] i am using
target_unit = US
is the usUnits ok in my case? what are the possibilities?

@Distortionist
Copy link

I got mine working for US standards with these settings for my aqi sensor :

[AqiService] [[standard]] data_binding = aqi_binding standard = user.aqi.us.NowCast [[air_sensor]] data_binding = wx_binding usUnits = usUnits dateTime = dateTime temp = outTemp pressure = pressure pm2_5 = pm2_5
My issue is I live in Canada. I can't figure out how to tell it to use the Canadian AQHI standard, and it mentions that it is the only one that needs more than one reading but doesn't mention what readings I need for it.

I think this project needs better documentation for newbs like me.

@misko903
Copy link
Author

I got mine working for US standards with these settings for my aqi sensor :

[AqiService] [[standard]] data_binding = aqi_binding standard = user.aqi.us.NowCast [[air_sensor]] data_binding = wx_binding usUnits = usUnits dateTime = dateTime temp = outTemp pressure = pressure pm2_5 = pm2_5
My issue is I live in Canada. I can't figure out how to tell it to use the Canadian AQHI standard, and it mentions that it is the only one that needs more than one reading but doesn't mention what readings I need for it.

I think this project needs better documentation for newbs like me.

Are you using Purple Air Sensor? Me not, I just want to read everything from weewx database.
Try to add humidity to your settings.

M.

@Distortionist
Copy link

Distortionist commented Dec 31, 2020 via email

@misko903
Copy link
Author

misko903 commented Jan 3, 2021

ok, what did you changed that it work? Mine is still crashing.

@Distortionist
Copy link

absolutely nothing! just the config file I posted works for me. Are you sure your temperature and pressure is stored correctly? I see you are using temp = extraTemp1 are you sure that is correct? My outdoor array has the temperature in it stored as outTemp in my database weewx database. I don't really know what FilePile is and why you have it set to something different than weewx, if you are storing to the same weewx database. I think it is storing metric numbers and weewx thinks they are US numbers. I suggest trying to convert them to match or start over. I am just a newb I wish I can help more, but by the looks of it, one of those values is not what it is expecting. Something like your pressure is 1013 in metric, and it is like WOW that is crazy if it was expecting 29.91.

@Distortionist
Copy link

Distortionist commented Jan 3, 2021 •
edited
Loading

"Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL main: **** temp_kelvin = weewx.units.CtoK(weewx.units.FtoC(row['outTemp']))
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL main: **** File "/usr/share/weewx/weewx/units.py", line 43, in FtoC
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL main: **** return (x - 32.0) * 5.0 / 9.0
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL main: **** TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
Dec 28 03:50:12 WeeWX weewx[32674] CRITICAL main: **** Exiting."

It is definitely complaining about your out temp. I really suck at this but something is wrong with the way your out temp is stored. It is trying to convert F to C, then C to K, but it can't because of something not making sense is the best I can make of what is happening here.

@Distortionist
Copy link

https://stackoverflow.com/questions/23037408/typeerror-unsupported-operand-types-for-float-and-nonetype-python

Reading that, I think outTemp one is empty or something. But from what you say too, the units may be stored in C to begin with and even if the conversion works, the value would be wrong. I think you need to tell your FilePile to store in us since that is what weewx thinks the database is, and make sure it is actually getting stored correctly. The error references outTemp not extraTemp1. I suggest Telling FilePile to store it as outTemp, in F, and match the setting temp = outTemp. See if that works! The code may be wrong and is not looking up the temp = setting somewhere, or your database has it missing or empty, I have no idea whos fault it is but I am pretty sure doing what I suggested will fix, at least this part of it. It works for me!

@misko903
Copy link
Author

misko903 commented Jan 3, 2021

So i changed the temp = outTemp. now it looks like this:

Jan  3 20:35:03 WeeWX /weewxd: AqiService: ValueError AQI calculation for pm2_5 on 1609702440 failed: NowCast AQI could not be calculated for the observations. Too many missing. Missing 2, which meets or exceeding the limit of 1
Jan  3 20:35:03 WeeWX /weewxd: AqiService: ValueError AQI calculation for pm10_0 on 1609702440 failed: NowCast AQI could not be calculated for the observations. Too many missing. Missing 1, which meets or exceeding the limit of 1
Jan  3 20:35:03 WeeWX /weewxd: AqiService: not storing record for dateTime 1609702440

Solution to your Canadian problem maybe here:
https://github.com/jonathankoren/weewx-aqi/blob/master/bin/user/aqi/standards.py

@Distortionist
Copy link

Distortionist commented Jan 3, 2021 via email

@misko903
Copy link
Author

misko903 commented Jan 3, 2021 •
edited
Loading

I have checked all the values, they are filled in - as US units in weewx.sdb

        usUnits = usUnits
        dateTime = dateTime
        temp = outTemp
        pressure = pressure
        pm2_5 = pm2_5
        pm10_0 = pm10_0

The PM values are measured/written every 3rd minute. There are no missing fields, the value is written 3 times unchanged. Can this be an issue?

@Distortionist
Copy link

Distortionist commented Jan 3, 2021 via email

@Distortionist
Copy link

Distortionist commented Jan 3, 2021 via email

@misko903
Copy link
Author

misko903 commented Jan 3, 2021

Added to write values every 60 seconds. Ending up with the same error...
maybe @jonathankoren can help here?

I'm glad that my conf helped you at least!

@Distortionist
Copy link

Distortionist commented Jan 3, 2021 via email

@misko903
Copy link
Author

misko903 commented Jan 3, 2021

It works somehow :)
Dont know why... I am setting up plotting now and coloured values. I dont know which AQI is used in my country - Slovakia

@misko903
Copy link
Author

misko903 commented Jan 4, 2021

OK, it doesn't work again.
My sensor does not have cut of the extreme high values, so this is done by weewx.conf MinMax values - weewx will simply delete them. Then there are missing fields of pm2_5 and pm10_0 in the database (NULL) and weewx-aqi can't calculate the values.
@jonathankoren, is it possible to ignore some predefined number of NULL fields?

@jonathankoren
Copy link
Owner

@misko903 I think it's better if you just delete the records from your DB

@jonathankoren
Copy link
Owner

@misko903 I believe Slovakia will use the EU standards that are in the new release. I found two online. I believe EuropeanAirQualityIndex is the new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jonathankoren @misko903 @Distortionist