Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: The first sensor to be read won't show the correct value #418

Closed
Kattihatt opened this issue Mar 1, 2018 · 60 comments
Closed

Bug: The first sensor to be read won't show the correct value #418

Kattihatt opened this issue Mar 1, 2018 · 60 comments
Labels

Comments

@Kattihatt
Copy link

Hello again!

So I'm running: 5.5.24 and I had this issue:

I'm trying to read values from the MCP3008, but the first value in the data input list, will not read correct values.

When trying to read values from the MCP3008, I use a transistor switch to the soil moisture sensor circuit and when I had this turned on at all times, it worked as expected. When I on the other hand turned this off, and turned it on by choosing pre-output, I got no reading on the first sensor in the list. This of course, made me think it was a hardware error, and I looked at the circuit but found nothing wrong, also it was working before, when it was turned on all the time. So the next thing to do was to try to increas the on timing before grabbing the data, which changed nothing, even if I set a pre-output duration of 30 seconds.

So the next thing I tried, was to simply add another data input from the MCP3008, and place it above the first actual value to be read. Now the first one still shows a nonsensical value, but the second one, which is the value I want, works fine. Now this made me come to the conclusion that it is a software error.

I see nothing of relevance in the logs, no complaints or errors.

I'd say it's not a top priority to try to settle this since the workaround was as simple as creating a new data input, but it might be worth knowing that the problem exists and if someone else has it maybe they have something more to contribute. I hope it's good for something :)

@Kattihatt
Copy link
Author

There was a connection between analog output and digital output, for some reason this made it work in above mentioned way. I can't understand why keeping the power on, or why changing in the above mentioned way would affect it. But I blamed your fine software for no reason and I apologize :)

@Kattihatt
Copy link
Author

Kattihatt commented Mar 1, 2018

The issue persists, even though the hardware flaw is fixed. Even tried switching out the board, which pretty much only consists of a voltage divider. I tried putting channel 1 as the first to read, then channel 2, it's always the same issue. first to be read is showing around 0 V. Whilst the ones coming after works fine. 10 second pre-output is way more time than is really needed.

@Kattihatt Kattihatt reopened this Mar 1, 2018
@kizniche
Copy link
Owner

kizniche commented Mar 2, 2018

Does the test script that displays all the measured values at once produce this issue?

10 second pre-output

What are you using the pre-output for, in relation to this issue?

kizniche added a commit that referenced this issue Mar 2, 2018
@kizniche
Copy link
Owner

kizniche commented Mar 2, 2018

I have a hunch what might be happening. I just pushed a potential fix, but I'm not sure if it will work. Could you test? I haven't made a release, so you can't do a release upgrade. However, you can upgrade to the latest github commit by doing the following:

  1. Edit Mycodo/mycodo/config.py and change line 18: FORCE_UPGRADE_MASTER = False from False to True
  2. Restart the frontend by selecting [Gear Icon] -> Restart Frontend
  3. Go to the upgrade page, [Gear Icon] -> Upgrade

Then you will have the option to upgrade to the repo master.

@kizniche kizniche added the bug label Mar 2, 2018
@Kattihatt
Copy link
Author

Kattihatt commented Mar 10, 2018

Hi! I forgot to update the thread, sorry! I haven't been sure it's not a hardware error, so I redesigned the circuit, and redesigned it again. I'm pretty certain there's not a problem with the hardware and I've control measured. The problem doesn't occur if the the probes are constantly powered, but that is not a viable solution since that means they'll corrode in no-time, so I've been trying to fiddle with different pre-output settings but to no avail.

Here are some printscreens, the sensors are working fine and the script returns correct values. https://imgur.com/a/kx3OY All the sensors in channel 1, 2, 3, 6 and 7 are all using the same kind of probes sitting in the same test pot with the same moisture level and should therefore return the same values. The noname MCP3008 input is a junk input that I tested using, because this actually worked for some reason when I first tried it, but then it went back to the same issue with reading number one being wrong.

The first two parts of the soil moisture sensor readings in the graph, where they go up and down was me measuring things, so that's nothing to worry about. There are no sensors connected to channel 0 and 5, so those readings are correct and channel 4 is an LDR so that reading is correct.

What are you using the pre-output for, in relation to this issue?

I have a transistor switch on the ground side of the soil moisture probe, so that there is no voltage across the probe when it's not taking readings to avoid electrolysis, corrosion and all that.

@Kattihatt
Copy link
Author

Kattihatt commented Mar 10, 2018

I upgraded, restarted backend and frontend, tested, but the first reading is still not showing the correct value. The script grabs the correct value each time, so I wouldn't know what's wrong. In the datasheet: https://cdn-shop.adafruit.com/datasheets/MCP3008.pdf I got told in #raspberrypi to check 6.1 and 6.2 and that it might explain the error. I am trying to understand how but then I know nothing of how the library is working and the script is using the same library as Mycodo, so that is still confusing. Also, this is the same library that gives me correct results through the script, but have issues in Mycodo and it's very unlikely that this wouldn't have been mentioned before if it was the library itself.

@Kattihatt
Copy link
Author

It seems to have no issues with turning on and off in itself: https://i.imgur.com/SL5kM56.jpg Did it repeatedly here and I could not see any deviations here. I looks just as it should.

@kizniche
Copy link
Owner

The pre-output turns off directly before taking a measurement. Do you think this could be causing the issue?

@Kattihatt
Copy link
Author

I guess it could, I've wondered how it all works. What I really would need would be one pre-output, turn on the transisor switch before taking ALL the readings, not turning it on and off as I suspect is happening now. Turn on 5 seconds before, read channel 0 to 7, turn it off. If it turns off right before, then I guess that might explain it, if it turns off the switch before taking a reading, since that is the probe floating and it's pretty much just taking a reading across one resistor, but why wouldn't it affect the other readings, but only the first?

@kizniche
Copy link
Owner

I could add the option to run the pre-output during the measurement. That will at least rule out that as the cause.

@Kattihatt
Copy link
Author

Yes, I think that might be a very useful option to have anyway. This is probably not the only scenario where it might be used. I wish I could add these things myself! I'm glad your so helpful and seem to like working on this project so much at least!

kizniche added a commit that referenced this issue Mar 11, 2018
@kizniche
Copy link
Owner

I just released v5.6.4 that includes the new Pre Output During Measurement option for Inputs. Give it a test when you can and see if it fixes the measurement issue.

@Kattihatt
Copy link
Author

Now I have no idea what's going on. https://i.imgur.com/7iXD2AG.jpg

Two of the sensors are at 90% which I guess is somewhat realistic, three of them are close to 0%. The Adafruit MCP3008 library example script still returns values close to eachother and that seems representative. I can't even begin to speculate why this happened.

@kizniche
Copy link
Owner

Is the scale set up properly for each input?

@kizniche
Copy link
Owner

To me it looks like the voltages are being measured correctly, it's just the scales that aren't working nicely with the measured voltage.

@Kattihatt
Copy link
Author

The voltages vary? Sensor #1 and #3 are around 1,2 V, whilst the other three are around 3,3 V. They're all in the same pot, so I didn't think that they were correct. I'm also just looking at the voltage right now.

They are set up identically: https://i.imgur.com/Eey9W6l.png and should therefore show the same result.

@kizniche
Copy link
Owner

Can you switch a low-reading probe with a high-reading probe and verify if the two channels' measurements adjust appropriately?

@Kattihatt
Copy link
Author

I switched the probes, not the boards themselves, since they're soldered. I waited until it took a new reading, looks completely different. Can't see anything depending on the switching of probes, it seams to be either completely off or a reasonable value that I've found for a bit below 100%.

https://imgur.com/a/050vQ

@kizniche
Copy link
Owner

Which probes did you switch?

@Kattihatt
Copy link
Author

#2 with #3.

@kizniche
Copy link
Owner

kizniche commented Mar 11, 2018

I'm not sure what to make of your readings. Is the measurement difference associated with the hardware?

@Kattihatt
Copy link
Author

Kattihatt commented Mar 12, 2018

Before, I would've said "Maybe, I'll have to check". Now I've checked and redesigned three times with no different result so I consider it ruled out. It's not a hardware problem. I would also say it's confirmed by my multimeter and the Python script returning correct values. I did another test like the one above and it shows no incorrect readings in connection to turning the transistor switch on and off.

https://imgur.com/a/fNN51

The first reading is lower, but I guess this is because it takes a reading every half second and it takes about 2 seconds before the voltage stabilizes based on what I've seen when measuring with the multimeter. But it's not close to what is being shown in Mycodo on some of the readings. This time it's only the first reading that is correct, and the other ones are showing near to 0%. So like I said, I have no clue what's going on but I'm also starting to suspect the pre-output settings. Is there a way to make a pre output setting for a group of data inputs rather than doing it individually? Maybe if it wouldn't turn on and off, but be on for the whole duration of taking readings from 0 - 7 and then turn off when it has the values already? I'm suspecting that it turns the transistor switch on and off, whilst it is taking a reading from one of the other channels, so that it becomes very unrealiable. I guess one way to test this could be to do very different timers on the data inputs. I will try this right now.

EDIT: Also another thing I wondered about readings never showing more than 3.3 V. I run the ADC on 5 V and my 0% readings are 5 V, not 3.3 shown as max here. It doesn't matter if I put the maximum voltage at 5 V, since the readings show maximum 3,3 V then that means I get incorrect % values. Shouldn't this be based on a customizeable V_ref rather than a default of 3,3 V? Don't know if it's causing issues, just thinking out loud.

@kizniche
Copy link
Owner

kizniche commented Mar 12, 2018

I would also say it's confirmed by my multimeter and the Python script returning correct values.

So when switching a probe from a low-reading channel with a probe from a high-reading channel, the low reading channel continues to read low values (and vise-versa with the other channel)?

I see why your percentages are not working correctly. You have "Inverse Scale" enabled. I'm not sure why you enabled it, but it is effectively making your 3.3v measurement register 0% and 0v register 100%.

I run the ADC on 5 V and my 0% readings are 5 V

I thought it only ran with a max of 3.3v. I will fix the code to allow a voltage of whatever you set in the Input settings.

@kizniche
Copy link
Owner

Did you update to the latest code with my instructions, or not?

@kizniche
Copy link
Owner

What you just described is unexpected behavior. We can't have the system running unexpectedly. If you can test my latest code (the fix), we can see if the expected behavior is restored.

@kizniche
Copy link
Owner

kizniche commented Mar 15, 2018

Input measurements can become out of sync, and if that happens, your workaround will no longer work. My fix incorporates a lock for each input during the pre-duration, so only one can utilize a specific pre-output at a time, ensuring the inputs are read sequentially. I just need someone with the hardware that can test the software (the fix).

@Kattihatt
Copy link
Author

Kattihatt commented Mar 15, 2018

I did not. I will do that tomorrow. Already in bed. Just checked the graph, and for most of the time, the values are correct now, but suddenly drops whilst taking a reading, just to go back to normal right after. Don't know how to deal with this. Got to get back to it later :) Thank you for your effort and I should be able to do it early tomorrow.

@Kattihatt
Copy link
Author

So I did:

"FORCE_UPGRADE_MASTER = True
Then save your edit.

Restart the Mycodo frontend
sudo service mycodoflask restart"

It did an update. I removed the temporary "fix" from yesterday and turned the pre-output back to 5 seconds. Now I got the same thing back, first reading is correct, the rest are at 0%.

@kizniche
Copy link
Owner

It is very difficult to test fixes without the hardware. Diagnostics can occur exponentially faster if I have access to the Pi and Mycodo. Could you provide me SSH and Mycodo credentials to test code to come up with a fix?

@Kattihatt
Copy link
Author

I was going to suggest that but forgot about it. I can't find any PM function here, shall I mail the information to you?

@kizniche
Copy link
Owner

Contact me at kylegabriel.com/contact

@Kattihatt
Copy link
Author

I did send. Don't know if you got it, it just stays at "Processing"

@kizniche
Copy link
Owner

My recaptcha key had expired. Try it again. I just tested and it now works.

@Kattihatt
Copy link
Author

Yes, now it worked.

@kizniche
Copy link
Owner

Success!

screenshot-83 250 133 173-2018-03-16-22-07-38

@kizniche
Copy link
Owner

I'm testing a few more things, so don't cut my access just yet ;)

I found the issue was the locking library I was using wasn't thread safe.

@kizniche
Copy link
Owner

You can now see the measurement times are now 5 seconds apart.

@kizniche
Copy link
Owner

Okay. I think I fixed all issues with locking throughout Mycodo. I also slated potential support for multiple ADC measurements per ADC Input in the 6.0 Changes wiki page that I've been tracking major code refactors.

Thanks for letting me test on your system. Everything looks like it's running nicely and I've logged out. If you notice any more issues, let me know.

@kizniche
Copy link
Owner

kizniche commented Mar 17, 2018

Also, I set the sample rates to 60 seconds, but forgot to change them back to 300 before logging out.

@Kattihatt
Copy link
Author

Yeah I just noticed! Good work! The readings now look a lot more wobbly and like they're taking accurate readings, but the first sensor is completely straight and looks different than the others. Don't know why but I'll see if that is something to worry about.

I'll keep trying it out and see if it works. I'll let it sit for a day or two and if it works fine then I'll start working on conditionals :)

@kizniche
Copy link
Owner

I suspect it's due to the power being turned on/off (pre-output). Try disabling the pre-outputs and having it stay on constantly and acquire measurments, to test.

@Kattihatt
Copy link
Author

Kattihatt commented Mar 18, 2018

It's all been running fine up until now when I checked, and saw that that two of the channels show 0%. I check the graph, and at 19:00, they appear to have stopped getting any results. The line just ends abruptly. It looked to be working well up until now.

https://i.imgur.com/B80DbCn.jpg
https://i.imgur.com/4BuHUN9.jpg

@kizniche
Copy link
Owner

Can you check the daemon log for errors?

@Kattihatt
Copy link
Author

@kizniche
Copy link
Owner

Is that the only error or are there more?

@Kattihatt
Copy link
Author

Those are the last 100 lines and the only errors that occurred today at least.

Here are the last 200 lines: http://paste.debian.net/1015393/ and apparently something similar happened the other day as well, don't know why I didn't notice that.

@kizniche
Copy link
Owner

I'm not exactly sure why measurements stopped, but I can fix the locking issue. I've also pushed a potential fix for the MCP3008 not working. Try a manual update and give that code a run.

@Kattihatt
Copy link
Author

The temperature on the RPi spiked at the same time as it locked. Could this be connected? I had this last week too when the RPi crashed. It was really hot to the touch, to the degree I added a heatsink.

@kizniche
Copy link
Owner

Not sure. The CPU is throttled if it gets too hot, so temperature shouldn't be an issue, unless you have your Pi in an environment with a high ambient temperature.

@kizniche
Copy link
Owner

In version 7.0 (yet-to-be-released), the ADC system has been completely redesigned to work better. I'm closing this issue for now, and if you would like to test the 7.0 branch either when it's released or prior to release (this comment details how), please let me know if your issues are resolved. Part of why I redesigned the ADC system was from several user issues, including yours.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants