Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature request - flow meter add in #143

Closed
mattip opened this issue Jan 23, 2016 · 72 comments
Closed

Feature request - flow meter add in #143

mattip opened this issue Jan 23, 2016 · 72 comments

Comments

@mattip
Copy link

mattip commented Jan 23, 2016

Thanks for this wonderful application.

I have a Seed Studio flow meter http://www.seeedstudio.com/depot/G12-Water-Flow-Sensor-p-635.html hooked into my raspberry pi.

I am using wiringpi like the code posted here https://www.raspberrypi.org/forums/viewtopic.php?f=37&t=94656 to set up an interrupt handler and increment a counter. I see how I can write this as a so and read the counter from Python. I would like to get feedback in SIP about overall flow rate and expected vs actual flow rate (to detect leaks). What would be a strategy to get this into SIP? I see the way the clock time is updated, should I hook into that and update another div under it?

Note that I have not calibrated the flow meter, from some preliminary tests it seems the meter peaks out at about 100 pulses per second, so this might not be an accurate way to measure water quantity rather a qualitative go-no go check of the system, an arduino and maybe a better meter may be needed to get accurate measurements, but that is a different issue.
@Dan-in-CA
Copy link
Owner

Welcome,
Thanks for your interest in the SIP project.

There is a user in Chile who is currently refactoring some of the SIP code to make it easier to interface various hardware with the Raspi and SIP. He is also planning on using flow meters in his system.
It would be a good idea if you could post your feature request on the SIP discussion forum:
http://nosack.com/sipforum/index.php

I will contact Matias as I think he may be interested in working with you on this.
There may also be others who would have an interest in this.

@bkoblenz
Copy link
Contributor

If you use pigpio as the gpio interface, then you can set up a callback
function (called wind_cb) below and that callback function has a tally
method so you dont have to worry interrupt handling etc.

Just a couple of lines of code in your loop taking samples. (Assumes there
was an initial call to set wind_cb prior to the loop.

                    flow = wind_cb.tally()
                    wind_cb = pi.callback(gv.pin_map[3],

pigpio.FALLING_EDGE)

Tally gives you the number of "clicks" since the last time wind_cb was
defined.

On Sat, Jan 23, 2016 at 1:11 AM, Matti Picus notifications@github.com
wrote:

Thanks for this wonderful application.

I have a Seed Studio flow meter
http://www.seeedstudio.com/depot/G12-Water-Flow-Sensor-p-635.html hooked
into my raspberry pi.

I am using wiringpi like the code posted here
https://www.raspberrypi.org/forums/viewtopic.php?f=37&t=94656 to set up
an interrupt handler and increment a counter. I see how I can write this as
a so and read the counter from Python. I would like to get feedback in SIP
about overall flow rate and expected vs actual flow rate (to detect leaks).
What would be a strategy to get this into SIP? I see the way the clock time
is updated, should I hook into that and update another div under it?

Note that I have not calibrated the flow meter, from some preliminary
tests it seems the meter peaks out at about 100 pulses per second, so this
might not be an accurate way to measure water quantity rather a qualitative
go-no go check of the system, an arduino and maybe a better meter may be
needed to get accurate measurements, but that is a different issue.

—
Reply to this email directly or view it on GitHub
#143.

@Pelado-Mat
Copy link
Contributor

I plan to add a pressure sensor to do a go-no-go check, as my irrigation water comes from a pump and a cistern I must be sure that the pump is actually moving water and not over-pressure the system.

When we finish the work with Dan, I plan to modify the pressure sensor plugin to log and plot the pressure someway, also to stop all the stations if the pressure drops too much or get to high.

A Flow meter plug in could work in a similar way, it should be easy modification of the pressure adj plugin using bkoblenz code.

@shrakrak
Copy link

Has anyone eventually got around to writing a flow meter plugin.
i am also interested in this feature and i wanted to check if something is available before i start my own solution.

BTW - i think a flow meter should be part of the core application like the rain sensor. The reason is that it can effect many system functions like logging, active display of flow, accounting. it can also be used to detect solenoid malfunction and even detect leakage/clogging per station (comparing actual station flow Vs expected or historical flow data)

any opinions ?

@DaveSprague
Copy link

For what it's worth, three years ago I grabbed a copy of what was called OSPi, at the time, and hacked in support for reading data from an arduino board that interfaces to three (or more) of the Seeed Studio flow sensors. I was in a hurry at the time and it's worked so well and reliability that I haven't taken the time to merge this into the newer versions.

I agree that a flow meter touches several subsystems of the application so it's maybe be best to build support into the core app.

I'm happy to provide more details if that would be useful. Here's an overview:

  1. the Arduino code, which I'm happy to share if it's useful, keeps track of multiple pulse counters for each flow sensors. Each time the RPi code reads a counter, via a USB serial interface, it resets that counter back to zero.

  2. The RPi code reads the first counter for each flow sensor every few seconds to provide a real-time flow rate display. I use the second counter for each flow meter to track the flow during a program run (by reading that meter's counter it at the start of a program to reset it, the reading the total number of pulses after the program completes to determine the total flow). I then use a third counter to track the amount of flow through each flow sensor over a twenty-four hour period to check if any of the irrigation system's valves is not closing properly.

  3. I added the usage info the the log and also created a separate page to view the current flow rate of each channel. At the time, I didn't have enough knowledge or time to integrate that into the primary display pages of the app.

I don't do any automatic checking of usage versus history or automatically checking a bad valve but I agree those would be great features to have. I do email the recently log entries to myself once day to so that when we're away we can check that things seem to be working okay.

The reason I implemented multiple counters per flow meter is that I wasn't sure if the pulse count that was read every few 2-3 seconds for the flow rate display could be accumulated accurately enough over a longer period to get the total flow. Looking at the usage per day though, there's enough unexplained variation in usage for a given program that it' s not clear that this is necessary. I'll try to provide some more quantitive info on the amount of variation in usage from day to day for a particular program.

Another advantage of having multiple counters per flow meter is that it kept the RPi code somewhat simpler since I could devote a separate counter to each function (real-time flow, program usage, and valve malfunction / leak-detection).

I briefly considered implementing the pulse counting on the RPi using pigpio or similar library but at the time decided that using an Arduino to maintain the pulse counts was simpler for me -- particularly since I planned to maintain multiple pulse counters per flow meter.

Dave

@Dan-in-CA
Copy link
Owner

Hi Dave,
This sounds very useful. I would be interested in more details.
You can post info on the SIP forum at:
http://nosack.com/sipforum/index.php
or email me at:
dkimberling59@gmail.com
or post code on GitHub and provide a link.

Thanks.
Dan

@jakkuvaavinash
Copy link

the exact code for the pulse counting was not described in the fallowing so please get a valid code for water flow meter YF-S201 Hall Effect Water Flow Meter / Sensor so that will help to my project please refer me as soon as possible thanking you

NOTE : PYTHON PROGRAMMING FOR RPI

avinashvera@gmail.com
8985797658
please contact me as soon as possible

@Dan-in-CA
Copy link
Owner

I am not sure what code source you are referring to.
Can you please provide more detail?

@Dan-in-CA
Copy link
Owner

Also, the YF-S201 flow meter requires 5V to work.
The GPIO pins on the Raspberry Pi are 3.3V max. You will need to connect the flow meter to something like an Arduino.

@DaveSprague
Copy link

Following along from my previous post in this thread from back in April, here's the Arduino code I use to count pulses from the flow meters. It uses a library called from here: http://playground.arduino.cc/Main/PinChangeInt, although I used a version of this from 2013 so I'm not certain it will work with the current version without modifications.

This code is written for three flow sensors -- one for each valve in my system. I maintain four counters for each flow meter so that I could accumulate counts over different time durations. I did this because I wasn't sure if I read the sensors every few seconds and accumulated those values in the ospi python code would be accurate for looking at total flow over the course of a few hours. It also makes the python code cleaner in cases where I wanted to keep logs of the total flow during a ospi program even or over a 24 hour period for each channel in addition to the flow during a particular scheduled program even.

I'm not sure this approach is necessary given the accuracy of the flow sensors but I haven't done any analysis of this yet.

I'll send the python code along some time soon, once I've had a chance to relearn it again and put a bit of documentation together. It was based on a version of the ospi code from back in 2013.

`
// PinChangeIntExample, version 1.1 Sun Jan 15 06:24:19 CST 2012
// See original example for more details on defines, etc
//#define NO_PORTB_PINCHANGES // to indicate that port b will not be used for pi
n change interrupts

#define NO_PORTC_PINCHANGES // to indicate that port c will not be used for pin
change interrupts
#define NO_PORTD_PINCHANGES // to indicate that port d will not be used for pin
change interrupts

// #define DISABLE_PCINT_MULTI_SERVICE

#include <PinChangeInt.h>

int p1 = 8; // digital pins 8, 9, and 10 are all on Port B on the Uno
int p2 = 9;
int p3 = 10;
int led = 13;

// There are four counters maintained for each of three flow sensors.
// This allows for keeping track of the total flow over different timeframes
// e.g. 5 seconds, 1 minute, full program run, 24 hour period
long interrupt_count[4][3] = {{0L, 0L, 0L},
{0L, 0L, 0L},
{0L, 0L, 0L},
{0L, 0L, 0L}
};
long unsigned startTime[4];
long elapsedTime;
boolean LED_State = HIGH;

int inByte = 0;

void quicfunc1() {
digitalWrite(led, HIGH);
interrupt_count[0][0]++;
interrupt_count[1][0]++;
interrupt_count[2][0]++;
interrupt_count[3][0]++;
};

void quicfunc2() {
digitalWrite(led, HIGH);
interrupt_count[0][1]++;
interrupt_count[1][1]++;
interrupt_count[2][1]++;
interrupt_count[3][1]++;
};

void quicfunc3() {
digitalWrite(led, HIGH);
interrupt_count[0][2]++;
interrupt_count[1][2]++;
interrupt_count[2][2]++;
interrupt_count[3][2]++;
};

long flow_milliliters(long pulse_count, long milliseconds) {
if ((pulse_count * 1000 / milliseconds) > 1000) {
return -1;
}
long milliliters = pulse_count * 1000L * 2L / ( 11L * 60L);
return milliliters;
}

void establishContact() {
while (Serial.available() <= 0) {
Serial.println("0,0,0,"); // send an initial string
delay(300);
}
}

void setup() {
pinMode(led, OUTPUT);
digitalWrite(led, LOW);

pinMode(p1, INPUT);
PCintPort::attachInterrupt(p1, &quicfunc1, FALLING); // add more attachInterr
upt code as required

pinMode(p2, INPUT);
PCintPort::attachInterrupt(p2, &quicfunc2, FALLING);

pinMode(p3, INPUT);
PCintPort::attachInterrupt(p3, &quicfunc3, FALLING);

startTime[0] = millis();
startTime[1] = millis();
startTime[2] = millis();
startTime[3] = millis();
interrupts();
Serial.begin(9600);
// while (!Serial) {
// ; // for Leonardo board
// }
// establishContact();
}

void loop() {
digitalWrite(led, LOW);
long count;
if (Serial.available() > 0) {
// get incoming string and convert to an integer for which counters are requested
digitalWrite(led, HIGH);
char c = Serial.read();
int freq = c - '0';
int req = freq;
int rst = 1;

// bits 0-2 determine which of four counters is requested.
// if bit 3 is low (default) we'll reset each of the four counters when
//   reading them, otherwise (if bit 3 is high) we don't reset the counters
if (freq >= 8) {
  rst = 0;
  req = freq - 8;
}

elapsedTime = millis() - startTime[req];
String out_str = "";
noInterrupts();
// read the flow amount (in milliliters since the counter was last reset)
//  for each of three flow sensors
for (int i = 0; i < 3; i++) {
  count = interrupt_count[req][i];
  if (rst == 1) {
    interrupt_count[req][i] = 0;
  }
  String flow = String(flow_milliliters(count, elapsedTime), DEC);
  out_str += flow;
  out_str += ", ";
}

if (rst == 1) {
  startTime[req] = millis();
}
interrupts();

// the output string are the values of the total flow for each flow
//  sensors since this level of counter was reset, followed by the
//  elapsed time since the flow sensors was reset.
//  The serial input value as well as it's decoded values (the requested
//  counter and the Reset flag) are also returned for debugging purposes.
out_str += String(elapsedTime, DEC);
out_str += ", ";
out_str += String(freq) + ", " + String(req) + ", " + String(rst);
Serial.println(out_str);
digitalWrite(led, LOW);

}
}
`

@Pelado-Mat
Copy link
Contributor

Pelado-Mat commented Nov 24, 2016 •
edited

I still plan to add flow sensors to my installation, but Im busy with other stuff right now.

My two cents:
To get the arduino to connect to SIP, you could use a plugin as I wrote for the pump_control.

I connect the arduino to the pi via i2c and get the sensor readings of a pressure transducer (among other things).
See the plugin and arduino code from: https://github.com/Dan-in-CA/sip_plugins/tree/master/pump_control

If you decide to go for that route:

  • The i2c code on the arduino side was kind of tricky to get right, specially when you have to send floats or ints via i2c, after reading tons of documents I came across https://github.com/MikeOchtman/Pi_Arduino_I2C that point me to the right direction
  • If you power the arduino from any other source.... connect the GND of the ardunio to de GND of the PI. (I fall into that....)

BTW, every time I start thinking about this I feel that SIP do not have the tools to really take advantage of sensors readings. I imagine some nice graph where I could see the different stations running overleaped with my sensors readings (pressure and/or water flow, etc).
One path could be add some infrastructure to SIP to make that kind of graphs, using something like http://www.flotcharts.org/ (way over my current programming skills)
Other way could be using @DanielCasner mtqq plugin to connect SIP to something like Domoticz, or PiDome and get the flow meters data into that with mySensors

@DaveSprague
Copy link

I'm doing a first cut adding Flow Sensor support, based on a simplified version of the implementation I described above and ported to the current code base. The initial version will support flow sensors connected to an Arduino and accessed via USB serial. It will show the real-time flow rate and flow amount while programs (including Run Once Programs) run and I plan to have it add total water usage to the log for each program run.

My plan is to release an early, rough version in the next several days.

I have included a stub that simulates flow sensors so that others can try it out prior to actually hooking up sensors (it just generates some pulse count values rather than accessing an Arduino). I'm also trying to make it general enough that it would be fairly easy to connect flow sensors to directly to the RPi.

My assumption for this first version is that there's a flow sensor for each valve. My own system only has three valves so that's how I've implemented it. For larger systems it may make sense to have one shared flow sensor on the supply line that's shared by multiple valves or some other configuration like that. There should be a fairly simple way to support this type of configuration with some settings options.

One thing I noticed is that the real-time update that's used to show elapsed time when running programs isn't used during Manual operation of the valves so we'd need to somehow add that into the Manual mode in order to be able to see flow rates/amounts when running in Manual mode (which I think would be quite useful).

I agree with one of the comments above that graphing would be useful. My thought is that the webpage associated with the flow_sensor plugin would be a place to show these graphs. It would access log data and ideally be able to show tables and graphs of water usage over different time periods (weekly, monthly, yearly?).

Another feature I'd like to implement is a way to look for stuck or leaking valves by watching for water flow during times when there shouldn't be any.

Let me know your thoughts on what features/configurations would be useful.

@Dan-in-CA
Copy link
Owner

Dan-in-CA commented Apr 9, 2017 •
edited

This sounds really cool!
One question I have is how to handle sensors that are some distance from the pi. I have been working with a USB to RS485 serial adapter at the Pi and an RS485 to RS232 adapter attached to an arduuino micro. The micro has a second hardware serial port which makes this easy. RS485 serial can work up to 4000 feet. Of course a lot of folks like to use wireless and since the latest Pi 3s have built-in wifi it should not be too hard.

Regarding data graphing, I have been testing chart.js. and have been able to make a plugin page with nice graphs. There is an intro to chart.js with some examples here:
https://www.sitepoint.com/introduction-chart-js-2-0-six-examples/

Looking forward to seeing what you have put together.

Dan

@DaveSprague
Copy link

That charting package looks great.

Good question about remote sensors. I really haven't given that any thought so far. In my installation, I have my flow sensors installed right after each valve and since the PI (with the OSPi board, in my case) need to be fairly close to the valves then I assumed the flow sensors would also be close -- although the valves solenoids can probably be driven over a longer distance than the return signals form the flow sensors can travel. So yes, I would think we could/should accommodate both RS485 and WiFi connected Arduinos.

Do you power the Arduino over the RS485 connection or just provide power to it locally?

Just finished writing a simple version of the Arduino code that maintains pulse counters for up to 8 flow sensors. It could be expanded beyond that number fairly simply. I'm using the EnableInterrupts library (https://github.com/GreyGnome/EnableInterrupt) which which can use up to 18 inputs on an Arduino Uno.

If you're trying to accurately monitor the flow from several valves/lines at once, this code may not be fast enough so if that's an important issue, we may need to use the HiSpeed mode of the EnableInterrupt library or write some other custom Arduino code.

My current implementation has the Arduino code just count pulses for each flow sensor and allows all the counters to be reset at once. The conversion to flow rate in Gallons or Liters per second and total flow amounts in Gallons or Liters is done in the flow_sensor plugin. I partitioned it this way since I thought we might want to support flow sensors connected directly to the Pi, using an interrupt routine to count the pulses. In some ways it would be cleaner to have those conversion details encapsulated in the Arduino code but at this point, that's not how I've structured the code.

As you mentioned above, the flow sensors are designed to run at 5v so maybe the directly interface to the Pi isn't something we really need to support?

My testing is using the 1/2" and 3/4" plastic flow sensors similar to this one:
http://wiki.seeedstudio.com/wiki/G1/2_Water_Flow_sensor

Dave

@Dan-in-CA
Copy link
Owner

Dan-in-CA commented Apr 11, 2017 •
edited

At this point I have been connecting to the arduinos over standard 18guage sprinkler wire. I use 2 wires for the serial communication and 2 wires carrying 12V DC to power the arduinos. I have been using Arduino micros. They are small and have 2 hardware serial ports. They are soldered to an Adafruit 1/2 size perma-proto board along with the RS485 to RS232 adapter and the whole thing fits into a 1 gang weather proof junction box which makes a nice inexpensive water tight enclosure.
sensor_box-sm

Of course there are larger weather tight junction boxes that could b used for the Arduino Uno.

I have one of the 1/2 inch plastic flow sensors that I can use for testing.

This plugin will be extremely useful for use with drip irrigation systems as it will enable the user to determine when the drip emitters are getting clogged.

I will look into adding the real-time update for the manual mode.

@DaveSprague
Copy link

DaveSprague commented Apr 12, 2017 via email •
edited

@Dan-in-CA
Copy link
Owner

Dan-in-CA commented Apr 12, 2017 •
edited

If you let me know when the SIP code is ready I can clone your fork from GitHub. That way I can pull updates as things progress and you won't need to do anything special.
You can either send a zip of the arduino code or post it to a gist here in GitHub. Just click Gist at the very top of your repo page.

I should mention that the Arduino micro has been discontinued by Adafruit who helped design it but it is being manufactured in Italy and is available from Newark Element 14.

@DaveSprague
Copy link

DaveSprague commented Apr 12, 2017 •
edited

My branch of SIP with the flow_sensor support is here: https://github.com/DaveSprague/SIP

The Arduino code is in a public Gist here: https://gist.github.com/DaveSprague/88c2380297e0ae53605ac360299f1b6b

Near the top of the flow_sensors.py plugin you can set the type of sensor you're using (only 1/2" and 3/4" plastic Seeed types are supported now) and whether you want the units in Liters or Gallons.

It currently defaults to flow sensors connect to an Arudino that's connected to the Pi via the USB serial interface. You may need to change the /dev/tty??? name in the flow_sensors.py plugin code to connect to a different serial interface.

Also, near the top of the plugin you can change from using Arduino connected flow sensors to using a very basic simulated flow sensors feature so that the program can be tested even if hardware sensors aren't connected yet.

When you run a Program or Run-Once program you'll see the flow rate and total flow amounts for each station updated every 3 seconds on the Home page. It also adds the water usage amounts for each Program or Run-Once program to the log file.

Doesn't currently show any flow information for Manual Mode yet.

I've left in a bunch of print statements I used for debugging for now so you can use them to identify any issues.

Let me know how it goes and by all means give me feedback on any code issues or functionality issues.

This is my first open source code contribution!

Dave

@Dan-in-CA
Copy link
Owner

OK. I have cloned your fork, copied the Arduino code and downloaded EnableInterrupt to my arduino library. The Arduino code compiles and I loaded it onto an Arduino Micro that I have on a breadboard.

I needed to install pyserial on the Raspi and the plugin is loading properly. I connected the arduino to the Pi with a USB cable and am getting:

Writing to Arduino
serial input from Arduino is: 0,0,0,0,0,0,0,0

So it looks like they are communicating but I have not connected the flow sensor yet.
I have not taken a close look at the Python code yet but this looks like a great start.

@DaveSprague
Copy link

DaveSprague commented Apr 12, 2017 via email

@Dan-in-CA
Copy link
Owner

I hooked up the flow sensor I have to the Arduino micro but I didn't get any reading (blowing air through the sensor). After switching to an Arduino Uno it is working.

There is probably some difference between the 2 Arduino models.

@Dan-in-CA
Copy link
Owner

Looks like the Micro has a different set of interrupt-able pins.
I am getting a reading on the Micro with the sensor connected to pin 8 using your original code. It may be necessary to have different versions of the Arduino code for different arduinos.

@DaveSprague
Copy link

You might want to try some different pins on the Arduino Micro. The Arduino versions are a bit confusing to me but this page on the EnableInterrupts library talks about what pins can be used on different Arduino versions:

https://github.com/GreyGnome/EnableInterrupt/wiki/Usage#pin--port-bestiary

@DaveSprague
Copy link

Yes, or if the Arduinos are "self-aware" enough we might be able to configure, at startup, certain pins to be used based on what type of Arduino it is.

@Dan-in-CA
Copy link
Owner

Not sure if it is possible to to determine the Arduino model automatically but there are ways to handle it in a single code base. That does not seem to an urgent matter.

@DaveSprague
Copy link

Yes, agreed. We can find a way to support a few popular Arduino models pretty easily I think.

@DaveSprague
Copy link

DaveSprague commented Apr 12, 2017 •
edited

The product description of the Arduino Micro, here: http://www.newark.com/arduino/a000053/atmega32u4-arduino-micro-dev-board/dp/63W3544
and the spec sheet you can download there seems to
say it's similar to a Leonardo so you might want to check it against this list of pins from the EnableInterrupts usage:

Arduino Leonardo

Interrupt Type Pins
External 0-3 and 7
Pin Change 8-11 and SCK, MOSI, MISO, SS (SS on 3rd party boards)

So you should be able have a flow sensor input on any of these pins (External or Pin Change)

@DaveSprague
Copy link

Were you able to get the Micro working okay? I have a Leonardo which appears to be identical except for the form factor so I can do some testing here as well. Here's a link I found on the Leonardo/Micro -- we may have to modify the serial code somewhat.

https://www.arduino.cc/en/Guide/ArduinoLeonardoMicro

@Dan-in-CA
Copy link
Owner

Yes. The micro works using ports 8-11 with the code as is which is fine for testing.

Considering that a user will need to open the Arduino file so they can upload it it would be easy to have (commented out) lines defining the pins for different models and instructions at the top directing the user to un-comment the line for their particular model.

@DaveSprague
Copy link

I just noticed that in home.html, in the updateStatus function, around lines 69-70 I should have put in a test to make sure station.flowrate and station.flowAmt are available before I append them into the display string (to properly handle the case where the flow-sensor plugin isn't being used). Do you want me to make that change?

@Dan-in-CA
Copy link
Owner

I de-installed and reinstalled the plugin to get the updates.
It seems to be working as expected now. The sensor had been connected to a prototyping Arduino Micro on a breadboard with a bunch of other stuff connected to it. That may have contributed to the problem.
Now I have the sensor connected to an Arduino Uno without anything else connected.

@Dan-in-CA
Copy link
Owner

Hmm...
After rebooting the Pi it is back to running in simulation mode with the plugin set to Arduino-serial.
The flow_sensors.json file has Arduino-Serial as the interface so the setting is saved properly.

@DaveSprague
Copy link

Hmm...let me try that and see if I can replicate the problem

@DaveSprague
Copy link

Actually thinking about for a second, I believe the problem is that I only access the json settings file for the addon when I go to the settings page. I need to do that when the plugin is initialized, otherwise, it's switching to the default values with is Simulated. Should be a fairly quick fix

@Dan-in-CA
Copy link
Owner

I think that's it. I went to the settings page and resaved then it started working.

@DaveSprague
Copy link

Yes, I think so too. I fixed the startup code in flow_sensors.py and also noticed a case where the quantity and rates units (e.g. gallons and gallons per hour) could get out of sync (e.g. Liters and Gallons per hour) so added a bit of code that I believe fixes that too.

I've uploaded the changes to sip_plugins and tested it some and it seems to be working properly now. The very first time I reloaded the plugin and tried to run a Run Once program, I got this error:

Some Stations has been scheduled: [[0, 0, 0, 0], [1493139588, 1493139600, 12, 98], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
192.168.1.118:55287 - - [25/Apr/2017 16:59:49] "HTTP/1.1 GET /cr" - 303 See Other
192.168.1.118:55287 - - [25/Apr/2017 16:59:49] "HTTP/1.1 GET /" - 200 OK
192.168.1.118:55287 - - [25/Apr/2017 16:59:49] "HTTP/1.1 GET /api/log" - 200 OK
Traceback (most recent call last):
  File "/home/pi/SIPdev/web/application.py", line 237, in process
    return p(lambda: process(processors))
  File "/home/pi/SIPdev/web/session.py", line 85, in _processor
    self._load()
  File "/home/pi/SIPdev/web/session.py", line 106, in _load
    d = self.store[self.session_id]
  File "/home/pi/SIPdev/web/session.py", line 250, in __getitem__
    return self.decode(pickled)
  File "/home/pi/SIPdev/web/session.py", line 210, in decode
    return pickle.loads(pickled)
EOFError

But I haven't been able to replicate it so keep an eye out for it.

@DaveSprague
Copy link

Just noticed that this "pickle EOFError" error is similar or the same as the one noted here: #37

@Dan-in-CA
Copy link
Owner

I see that error on rare occasions. I have never been able to find the cause or a solution.

@DaveSprague
Copy link

Some thoughts on what I'd work on next. Let me know which would be best to prioritize.

  1. Take a pass at cleaning up / refactoring the flow_sensors.py code so it's more readable.
  2. Start adding support for data gathering that will eventually be used for charting.
  3. Adding support for custom flow_sensor parameters to permit using a wider range of sensors and supporting calibration.
  4. Adding some documentation linked to the Help button that would describe how to set up flow sensors and use the plugin.

Some initial thoughts on the data gathering (#2 above). I think we should add a Signal for End of Program (which I don't think exists currently?) and also for End of Day. This would allow us to capture the water usage on each station and accumulate it to get Daily, Weekly, and Monthly totals. We'd then maintain a separate log via a json file that stores the current day's usage, and the last 7-14 days, last 10-15 weeks, and last 12-24 months of water usage. As part of this we can also track any water usage that occurs when a program is not running on a station to check for a valve that's stuck on.

@Dan-in-CA
Copy link
Owner

Hi Dave,

As far as priorities, I think #3 would be good to work on. The calibration and parameters for different sensors are closely related. Doing a calibration should provide the values needed (volume / ticks) to use the sensor without even needing to know the published specs.

Of course documentation in important but if there might be changes to how things work it might be a good idea to start an outline and develop toe details when things are more complete.

On #2, Agreed that signals that mark time intervals like days, weeks, etc should be added to the core code. In fact another user is working on a plugin that will send notifications to e.g. a smart phone when selected events occur. See the SIP forum:
http://nosack.com/sipforum/showthread.php?tid=72&pid=275#pid275

He has used a signal that marks the end of a station's run. It might be good to include some signals in the flow_sensors plugin that could be used to notify a user of unusual flow rates. There is also the telegramBot Plugin that could notify a user of abnormal operation.

#1 cleaning up the code should make further development easier. I guess how you prioritize things depends on your particular way of doing things.

I have been trying to get communication working over RS485 so I can connect to sensors that are not physically close to my Raspi. I have 7 valves that are in 4 locations around the house. The closest ones to the SIP controller are about 30 feet away (beyond the 16' max for USB). I think this situation is probably fairly typical.
So far I am able to write from an Arduino micro to the Pi but can't seem to get it working in the other direction. Probably just not doing something right. This should not involve much change to the plugin code. The RS485 adapter plugs into a USB port on the Pi and converts RS232 to RS485. On the Arduino side there is an adapter the translates from RS485 to RS232 so it is just basic serial communication. There would also need to be some sort of addressing method if there are 2 or more arduinos in the system.

@DaveSprague
Copy link

Yes that sounds like a good priority for me to address items. I'll start work on items #3 and #4 for now.

As for the end of day, week, month type signals one thing we'll have to think about is what order those should be generated in the case where the end of day and end of week and possibly end of month occur at the same time. My first thought is that the end of day should be signaled first and then the end of week happens maybe a few seconds later, and then when necessary, the end of month would be send a few seconds after that??

The other option would be to just send an end of day signal and let the code/plugin that uses the signal go through and check if it's also the end of week and end of month if it needs to. That way it can handle those situations however it chooses. An example would be where at the end of each day I compute the daily water usage and then for the end of week I use that day's total plus the previous six day totals to compute the weekly water usage.

On the RS485 serial interface, you should check that you're flushing the output buffer on the Pi side after you write something? Here's the code I'm using (edited slightly for clarity) in the flow_sensors plugin to write to the arduino and then receive data back. I'm not completely happy with having those time.sleep calls in there but they are working for now.

ser = serial.Serial('/dev/ttyACM0', 9600, timeout=1)
time.sleep(0.1)
ser.write("RS\n")
ser.flush()
time.sleep(0.1)
line = ser.readline()

@mattip
Copy link
Author

mattip commented Jun 3, 2017

Hi. I would like to try this out, it would be great just to get feedback that water is flowing in the system when a program is running, no need for calibration, just a go/nogo.

What should I do to try out the code?

@DaveSprague
Copy link

Hi Matti,

If you'd like, you can use my modified clone of SIP here: https://github.com/DaveSprague/SIPdev

It has some changes to the base SIP code that allows the real-time gallons or liters per hour to display, on the home page, while a valve is running and it modifies the log code to permit plugins to add columns to the log. It also points to my own version of the plugins repository so that you can add the flow sensors plugin that handles the interface to the flow sensors.

The only configuration that's currently supported is one or more flow sensors that interface to an Arudino and then the Raspberry Pi that's running SIP reads the flow data from the Arduino via a USB serial interface. The Arduino code is located here: https://gist.github.com/DaveSprague/88c2380297e0ae53605ac360299f1b6b

The Arduino I use is an Uno so if you use a different type, there might be some debugging to get the connection working and you might need to attach the signals from the flow sensors to different input pins but I can work with you on that.

Since you only want to see whether water is running or not, you can just keep the flow sensor plugin default for the type of valve.

Do you expect to use one flow sensor per valve or a single flow sensors on the main supply? I haven't yet added the code to the flow sensor plugin options to support the later yet but I can do that in the next couple of days if that's how you expect to use it.

Thanks,

Dave Sprague

@mattip
Copy link
Author

mattip commented Jun 4, 2017 via email •
edited

@Dan-in-CA Dan-in-CA mentioned this issue Jun 8, 2017
Closed
@mattip
Copy link
Author

mattip commented Aug 13, 2017

@DaveSprague I have finally gotten around to testing this. Thanks for all the work, it certainly made it easy to get started. Here are some thoughts:

  • I did use the GPIO module + multiprocess.Process to set up a callback in another python process to increment a counter
  • I only have one flowmeter and am returning [value]*8 from read_flow_counters. Won't that give strange values for the watering amount on each of the 4 stations?

I will let it run for a few days to see if the reporting is useful. Already it has achieved my goal which was to verify that a running program actually opened the valve, my whole setup is a bit unreliable.

@mattip
Copy link
Author

mattip commented Aug 16, 2017

Wow, it actually works! Here is a log for the past few days, with a cycle of each of my 4 programs. The numbers are indicative, but need calibration. Note that since I only have one flow meter, the numbers simply increment across all channels

Date Start Station Duration Program Usage
2017-08-16 06:50:00 1: Flowerpots & vegies (pin 7) 29:00 4 2225.72
2017-08-16 06:00:00 2: Center (pin 11) 45:00 1 2203.62
2017-08-16 04:45:00 3? trees? (pin 22) 70:00 3 1996.42
2017-08-15 07:20:00 4Top: Terrace, sidepots (pin12) 60:00 2 1254.33
2017-08-14 19:07:48 4Top: Terrace, sidepots (pin12) 40:00 Run_once 642.81
2017-08-14 18:28:35 1: Flowerpots & vegies (pin 7) 10:00 Run_once 237.91
2017-08-14 18:15:27 1: Flowerpots & vegies (pin 7) 10:00 Run_once 229.26
2017-08-14 06:50:00 1: Flowerpots & vegies (pin 7) 29:00 4 217.08
2017-08-14 06:00:00 2: Center (pin 11) 45:00 1 205.20
2017-08-13 18:08:04 2: Center (pin 11) 45:00 1 0.00
2017-08-13 16:51:20 1: Flowerpots & vegies (pin 7) 29:00 4 0.00
2017-08-13 16:18:19 2: Center (pin 11) 20:00 Run_once 0.00

I am trying to work my changes (using GPIO on the RPI) to @DaveSprague 's code into a pull request

@mattip
Copy link
Author

mattip commented Aug 16, 2017

here are the very rough changes to support GPIO flowmeter reading with no arduino. DaveSprague/sip_plugins#1

@Dan-in-CA
Copy link
Owner

Cleaning up old issues.

@mattip
Copy link
Author

mattip commented Dec 30, 2019

Should I resubmit the water meter plugin against the new SIP plugins repo? Unless @DaveSprague wants to do it.

The plugin has been working well for me. It supplies needed feedback about the proper functioning of the system.

@Dan-in-CA
Copy link
Owner

Dan-in-CA commented Dec 30, 2019 •
edited

The latest version of SIP requires some changes to the plugins in order to function under Python3.
It would be best for @DaveSprague to do the updates to his plugin.

Otherwise you could stay with the older version.

@DaveSprague
Copy link

DaveSprague commented Dec 31, 2019 via email

@DaveSprague
Copy link

DaveSprague commented Dec 31, 2019 via email

@Dan-in-CA
Copy link
Owner

Most of the changes were to enable the program to run under Python 2 and 3.
Another big change was a new irrigation program file format. That may not affect your plugin.

@DaveSprague
Copy link

DaveSprague commented Jan 1, 2020 via email

@mattip
Copy link
Author

mattip commented Mar 22, 2020

ping. Anything I can do to get this merged to SIP4 ?

@DaveSprague
Copy link

Thanks @mattip. As I recall, the main thing that's missing is a way, on the plugin settings page, to calibrate a flow sensor -- to be able to directly enter a gallons per minute or hour number that someone has calculated. That should be fairly easy to do but I am at a different location for the next several months so I don't have direct access to my irrigation system to do the necessary testing.

The other more complicated issue, that I believe is still open, is to come up with a way for plugin's like this one to display live info on the home screen while a program is running and a way for a plugin to add data to the logs. I haven't kept up enough with the project lately to know if a way to do this now or not.

Dave

@DaveSprague
Copy link

One other item that's important to add soon, I believe, is a way to flexibly describe which flow sensor are connected to which valves to permit applications where there's just a single flow sensor that monitors the water going to all the valves or cases where a single flow sensor might be monitoring the supply to a subset of the valves.

@mattip mattip mentioned this issue Apr 19, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@mattip @DaveSprague @shrakrak @Dan-in-CA @bkoblenz @Pelado-Mat @jakkuvaavinash