Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remote Logging #277

Closed
tgangso opened this issue May 7, 2018 · 60 comments
Closed

Remote Logging #277

tgangso opened this issue May 7, 2018 · 60 comments

Comments

@tgangso
Copy link

tgangso commented May 7, 2018

Hi @thorrak,

Does fermentrack support remote logging to external HTTP endpoint? (Similar to BrewPiLess)

That would be golden as i want to integrate this into my Brewfather app.
@thorrak
Copy link
Owner

thorrak commented May 7, 2018

I don’t think so. How does it work with BrewPiLess? Every time it receives/generates a reading it sends it to an external website/endpoint?

@tgangso
Copy link
Author

tgangso commented May 7, 2018 •
edited

Basically you set a time interval (ie every 15 minutes) and it sends a HTTP (post) to an endpoint you define. So it will send the last know values i guess, then at that interval.

You can define the format on the data it sends, but having a default JSON structure is the best.

See attached image how the setup works in BPL.
brewpiless_setup_brewfather_

@thorrak
Copy link
Owner

thorrak commented May 7, 2018

You can pull this data from Fermentrack right now through the existing (admittedly undocumented) API endpoints used for the “dashboard” views. If pulling the data works for you, let me know, and I can get you some brief documentation on the endpoint.

If you need the data pushed, that’s not something that currently exists. I expect that writing the client to do the push would actually be pretty easy (pull from the database, encode using JSON, send to the endpoint - all using Huey to handle the timing) but there would be some work in writing the screens to allow configuring the endpoint, managing the Huey process, etc.

This (push support) isn’t something I envision building in the near future absent either additional requests for the feature or coming up with additional features that could be combined into a single configuration page - but if you wanted to build it and submit a pull request I’d be open to it.

@tgangso
Copy link
Author

tgangso commented May 8, 2018 •
edited

Thank you for the quick feedback. I will ask how many of my users are looking for this features, and then we can see how many requests we get.

@BrewChef
Copy link

BrewChef commented May 8, 2018

This feature would be realy nice.

@lggrizendi
Copy link

I don't know if it help but a look to my devices in two ways:

  1. I create a bash script that send me e email with may external ip every time it changes so i can access at fermnetrack at any time.
    2)The other i create a bash script that cut the last 3 log temp and send me email whit the information at the time that i want. Ex mine send me a email whit the temperature at 8h,12h and 20h.
    If anyone want i can send instructions im just have to translate to english .
    Sorry for my bad english it not may native language .

@lalo-uy
Copy link

lalo-uy commented May 8, 2018 via email

@lggrizendi
Copy link

I know its very easy use, i use with windows for some time. But i don't want to change my host-name every 30days(its a very bad excuse ) but i like with the scrip and its not that difficulty to put to work. I never use on raspberry but here is the tutorial.
https://www.noip.com/support/knowledgebase/install-ip-duc-onto-raspberry-pi/

@tgangso
Copy link
Author

tgangso commented May 8, 2018

For now I am only interested in getting data pushed by Fermentrack, since pulling data from a device on a local network by an external server is far to cumbersome.

@thorrak
Copy link
Owner

thorrak commented Jun 11, 2018

@tgangso - Is there a documented format for what Brewfather expects?

@tgangso
Copy link
Author

tgangso commented Jun 11, 2018

@thorrak From Brewpiless i have defined the following JSON structure, but i can adapt if you don't have the same:

{
id: 'UserDefinedIdOfFermenter01',
beerTemp: 19.7, // beer temp in celcius
beerSet: 19.9, // set beer temp in celcius
fridgeTemp: 19.4, // fridge temp in celcius
fridgeSet: 21, // set fridge temp in celcius
roomTemp: 20.7, // room temp in celcius
gravity: 1.031, // gravity as specific gravity
tiltValue: 24.64, // ispindel/tilt angle value
auxTemp: 20.1, // ispindel/tilt temp value in celcius
extVolt: 3.8, // voltage of ispindel
timestamp: 1528719303 // timestamp in milliseconds utc
}

@danielatkins
Copy link

I would also find remote logging useful. I've been using the feature in BPL to push data up into Thingspeak, essentially just to enable remote temp checking and alert options based on temp thresholds.

@tgangso
Copy link
Author

tgangso commented Jan 10, 2019 •
edited

Any news regarding this @thorrak ? I have users still requesting this :)

@Option113
Copy link

It would be really nice to have the possibility to push/integrate the data from Fermtrack to Brewfather.
I will be following this for updates:)

@thorrak
Copy link
Owner

thorrak commented Feb 11, 2019

I spent quite a bit of time on this feature this weekend, and it's nearing completion. It works, but isn't easily accessible from within the Fermentrack UI just yet. Once the forms get added (hopefully in the next week) this should be good to go.

Soon!

@Option113
Copy link

Awesome @thorrak

@tgangso
Copy link
Author

tgangso commented Feb 11, 2019

Excellent news @thorrak, do you have a sample of the final message format?

Also, does it always log in Celcius and gravity in SG (1.XXX), or is the unit specified in the format?

Thanks!

@thorrak
Copy link
Owner

thorrak commented Feb 11, 2019

Excellent news @thorrak, do you have a sample of the final message format?

Also, does it always log in Celcius and gravity in SG (1.XXX), or is the unit specified in the format?

Thanks!

This isn't yet final, but this is what it looks like at the moment (this is the pretty-printed version of the JSON, I'll post the raw JSON as well later)

{'api_key': 'abcde',
 'brewpi_devices': [{'control_mode': 'f',
                     'fridge_temp': 38.7,
                     'internal_id': 1,
                     'name': 'Legacy 2',
                     'temp_format': 'F'},
                    {'beer_temp': 30.62,
                     'control_mode': 'f',
                     'fridge_temp': 35.36,
                     'internal_id': 2,
                     'name': 'Kegerator',
                     'temp_format': 'F'}],
 'gravity_sensors': [{'internal_id': 1, 'name': 'Pinky', 'sensor_type': 'tilt'},
                     {'internal_id': 3,
                      'name': 'Spindly',
                      'sensor_type': 'ispindel'},
                     {'internal_id': 4,
                      'name': 'Purple Bridge',
                      'sensor_type': 'tilt'}]}

@tgangso
Copy link
Author

tgangso commented Feb 11, 2019

Thanks, i can not see the gravity number in this format?

What are the possible temp_format values?

@thorrak
Copy link
Owner

thorrak commented Feb 11, 2019

That’s the dump of what my Fermentrack install sent ~1 minute before I posted the comment rather than a true exemplar of all the fields. I think my Tilts need new batteries. If a gravity reading was available it would have appeared as ‘gravity’: 1.032. For gravity sensors that also report temperature (tilts and iSpindels) a temperature may also be provided.

Right now, all (selected) gravity sensors & BrewPi devices are sent with every push. I haven’t yet decided how I want to handle caching of values, so right this second it’s a bit of a mess.

The formats at the moment are C and F, and are specified whenever a temperature is provided. If someone else wants to add support for Rankine, they’re welcome to. ;)

@thorrak
Copy link
Owner

thorrak commented Feb 11, 2019

One other thing of note - at the moment, the only device identifier is the “API key”. The idea is that when you set up a push target in Fermentrack, you provide two pieces of information - an endpoint (in most cases a URL) and an API key. The API key can be literally anything, but is intended to be something known to the endpoint as an identifier for a Fermentrack installation.

I wouldn’t be averse to adding an optional “username” field or something of the sort in addition to API key if anyone thought that might be useful - in my head the API key should be enough of an identifier, but I can see other scenarios where an additional data point might be helpful.

@tgangso
Copy link
Author

tgangso commented Feb 12, 2019

API-key only should work fine i think

thorrak added a commit that referenced this issue Feb 17, 2019
@thorrak
External Push Support
94bad39 
* Add remote logging (external push) support (#277)
* Remove the constant polling for LCD refreshes (#278)
* Add "NewCCModelForm" for "modern" controllers (#282)
* Support debugging connection for ports other than 23 (Fixes #279)
* Add support for temperature calibration offsets (Closes #281)
* Update documentation
* Catch ObjectNotFound errors generated by the iSpindle calibration API
* Fix link to favicon (Fixes #313)
* Remove error not available in Python 3.4 (Fixes #321)
* Catch error when API called for nonexistant gravity sensor (Fixes #330)
* Map "Log1Temp" to "RoomTemp" value
@thorrak
Copy link
Owner

thorrak commented Feb 17, 2019

External push support is now live in dev. I'm expecting that there are going to be questions/concerns/requests/complaints - if you have any (or all!) of those, feel free to post them here. :)

@thorrak
Copy link
Owner

thorrak commented Feb 17, 2019

For anyone interested in developing for this, I have created a "sample" app which accepts Fermentrack pushes & logs them. The sample app is available at https://github.com/thorrak/fermentrack_push_target

If you just want to see the app in action, you can spy on my personal Fermentrack push targets by looking at http://pushtest.fermentrack.com/

@danielatkins
Copy link

Hey @thorrak, where would I find logging for this feature? Or any other thoughts as to where we might start looking? Set up a push target to help @tgangso integrate with Brewfather but we aren't seeing anything come through at his end. Have rebuilt my fermentrack twice and did notice an issue with redis not coming back up (which has now been fixed).

@thorrak
Copy link
Owner

thorrak commented Mar 1, 2019

It leverages Huey -- the logs are huey-stderr.log in the log directory of Fermentrack.

If you see any errors/bugs, post here and I'll try to squish them post-haste.

@danielatkins
Copy link

All i'm seeing in there is a lot of these, might be something I am doing, I am running a non-standard build. Let me know what you think

Traceback (most recent call last):
  File "/home/fermentrack/venv/lib/python3.5/site-packages/huey/consumer.py", line 225, in process_task
    task_value = self.huey.execute(task)
  File "/home/fermentrack/venv/lib/python3.5/site-packages/huey/api.py", line 407, in execute
    result = task.execute()
  File "/home/fermentrack/venv/lib/python3.5/site-packages/huey/api.py", line 937, in execute
    return func(*args, **kwargs)
  File "/home/fermentrack/venv/lib/python3.5/site-packages/huey/contrib/djhuey/__init__.py", line 125, in inner
    return fn(*args, **kwargs)
  File "/home/fermentrack/fermentrack/external_push/tasks.py", line 19, in generic_push_target_push
    push_target.send_data()
  File "/home/fermentrack/fermentrack/external_push/models.py", line 229, in send_data
    json_data = self.data_to_push()
  File "/home/fermentrack/fermentrack/external_push/models.py", line 196, in data_to_push
    for sensor in grav_sensors_to_send:
TypeError: 'NoneType' object is not iterable

thorrak added a commit that referenced this issue Mar 4, 2019
@thorrak
Add fix when certain object types aren't available (#277)
bf44f6d
@thorrak
Copy link
Owner

thorrak commented Mar 4, 2019

I just pushed out a fix to dev that should resolve that.

@tgangso
Copy link
Author

tgangso commented Apr 5, 2019

I have had multiple users move from Fermentrack to BPL because they are not able to get any data in the body from this feature

@thorrak
Copy link
Owner

thorrak commented Apr 5, 2019

I’m not surprised - BPL is a damn food piece of software, and is a great solution for a number of people.

My focus has been on TiltBridge for the past two months, but now that it’s complete I’m shifting back to push support to try to finalize Brewers Friend support and get this bug fixed.

Should be ~1-2 weeks out if I was to make a guess based solely on the number of hours I was to have to work on it.

@tgangso
Copy link
Author

tgangso commented Apr 5, 2019

I’m not surprised - BPL is a damn food piece of software, and is a great solution for a number of people.

My focus has been on TiltBridge for the past two months, but now that it’s complete I’m shifting back to push support to try to finalize Brewers Friend support and get this bug fixed.

Should be ~1-2 weeks out if I was to make a guess based solely on the number of hours I was to have to work on it.

Would you mind enlighten me about the push support? I guess it will be the same thing i am after for Brewfather. Maybe add a option for Brewfather aswell?

Thanks

@thorrak
Copy link
Owner

thorrak commented Apr 5, 2019

It probably is, depending on what data you want. The Brewers Friend support is gravity only I believe - It uses their stream API.

@tgangso
Copy link
Author

tgangso commented Apr 5, 2019 •
edited

It probably is, depending on what data you want. The Brewers Friend support is gravity only I believe - It uses their stream API.

Brewfather also has a custom stream, with similar field names and function. If you can tie into that aswell, that would be awesome. https://docs.brewfather.app/integrations/custom-stream there is the spec.

Loggin url would be http://log.brewfather.net/stream?id=xxxx

@thorrak
Copy link
Owner

thorrak commented Apr 5, 2019

Is support for stream/tilt/etc limited to premium accounts?

@tgangso
Copy link
Author

tgangso commented Apr 5, 2019

Is support for stream/tilt/etc limited to premium accounts?

Yes, if you register an account you have premium for 30 days. Send me a private message and i can set your account up with premium.

@potseeslc
Copy link

Hey thorrak,

I have Brewfather and this app is great. I just updated Fermentrack to the Master and it says that it has the ability to export. I was wondering how I would do this for Brewfather? I see there is some troubleshooting here but I would love the instructions if possible. Thanks for all you do, and this project is amazing.

The current setup is Fermentrack running Uno and Tilt.

@danielatkins
Copy link

@potseeslc I personally haven't been able to get this working yet, however this could be due to my configuration (running Fermentrack inside Docker) and i haven't had a chance to confirm either way. I'd be curious how you go with it.

There is an 'External Push Target' option under the settings section in Fermentrack where you would configure, and potentially use the Custom Stream option in Brewfather. I know @tgangso was looking at setting up a specific option for it in Brewfather but as we haven't been able to get any data written to an endpoint this hasn't happened yet.

@thorrak
Copy link
Owner

thorrak commented Apr 23, 2019

Yeah - @potseeslc reached out on HBT as well. I think you're right @danielatkins that BrewFather doesn't work with it yet. I'm going to try to get a BrewFather account set up to play with/test, but haven't had a chance just yet. Depending on how (non-beer-related) things go this week I may end up stuck in a car for 7+ hours on Friday. If that happens, bugfixes & revisiting why this is causing issues are at the top of the list!

@danielatkins
Copy link

Am also travelling at the minute, I'll make some time when I'm back to spin up fermentrack on a spare Pi and see if the ext push target works on that. That'll confirm if my Docker image needs something added to it - i suspect it will.

@potseeslc
Copy link

potseeslc commented Apr 23, 2019 via email

@bjornsol
Copy link

Any update on this feature? Would be a really nice addition to both Fermentrack and Brewfather

@TripitakaBC
Copy link

I have Fermentrack running on a Pi (not in Docker) and a Premium Brewfather account so I'm very willing to pitch in and help run some tests or figure it out. Just need some guidance.

@tgangso
Copy link
Author

tgangso commented May 29, 2019

Any news @thorrak ?

@bjornsol bjornsol mentioned this issue Jun 19, 2019
Closed
@auster55
Copy link

auster55 commented Aug 6, 2019 •
edited

@thorrak Having the possibility to customize the JSON POST format for other apps would be ideal. I'm using BrewFather and the JSON format is different. It should be quick and easy to implement.

@potseeslc
Copy link

Hi @thorrak have you stopped working on fermentrack? I haven't seen an update in months. I was hoping to see movement on this brewfather push issue.

@lbussy
Copy link
Collaborator

lbussy commented Aug 18, 2019

I know he's still around and wishing he had a little time. As a fellow maintainer of a software package or two, I can confirm there are more things to do than time to do it. He just started a new job and is pretty busy but I know this is one of the things he really wants to do. I'm looking at it as well but there are other products/services out there which are much friendlier to developers.

@potseeslc
Copy link

potseeslc commented Aug 19, 2019 via email

@wpeterw
Copy link

wpeterw commented Oct 7, 2019

Any updates on this ?

@thorrak
Copy link
Owner

thorrak commented Nov 29, 2019

Brewfather support should now be live on the dev branch of Fermentrack. I've tested it briefly - if you have any questions/issues/requests, please let me know!

Note - For gravity sensors attached to temperature controllers, the current implementation does not include any of the data coming off the temperature controller itself (e.g. beer temp, fridge temp). I plan to resolve this prior to pushing this patch live into master, but wanted to get this out now to let people start playing with it.

@potseeslc
Copy link

potseeslc commented Nov 29, 2019 via email

@TripitakaBC
Copy link

TripitakaBC commented Nov 29, 2019 via email

@danielatkins
Copy link

danielatkins commented Nov 30, 2019 via email

@potseeslc
Copy link

potseeslc commented Dec 8, 2019 via email

@auster55
Copy link

auster55 commented Dec 16, 2019 •
edited

Nice job, I still see only gravity sensor support on Dev branch. Dio you plan to also support data comming from the temp controller (beer temp, fridge temp, etc.)?

@thorrak
Copy link
Owner

thorrak commented Dec 16, 2019

Nice job, I still see only gravity sensor support on Dev branch. Dio you plan to also support data comming from the temp controller (beer temp, fridge temp, etc.)?

Yeah - assuming I understand your request correctly, #401 should add this to the dev branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests