Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forwarding does not work #143

Closed
bw1faeh0 opened this issue Jan 16, 2023 · 19 comments
Closed

Forwarding does not work #143

bw1faeh0 opened this issue Jan 16, 2023 · 19 comments
Assignees
Labels
bug Something isn't working

Comments

@bw1faeh0
Copy link

Hi,

I'm trying to forward my ecowitt data (ecowitt protocol) to my (custom) server. On this server I'm logging my data, too.

The setup looks like this:

  • Listen on all IPs, Port 4500
  • path: /index.php
  • protocol: Ecowitt
  • forwarding url: http://wetterupdate.flaemig42.de/index.php

I can read all weather data from the objects within IOBroker, so basically the adapter and instance is working.

But I can read from my access logs of my server, that the plugin does not forward any data, since there is no POST request on the index.php from the IP of my IObroker (home).

How can I do further debugging?

@phifogg
Copy link
Owner

phifogg commented Jan 16, 2023

Hi,
you can set the adapter to debug mode. Wait for one incoming listener call and send me the (filtered) log entries. This should at least show if/where it tries to forward to.

@bw1faeh0
Copy link
Author

bw1faeh0 commented Jan 16, 2023

Since I'm new to IOBroker, I'm not sure, if this log is the one you are looking for...

debug.log

But it looks for me, that the adapter is not trying to forward at all. At least I can't see any debug information regarding this.

@phifogg
Copy link
Owner

phifogg commented Jan 16, 2023

Ok, I see the forward URL is set. Let me check the codebase on when this is actually used. Can't find another output with that.

What I do see though is your weather station seems to send the data every 1 second. This is a high frequency as ioB needs almost a full second to process. In case your ioB is really busy (high load) this could be a reason. Might be an idea to lower the frequency for updates a little bit in the weather station.

@bw1faeh0
Copy link
Author

The weatherstation is set to send data every 60 seconds. An this ist what I can see in IOBroker, too.
The values get updated every 60 seconds.

image

@bw1faeh0
Copy link
Author

Which port is used for forwarding? 45000? 80?

@phifogg
Copy link
Owner

phifogg commented Jan 16, 2023

Forwarding uses standard http port (80).

@bw1faeh0
Copy link
Author

I just created a workaround:

I set up my custom server within the weatherstation again (as it was before using sainlogic). Then I created a ssh tunnel from my custom server to the device my IObroker is running on (forwarding http traffic to port 45000 on the device).
My http server on my custom server is now (additionally) forwarding the POST data from the weatherstation through the ssh tunnel to the sainlogic instance.

@phifogg
Copy link
Owner

phifogg commented Jan 18, 2023

Hm.. ok. this seems a viable choice if you can run a custom server.
I'd still be interessted what works or doesn't work in your scenario.

Based on the code, the forwarding will happen anytime I get a request just after processing all the data internally. A log message will be generated IF the receiving servers is giving any kind of response to the request. OR if there is any exception thrown.

As there is no log message in your case, it seems the receiving server just accepted it and didn't respond? Is that an option?

I can add more debug statements to try, or you could - sounds like you are quite good on linux hosts :D - use tcpdump to monitor outgoing traffic from your iob host.

@phifogg
Copy link
Owner

phifogg commented Jan 18, 2023

AH.. if you need a specific port in the forward, you need to specify it as part of the URL. Like https://my.host:45000/upload. This should work for custom ports.

@bw1faeh0
Copy link
Author

Based on the code, the forwarding will happen anytime I get a request just after processing all the data internally. A log message will be generated IF the receiving servers is giving any kind of response to the request. OR if there is any exception thrown.

I took a look into you code, too, and noticed the same: data should be forwarded every time. Maybe got has a problem

As there is no log message in your case, it seems the receiving server just accepted it and didn't respond? Is that an option?

The nginx server logs every access attempt, even it would result in an error (not found, not allowed, etc.). As there is no connection attempt logged, my guess is, that there is no traffic going out from IObroker/sainlogic.

I can add more debug statements to try, or you could - sounds like you are quite good on linux hosts :D - use tcpdump to monitor outgoing traffic from your iob host.

I will check in the next days what traffic is generated using tcpdump.
As I found a workaround, the urgency has decreased

@bw1faeh0
Copy link
Author

AH.. if you need a specific port in the forward, you need to specify it as part of the URL. Like https://my.host:45000/upload. This should work for custom ports.

Nope, my server is using port 80 for incoming weather data.

@PastCoder
Copy link

PastCoder commented Mar 26, 2023

I am also trying to forward information, but don't succeed.
When a forward URL is defined, I get periodic error message:
RequestError: Expected value which is plain object, received value of type number.
Unfortunately I don't see what the result was to investigate in the problem. Even better would to see (in debug log or silly settings) the URL which was used to upload. Not only the base formward URL when starting but with each access. Then I could copy&paste it to see the server (forward target) answer.

Added: If I copy the used forward URL and add "?winddir=230&windspeedmph=12&windgustmph=12&tempf=70&rainin=0&baromin=29.1&dewptf=68.2&humidity=90" I get a successful upload answer. So the URL seems to be okay.

@tomschlde
Copy link

Hi devs, no idea how to fix it?
Or is the adapter depreciated?

@phifogg
Copy link
Owner

phifogg commented May 6, 2023

No, not dead... just slow as so many other things to do :)

Looking into this, issue could be in the code, retry policy is wrongly used -> need update
(Comment to myself to fix it: {limit: 0})

@phifogg phifogg self-assigned this May 6, 2023
@phifogg phifogg added the bug Something isn't working label May 6, 2023
@phifogg
Copy link
Owner

phifogg commented May 7, 2023

Added to release 0.10.1

@phifogg phifogg closed this as completed May 7, 2023
@tomschlde
Copy link

Unfortunately same bahavior with 0.10.3 as described in #154 .
No forwarding to be noted.
Maybe some logging hast tro be added here.

Here some logs from my test:

2023-05-09 15:50:18.103 - debug: sainlogic.0 (16734) Listener body is PASSKEY=XX&stationtype=GW2000A_V2.2.3&runtime=1883565&dateutc=2023-05-09+13:50:18&tempinf=74.12&humidityin=53&baromrelin=29.613&baromabsin=29.613&tempf=61.52&humidity=90&winddir=264&windspeedmph=1.57&windgustmph=3.80&maxdailygust=12.53&solarradiation=106.24&uv=0&rrain_piezo=0.378&erain_piezo=0.098&hrain_piezo=0.098&drain_piezo=0.098&wrain_piezo=0.185&mrain_piezo=0.484&yrain_piezo=3.012&ws90cap_volt=5.2&ws90_ver=126&soilmoisture1=61&soilmoisture2=52&soilmoisture3=49&soilmoisture4=60&soilmoisture5=61&soilmoisture6=50&soilbatt1=1.6&soilbatt2=1.6&soilbatt3=1.6&soilbatt4=1.6&soilbatt5=1.7&soilbatt6=1.6&wh90batt=2.62&freq=868M&model=GW2000A&interval=20 2023-05-09 15:50:18.104 - debug: sainlogic.0 (16734) Listener received EW update: {"PASSKEY":"XX","stationtype":"GW2000A_V2.2.3","runtime":"1883565","dateutc":"2023-05-09 13:50:18","tempinf":"74.12","humidityin":"53","baromrelin":"29.613","baromabsin":"29.613","tempf":"61.52","humidity":"90","winddir":"264","windspeedmph":"1.57","windgustmph":"3.80","maxdailygust":"12.53","solarradiation":"106.24","uv":"0","rrain_piezo":"0.378","erain_piezo":"0.098","hrain_piezo":"0.098","drain_piezo":"0.098","wrain_piezo":"0.185","mrain_piezo":"0.484","yrain_piezo":"3.012","ws90cap_volt":"5.2","ws90_ver":"126","soilmoisture1":"61","soilmoisture2":"52","soilmoisture3":"49","soilmoisture4":"60","soilmoisture5":"61","soilmoisture6":"50","soilbatt1":"1.6","soilbatt2":"1.6","soilbatt3":"1.6","soilbatt4":"1.6","soilbatt5":"1.7","soilbatt6":"1.6","wh90batt":"2.62","freq":"868M","model":"GW2000A","interval":"20"} 2023-05-09 15:50:18.106 - debug: sainlogic.0 (16734) No mapping found for dataelement: PASSKEY 2023-05-09 15:50:18.107 - debug: sainlogic.0 (16734) Extracting value for info.softwaretype(stationtype) 2023-05-09 15:50:18.108 - debug: sainlogic.0 (16734) No mapping found for dataelement: runtime 2023-05-09 15:50:18.109 - debug: sainlogic.0 (16734) No mapping found for dataelement: dateutc 2023-05-09 15:50:18.110 - debug: sainlogic.0 (16734) Extracting value for weather.current.indoortemp(tempinf) 2023-05-09 15:50:18.112 - debug: sainlogic.0 (16734) Extracting value for weather.current.indoorhumidity(humidityin) 2023-05-09 15:50:18.114 - debug: sainlogic.0 (16734) Extracting value for weather.current.pressurerel(baromrelin) 2023-05-09 15:50:18.116 - debug: sainlogic.0 (16734) Extracting value for weather.current.pressureabs(baromabsin) 2023-05-09 15:50:18.118 - debug: sainlogic.0 (16734) Extracting value for weather.current.outdoortemp(tempf) 2023-05-09 15:50:18.121 - debug: sainlogic.0 (16734) Extracting value for weather.current.outdoorhumidity(humidity) 2023-05-09 15:50:18.122 - debug: sainlogic.0 (16734) Extracting value for weather.current.winddir(winddir) 2023-05-09 15:50:18.123 - debug: sainlogic.0 (16734) Extracting value for weather.current.windspeed(windspeedmph) 2023-05-09 15:50:18.126 - debug: sainlogic.0 (16734) Extracting value for weather.current.windgustspeed(windgustmph) 2023-05-09 15:50:18.127 - debug: sainlogic.0 (16734) Extracting value for weather.current.maxdailygust(maxdailygust) 2023-05-09 15:50:18.132 - debug: sainlogic.0 (16734) Extracting value for weather.current.solarradiation(solarradiation) 2023-05-09 15:50:18.136 - debug: sainlogic.0 (16734) Extracting value for weather.current.uvi(uv) 2023-05-09 15:50:18.137 - debug: sainlogic.0 (16734) Extracting value for weather.current.rain_piezo(rrain_piezo) 2023-05-09 15:50:18.138 - debug: sainlogic.0 (16734) Extracting value for weather.current.eventrain_piezo(erain_piezo) 2023-05-09 15:50:18.140 - debug: sainlogic.0 (16734) Extracting value for weather.current.hourlyrain_piezo(hrain_piezo) 2023-05-09 15:50:18.141 - debug: sainlogic.0 (16734) Extracting value for weather.current.dailyrain_piezo(drain_piezo) 2023-05-09 15:50:18.142 - debug: sainlogic.0 (16734) Extracting value for weather.current.weeklyrain_piezo(wrain_piezo) 2023-05-09 15:50:18.143 - debug: sainlogic.0 (16734) Extracting value for weather.current.monthlyrain_piezo(mrain_piezo) 2023-05-09 15:50:18.144 - debug: sainlogic.0 (16734) Extracting value for weather.current.yearlyrain_piezo(yrain_piezo) 2023-05-09 15:50:18.145 - debug: sainlogic.0 (16734) No mapping found for dataelement: ws90cap_volt 2023-05-09 15:50:18.145 - debug: sainlogic.0 (16734) No mapping found for dataelement: ws90_ver 2023-05-09 15:50:18.147 - debug: sainlogic.0 (16734) Extracting value for weather.current.1.soilmoisture(soilmoisture1) 2023-05-09 15:50:18.148 - debug: sainlogic.0 (16734) Extracting value for weather.current.2.soilmoisture(soilmoisture2) 2023-05-09 15:50:18.149 - debug: sainlogic.0 (16734) Extracting value for weather.current.3.soilmoisture(soilmoisture3) 2023-05-09 15:50:18.149 - debug: sainlogic.0 (16734) Extracting value for weather.current.4.soilmoisture(soilmoisture4) 2023-05-09 15:50:18.150 - debug: sainlogic.0 (16734) Extracting value for weather.current.5.soilmoisture(soilmoisture5) 2023-05-09 15:50:18.151 - debug: sainlogic.0 (16734) Extracting value for weather.current.6.soilmoisture(soilmoisture6) 2023-05-09 15:50:18.151 - debug: sainlogic.0 (16734) Extracting value for weather.info.1.soilbatt(soilbatt1) 2023-05-09 15:50:18.152 - debug: sainlogic.0 (16734) Extracting value for weather.info.2.soilbatt(soilbatt2) 2023-05-09 15:50:18.152 - debug: sainlogic.0 (16734) Extracting value for weather.info.3.soilbatt(soilbatt3) 2023-05-09 15:50:18.153 - debug: sainlogic.0 (16734) Extracting value for weather.info.4.soilbatt(soilbatt4) 2023-05-09 15:50:18.153 - debug: sainlogic.0 (16734) Extracting value for weather.info.5.soilbatt(soilbatt5) 2023-05-09 15:50:18.154 - debug: sainlogic.0 (16734) Extracting value for weather.info.6.soilbatt(soilbatt6) 2023-05-09 15:50:18.154 - debug: sainlogic.0 (16734) No mapping found for dataelement: wh90batt 2023-05-09 15:50:18.154 - debug: sainlogic.0 (16734) Extracting value for info.freq(freq) 2023-05-09 15:50:18.155 - debug: sainlogic.0 (16734) Extracting value for info.model(model) 2023-05-09 15:50:18.155 - debug: sainlogic.0 (16734) No mapping found for dataelement: interval 2023-05-09 15:50:18.156 - debug: sainlogic.0 (16734) Extracting value for info.last_listener_update(last_listener_update) 2023-05-09 15:50:18.156 - debug: sainlogic.0 (16734) Setting value from data for info.softwaretype to GW2000A_V2.2.3 2023-05-09 15:50:18.157 - debug: sainlogic.0 (16734) Setting value from data for weather.current.indoortemp to 23.4 2023-05-09 15:50:18.157 - debug: sainlogic.0 (16734) Target for indoortemp unit is set: °C, using conversion: null 2023-05-09 15:50:18.157 - debug: sainlogic.0 (16734) Setting value from data for weather.current.indoorhumidity to 53 2023-05-09 15:50:18.158 - debug: sainlogic.0 (16734) Setting value from data for weather.current.pressurerel to 1002.7 2023-05-09 15:50:18.158 - debug: sainlogic.0 (16734) Target for pressurerel unit is set: hPa, using conversion: null 2023-05-09 15:50:18.158 - debug: sainlogic.0 (16734) Setting value from data for weather.current.pressureabs to 1002.7 2023-05-09 15:50:18.159 - debug: sainlogic.0 (16734) Target for pressureabs unit is set: hPa, using conversion: null 2023-05-09 15:50:18.159 - debug: sainlogic.0 (16734) Setting value from data for weather.current.outdoortemp to 16.4 2023-05-09 15:50:18.159 - debug: sainlogic.0 (16734) Target for outdoortemp unit is set: °C, using conversion: null 2023-05-09 15:50:18.159 - debug: sainlogic.0 (16734) Setting value from data for weather.current.outdoorhumidity to 90 2023-05-09 15:50:18.160 - debug: sainlogic.0 (16734) Setting value from data for weather.current.winddir to 264 2023-05-09 15:50:18.160 - debug: sainlogic.0 (16734) Setting value from data for weather.current.windspeed to 2.5 2023-05-09 15:50:18.160 - debug: sainlogic.0 (16734) Target for windspeed unit is set: km/h, using conversion: null 2023-05-09 15:50:18.161 - debug: sainlogic.0 (16734) Setting value from data for weather.current.windgustspeed to 6.1 2023-05-09 15:50:18.161 - debug: sainlogic.0 (16734) Target for windgustspeed unit is set: km/h, using conversion: null 2023-05-09 15:50:18.161 - debug: sainlogic.0 (16734) Setting value from data for weather.current.maxdailygust to 20.2 2023-05-09 15:50:18.161 - debug: sainlogic.0 (16734) Target for maxdailygust unit is set: km/h, using conversion: null 2023-05-09 15:50:18.162 - debug: sainlogic.0 (16734) Setting value from data for weather.current.solarradiation to 13461 2023-05-09 15:50:18.162 - debug: sainlogic.0 (16734) Target for solarradiation unit is set: W/m2, using conversion: roundTo(x / 126.7, 1) 2023-05-09 15:50:18.162 - debug: sainlogic.0 (16734) Setting value from data for weather.current.uvi to 0 2023-05-09 15:50:18.163 - debug: sainlogic.0 (16734) Setting value from data for weather.current.rain_piezo to 9.6 2023-05-09 15:50:18.163 - debug: sainlogic.0 (16734) Target for rain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.163 - debug: sainlogic.0 (16734) Setting value from data for weather.current.eventrain_piezo to 2.5 2023-05-09 15:50:18.168 - debug: sainlogic.0 (16734) Target for eventrain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.169 - debug: sainlogic.0 (16734) Setting value from data for weather.current.hourlyrain_piezo to 2.5 2023-05-09 15:50:18.169 - debug: sainlogic.0 (16734) Target for hourlyrain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.169 - debug: sainlogic.0 (16734) Setting value from data for weather.current.dailyrain_piezo to 2.5 2023-05-09 15:50:18.170 - debug: sainlogic.0 (16734) Target for dailyrain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.170 - debug: sainlogic.0 (16734) Setting value from data for weather.current.weeklyrain_piezo to 4.7 2023-05-09 15:50:18.170 - debug: sainlogic.0 (16734) Target for weeklyrain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.171 - debug: sainlogic.0 (16734) Setting value from data for weather.current.monthlyrain_piezo to 12.3 2023-05-09 15:50:18.171 - debug: sainlogic.0 (16734) Target for monthlyrain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.171 - debug: sainlogic.0 (16734) Setting value from data for weather.current.yearlyrain_piezo to 76.5 2023-05-09 15:50:18.172 - debug: sainlogic.0 (16734) Target for yearlyrain_piezo unit is set: mm, using conversion: null 2023-05-09 15:50:18.172 - debug: sainlogic.0 (16734) Setting value from data for weather.current.1.soilmoisture to 61 2023-05-09 15:50:18.172 - debug: sainlogic.0 (16734) Setting value from data for weather.current.2.soilmoisture to 52 2023-05-09 15:50:18.172 - debug: sainlogic.0 (16734) Setting value from data for weather.current.3.soilmoisture to 49 2023-05-09 15:50:18.173 - debug: sainlogic.0 (16734) Setting value from data for weather.current.4.soilmoisture to 60 2023-05-09 15:50:18.173 - debug: sainlogic.0 (16734) Setting value from data for weather.current.5.soilmoisture to 61 2023-05-09 15:50:18.173 - debug: sainlogic.0 (16734) Setting value from data for weather.current.6.soilmoisture to 50 2023-05-09 15:50:18.174 - debug: sainlogic.0 (16734) Setting value from data for weather.info.1.soilbatt to 1.6 2023-05-09 15:50:18.174 - debug: sainlogic.0 (16734) Setting value from data for weather.info.2.soilbatt to 1.6 2023-05-09 15:50:18.174 - debug: sainlogic.0 (16734) Setting value from data for weather.info.3.soilbatt to 1.6 2023-05-09 15:50:18.175 - debug: sainlogic.0 (16734) Setting value from data for weather.info.4.soilbatt to 1.6 2023-05-09 15:50:18.175 - debug: sainlogic.0 (16734) Setting value from data for weather.info.5.soilbatt to 1.7 2023-05-09 15:50:18.176 - debug: sainlogic.0 (16734) Setting value from data for weather.info.6.soilbatt to 1.6 2023-05-09 15:50:18.177 - debug: sainlogic.0 (16734) Setting value from data for info.freq to 868M 2023-05-09 15:50:18.177 - debug: sainlogic.0 (16734) Setting value from data for info.model to GW2000A 2023-05-09 15:50:18.177 - debug: sainlogic.0 (16734) Setting value from data for info.last_listener_update to {"PASSKEY":"XX","stationtype":"GW2000A_V2.2.3","runtime":"1883565","dateutc":"2023-05-09 13:50:18","tempinf":"74.12","humidityin":"53","baromrelin":"29.613","baromabsin":"29.613","tempf":"61.52","humidity":"90","winddir":"264","windspeedmph":"1.57","windgustmph":"3.80","maxdailygust":"12.53","solarradiation":"106.24","uv":"0","rrain_piezo":"0.378","erain_piezo":"0.098","hrain_piezo":"0.098","drain_piezo":"0.098","wrain_piezo":"0.185","mrain_piezo":"0.484","yrain_piezo":"3.012","ws90cap_volt":"5.2","ws90_ver":"126","soilmoisture1":"61","soilmoisture2":"52","soilmoisture3":"49","soilmoisture4":"60","soilmoisture5":"61","soilmoisture6":"50","soilbatt1":"1.6","soilbatt2":"1.6","soilbatt3":"1.6","soilbatt4":"1.6","soilbatt5":"1.7","soilbatt6":"1.6","wh90batt":"2.62","freq":"868M","model":"GW2000A","interval":"20"} 2023-05-09 15:50:18.358 - info: sainlogic.0 (16734) undefined 2023-05-09 15:50:18.358 - info: sainlogic.0 (16734) undefined

@phifogg phifogg reopened this May 10, 2023
@phifogg
Copy link
Owner

phifogg commented May 10, 2023

there is actually a log message once the forwarding has happened.
Can you put that startup part of your adapter log? This should print the config it has.

@phifogg
Copy link
Owner

phifogg commented May 28, 2023

0.10.4 has updated logging, it should not the URL as well as the response received from the request.

phifogg added a commit that referenced this issue May 28, 2023
@phifogg
Copy link
Owner

phifogg commented Jul 23, 2023

Can you confirm this still be an issue? If yes, check the logs.
Once you go to 0.10.5, make sure you put the adapter in debug mode, logs only show in debug.

@phifogg phifogg closed this as completed Aug 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants