Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Thing by URL #69

Closed
benfrancis opened this issue May 10, 2017 · 36 comments
Closed

Add Thing by URL #69

benfrancis opened this issue May 10, 2017 · 36 comments
Assignees
Milestone

Comments

@benfrancis
Copy link
Member

As a user I want to add a Web Thing by its URL, to be managed by my gateway.

This is for Web Things which use the direct integration pattern so provides its own Web Thing API but which the user might want to manage together with other Things using the gateway.

@hobinjk
Copy link
Contributor

hobinjk commented Feb 22, 2018

I have an initial mockup of an add-on implementing this feature at https://github.com/hobinjk/thing-url-adapter.

I found that we'll need this kind of proxy instead of making WoT requests directly in the UI since the Gateway is full-fledged HTTPS and the ESP8266 server I'm using is HTTP and would likely be at most self-signed HTTPS.

@benfrancis
Copy link
Member Author

I'd been wondering about this. We've talked about native web things using the "direct" integration pattern, but really if you're using the gateway to bridge a web server on the local network to the Internet then they're using the gateway integration pattern. The gateway is just acting as a proxy rather than bridging protocols.

I'm curious whether if you enabled CORS on the device's web server that would allow you to call its API directly from the front end, or whether you would get mixed content warnings? And does HSTS make a difference to this? But regardless, using a local IP address or .local domain from the client side would not work if you were outside the local network, the device needs to be accessible via the Internet.

So there are potentially two possible integration patterns with "native" web things which expose the Web Thing API:

  • Direct integration pattern where the device is directly accessible via the Internet using NAT or a tunnelling solution or similar and the device exposes its own URL on the Internet using its own domain or public IP address
  • Gateway integration pattern where the device just exposes a local web server (over a local IP or .local domain which can not use HTTPS without a self-signed certificate so won't be as secure) and the gateway acts as a proxy, giving the device its URL on the Internet

In the former case, the gateway could just store a record of the device's URL and Thing Description and the front end should theoretically be able to call the device's API directly. In the latter case the gateway would have to provide the API for the device, even if it is proxying those requests to the same kind of API.

We certainly need an adapter for the gateway integration pattern, but should we have a separate implementation for web things using the direct integration pattern? Note that this same implementation could also be used for adding native web things to your gateway UI which are using the cloud integration pattern, or are hosted elsewhere on the Internet on another gateway.

We probably do need a "local web thing" adapter like the thing-url-adapter you've started because that's the easiest approach for native web things to use and not need a custom adapter, but I wonder whether we should separately support calling native web things directly, without an adapter?

BTW I think manually adding a Thing by its URL will require different UI, which I'm working on now. How did you do the UI in this experiment? Searching for native web things using bluetooth beacons or mDNS could probably use the current UI.

@benfrancis
Copy link
Member Author

Note that if we do implement a native Things Browser mobile app, that will likely only add things by their thing URL and call the things' APIs directly.

@hobinjk
Copy link
Contributor

hobinjk commented Feb 23, 2018

I like the idea of supporting the direct integration pattern. All of the UI is written with use of the thing description's href so it wouldn't be difficult to do from a technical perspective. It would reduce latency and be a good demonstration of the power of exposing the WoT API.

I unfortunately couldn't find a combination of CORS and CSP that allowed bypassing the mixed active content blocking enforced when fetching an HTTP resource. HSTS does not have an effect either.

The addon has no custom UI since I figured the next step would be to hook it up to mDNS/bluetooth discovery

@ghost
Copy link

ghost commented Feb 27, 2018

Hello, this story is basically my story. Would it be possible to add the "local web thing" device to the gateway based on the device's dns name on the local network? For example, if the device had a DNS name of "coffeemaker", the user could enter 'coffeemaker' into a form and then the gateway would fetch the device's information via the WOT RESTful API.

@benfrancis
Copy link
Member Author

Hello, this story is basically my story. Would it be possible to add the "local web thing" device to the gateway based on the device's dns name on the local network? For example, if the device had a DNS name of "coffeemaker", the user could enter 'coffeemaker' into a form and then the gateway would fetch the device's information via the WOT RESTful API.

Yes! Please stand by for UI mockup

@ghost
Copy link

ghost commented Feb 27, 2018

Excellent news. I would also like to suggest that the gateway handles the communication with local Things and the UI for the Gateway only communicates with the Gateway and not with the Things. For example, suppose jezra forgets to turn off the coffee maker before leaving for work. Jezra would access the Gateway UI, press the "coffee maker Off" button, and the command would be sent to the Gateway, which in turn would issue an HTTP request to the coffee maker's API.

@rei-vilo
Copy link

rei-vilo commented Mar 17, 2018

Does release 0.3.1 include the Add Thing by URL feature?

Allow add-ons to register multiple adapters

I am not sure of the feature added.

Also at WebThingsIO/webthing-arduino#1 (comment)

@rei-vilo
Copy link

There seem to be two candidates for the WoT:

How do they differ? Which one do you recommend?

@benfrancis
Copy link
Member Author

Does release 0.3.1 include the Add Thing by URL feature?

No, as you can see this issue is still open. You can see UI mockups here and this is planned for 0.4.

There seem to be two candidates for the WoT:

https://github.com/fiz1962/ESP-adapter
https://github.com/hobinjk/esp8266-wot

How do they differ? Which one do you recommend?

The first one is an add-on adapter for the gateway, I'm not sure what it actually does. The second one is an example native web thing built with an esp8266 which supports some parts of the Web Thing API directly so won't need a custom adapter once we have the add thing by URL feature.

I'd recommend building a native web thing like @hobinjk's example which should work without an adapter in 0.4, we should have a more complete example ready by then.

@benfrancis
Copy link
Member Author

@rei-vilo
Copy link

@benfrancis The icons are not public and difficult to reach. Google Drive requires permission if and when your request is approved.

First...

You need permission

Want in? Ask for access, or switch to an account with permission. Learn more

...and then...

You need permission

Your request for access has been sent.

You will receive an email if and when your request is approved.

@fiz1962
Copy link

fiz1962 commented Mar 20, 2018

These 2 are basically the same. esp8266-wot responds at http://ipaddress/ with a "Thing" descriptor and ESP-adapter INO responds at http://ipaddress/things/esp with a Thing descriptor. I added an adapter simply because I couldn't find Add Thing by URL and hadn't seen esp8266-wot. Personally, I think esp8266-wot is better written. I anxiously await 0.4.0 :)

https://github.com/fiz1962/ESP-adapter
https://github.com/hobinjk/esp8266-wot

@OlavGullaksen
Copy link

@fiz1962, as you know, I have tried hard and used some time now on having the ESP-adapter up running using an ESP-01 unit, but with no success. I am sorry to say that this solution has not been properly checked before release and the Readme file is not for much help. But at least I have now a better knowledge of how the gateway server works :-)

I really look forward to be able to use the cheap ESP-01 unit to build all kind of IoT devices for the Mozilla Things gateway. When a solution is up running and tested well, I suggest to have some detailed tutorials on how to make your own IoT device. This could cause a big boom in use of Mozilla Things IoT in the MAKER society's around the world.

Thank you all for your great work with this ...

@mrstegeman mrstegeman self-assigned this Mar 27, 2018
mrstegeman pushed a commit to mrstegeman/gateway that referenced this issue Mar 29, 2018
mrstegeman pushed a commit to mrstegeman/gateway that referenced this issue Mar 29, 2018
@ghost ghost added review and removed in progress labels Mar 29, 2018
@ghost ghost removed the review label Mar 30, 2018
@ghost
Copy link

ghost commented Mar 30, 2018

How does a user implement this functionality?
I have performed a git pull to get the latest master branch, but I am not seeing any option to Add Thing By URL

@mrstegeman
Copy link
Contributor

You have to have thing-url-adapter installed and enabled, which we have not yet pulled into the add-on list. We'll do that shortly.

After that happens, when you click the + on the Things screen to add a new thing, you should see a clickable link that will allow you to "Add by URL...".

@ghost
Copy link

ghost commented Mar 30, 2018

excellent, thank you

@mrstegeman
Copy link
Contributor

By "shortly", I mean with the next week. We're working out a method for cross-compiling add-ons for different architectures, so this add-on is pending that work. In the mean time, you can install it yourself, if desired. Instructions here.

@ghost
Copy link

ghost commented Mar 30, 2018

I have installed, added 2 urls through the UI, and have not been able to get the gateway to control the 2 Things. Is https a requirement? and are there any port requirement?

@mrstegeman
Copy link
Contributor

No, those are not requirements. Can you attach the run-app.log file? What web thing are you testing with?

@ghost
Copy link

ghost commented Mar 31, 2018

It is a home-rolled Thing, and the log is showing errors in the data being returned by the Thing. I apologize for not being more thorough before writing.

@mrstegeman mrstegeman added this to Done in WebThings Gateway Apr 9, 2019
@rhladky
Copy link

rhladky commented Nov 15, 2019

I am not sure how to add URL. I have Yamaha receiver on my network supporting REST API (e.g. http://192.168.1.149/YamahaExtendedControl/v1/main/setPower?power=on to power on the receiver). I click on "Add by URL..." button and try to enter http://192.168.1.149/YamahaExtendedControl/v1/. After some time number "400" appears (looks like Bad Request) and nothing else happen. What exactly is the URL to be used?

@mrstegeman
Copy link
Contributor

The "Add by URL" functionality is specifically for native web things, which use the Web Thing API. For anything else, an adapter add-on is needed.

@rhladky
Copy link

rhladky commented Nov 16, 2019

@mrstegeman - Thanks for response. I misunderstood it as I thought the thing-url-adapter can be used for any REST API-enabled devices. So if I have a "proprietary" device exposing REST API (like my Yamaha receiver or Philips Andorid TV), I have to write a new adapter specifically for this device. Is there any instructions how to do so. I have found some article how to "create web thing", but what I cannot find is how to deploy my code to my Raspbarry Pi gateway.

@mrstegeman
Copy link
Contributor

Yes, exactly. Here are a couple articles:

In general, the easiest thing to do is git clone one of the existing add-ons into ~/.mozilla-iot/addons, change the directory name to something more appropriate, and then start changing the adapter. To test, you'll restart the gateway with sudo systemctl restart mozilla-iot-gateway and then enable your adapter via Settings -> Add-ons.

@tim-hellhake
Copy link
Member

@rhladky
I'm already working on a Yamaha adapter.
Would you like to test it?
Btw. there is a generic HTTP adapter you can use.

@rhladky
Copy link

rhladky commented Nov 18, 2019

@tim-hellhake - over the weekend I did play with the generic HTTP adapter. I did configure two actions in the adapter (Yamaha On and Yamaha Off). It works as expected, but the visual presentation in Gateway UI is not the best - it created two THINGS each with one action. It would be great to have one THING with two actions. So definitely I am keen on to test your native Yamaha adapter.

I have one more question - the primary reason I experiment with Gateway is I wanted to integrate my Google Home with devices on my local network (Yamaha, Philips TV) via IFTTT. Isn't there an article describing how to integrate IFTTT with Mozilla IoT Gateway?

@mrstegeman
Copy link
Contributor

It would be great to have one THING with two actions.

I agree! That's what tim-hellhake/http-adapter#1 is for. 😄

As for IFTTT, no, we don't have official support. See #178.

@tim-hellhake
Copy link
Member

tim-hellhake commented Nov 18, 2019

@mrstegeman
Damn, I missed that PR.
I wish there was an overview of all issues/PRs of your projects.

@rhladky
I remember someone playing with ifttt webhooks.
#1917

@mrstegeman
Copy link
Contributor

@mrstegeman
Damn, I missed that PR.
I wish there was an overview of all issues/PRs of your projects.

I have 7 others open against your repos if you want a list. 😆

@tim-hellhake
Copy link
Member

@rhladky

It would be great to have one THING with two actions.

As soon as WebThingsIO/addon-list#561 is merged you will be able to have multiple actions per thing.

@tim-hellhake
Copy link
Member

@mrstegeman

I have 7 others open against your repos if you want a list.

No need, I wrote a little script to claim all unassigned issues/prs 😬

@tim-hellhake
Copy link
Member

@mrstegeman
Wait... I see four PRs and one issue now.
Are there any open PRs/issues from you which are not assigned to me?

@tim-hellhake
Copy link
Member

@mrstegeman
Turns out the page size was to low 😀

@mrstegeman
Copy link
Contributor

That would do it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

8 participants