dmsgweb
Dmsgweb is an application which permits access to websites (public-key:port) over dmsg with a web browser via proxy configuration.
Dmsgweb is now available as skywire dmsg web
The idea for the dmsgweb implementation was inspired by that of i2p, which allows access to .i2p domains or websites in a similar manner via a proxy configuration in a web browser. Dmsgweb can be additionally configured with the proxy provided by i2p in order to access i2p websites as well as .dmsg websites. The TOR browser may also be configured with dmsgweb's resolving proxy, if desired, and if dmsgweb is configured with the i2p proxy in addition, it is possible to browse .onion, .dmsg, and .i2p domains at once, if desired.
$ skywire dmsg web --help
┌┬┐┌┬┐┌─┐┌─┐┬ ┬┌─┐┌┐
│││││└─┐│ ┬│││├┤ ├┴┐
─┴┘┴ ┴└─┘└─┘└┴┘└─┘└─┘
DMSG resolving proxy & browser client - access websites and http interfaces over dmsg
.conf file may also be specified with
DMSGWEB=/path/to/dmsgweb.conf skywire dmsg web
Available Commands:
srv serve http from local port over dmsg
Flags:
-d, --dmsg-disc string dmsg discovery url (default "http://dmsgd.skywire.skycoin.com")
-z, --envs show example .conf file
-f, --filter string domain suffix to filter (default ".dmsg")
-l, --loglvl string [ debug | warn | error | fatal | panic | trace | info ]
-p, --port uint port to serve the web application (default 8080)
-r, --proxy string configure additional socks5 proxy for dmsgweb (i.e. 127.0.0.1:1080)
-t, --resolve string resolve the specified dmsg address:port on the local port & disable proxy
-e, --sess int number of dmsg servers to connect to (default 1)
-s, --sk cipher.SecKey a random key is generated if unspecified
(default 0000000000000000000000000000000000000000000000000000000000000000)
-q, --socks uint port to serve the socks5 proxy (default 4445)
-v, --version version for web
example .conf file can be used if flags are not preferable
$ skywire dmsg web -z
#########################################################################
#-- DMSGWEB CONFIG TEMPLATE
#-- Defaults shown
#-- Uncomment to change default value
#########################################################################
#-- Set port for proxy interface
#PROXYPORT=4445
#-- Configure additional proxy for dmsgvlc to use
#ADDPROXY='127.0.0.1:1080'
#-- Web Interface Port
#WEBPORT=8080
#-- Resove a specific PK to the web port (also disables proxy)
#RESOLVEPK=''
#-- Number of dmsg servers to connect to (0 unlimits)
#DMSGSESSIONS=1
#-- Dmsg port to use
#DMSGPORT=80
#-- Set secret key
#DMSGWEB_SK=''
This application is very simple and has two main parts which are wrapped in a cobra cli framework to provide the help menu.
The application consists of
- a resolving socks5 proxy which can be configured in a web browser (default http port 4445)
- an http server which translates requests to and copies responses from a dmsg(http) client (default http port 8080).
Another socks5 proxy (i.e. the skywire socks5 proxy client - default socks5://127.0.0.1:1080) can be configured via the -r flag to force both the regular http traffic as well as the dmsg client connection to the dmsg network itself through another proxy. As mentioned above, it's also possible to configure the i2p proxy for dmsgweb.
It is suggested to first create a keypair for the dmsg client to connect with. This step is optional, but recommended:
skywire dmsg web gen-keys > dmsgweb.key
If desired, configure a locally running skywire visor's proxy client for this application. refer to the socks5 proxy user guide article from this wiki for starting the proxy client.
After completing the above two steps, start dmsgweb like this:
skywire dmsg web -s $(tail -n1 dmsgweb.key) -r 127.0.0.1:1080
If you skipped the previous two steps, start dmsgweb without flags or arguments; and a keypair will be generated automatically:
skywire dmsg web
Note: it is not advisable to whitelist any public key used by dmsgweb without explicitly generating the keypair and saving to a file as detailed in the above steps.
To configure this application in a web browser, use SOCKS5 proxy 127.0.0.1:4445 (default in dmsgweb).
The process of configuring a proxy in different web browsers is described in the socks5 proxy user guide article. Simply change the port to 4445.
A few experimental dmsg websites exist, notably the skywire reward system frontend fiber.skywire.dev which can be accessed here:
http://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f.dmsg
fiber.skywire.dev/health will indicate the dmsg address it is running on.
The dmsghttp log server's /health endpoint may be accessed over dmsg for any skywire visor - if that visor is indeed connected to dmsg.
The custom dmsghttp path field in the visor's config specifies a custom path which is served by the visor over dmsghttp. This is by default a folder called "custom" inside the local_path or local folder of the skywire installation.
Any files added to the custom dmsghttp path may be accessed over dmsghttp.
If an index.html is added, a static website is readily possible for any running skywire visor and may be accessed by the visor's public key.
For advanced integration of dmsg with existing websites, see the dmsghttp article
While dmsgcurl and similar utilities use a custom URL scheme: dmsg://<pk>:<port>/ this is not supported by browsers.
To access websites over dmsg, use the following scheme:
http://<pk>.dmsg:<port>/
If the application uses the default dmsg port 80, the port can be omitted and the URL specified as follows
http://<pk>.dmsg/
Web Browsers vary to some degree in their support for accessing .dmsg addresses which are entered directly in the URL / search bar.
It may be necessary to disable auto-https redirects in settings.
For Brave browser, no matter how the .dmsg URL is entered in the search bar, a search is performed. However, clicking the link to the reward system frontend here will permit access to the dmsg website(!):
http://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f.dmsg
There is no apparent way to disable this behavior, however, a workaround is to create an html document and include the link to the dmsg website which one desires to access.
Open the html document in the browser, and then click the link in the document to access the dmsg website.
The -t, --resolve flag was added to dmsgweb to allow resolving a single dmsg pk:port locally via the web port (default 8080) without serving socks5 proxy.
dmsgweb -t 036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80
the dmsg website at 036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80 may now be accessed at http://127.0.0.1:8080
This may be used to avoid http traffic to a remote server, or avoid the use of http 301 / 302 redirects to another domain, or to avoid port forwarding.
Simply reverse proxy the http port where the remote dmsg website is served locally to your desired domain.
The dmsg client is persistent with dmsgweb ; whereas with dmsgcurl the client is started for and stopped after each task, currently. The time of the initial connection can be a significant factor in scripted operations ; hence it is advantageous to leverage the completeness of existing utilities such as curl or others, if you prefer, to do standard http requests using the resolving proxy provided by dmsgweb.
Example:
$ curl -Lx socks5h://127.0.0.1:4445 http://$(skywire cli visor pk).dmsg/health
{"build_info":{"version":"v1.3.20","commit":"8e782b1","date":"2024-03-23T01:29:00Z"},"started_at":"2024-04-06T12:41:54.253875421-05:00"}
$ skywire dmsg web srv --help
DMSG web server - serve http interface from local port over dmsg
.conf file may also be specified with
DMSGWEBSRV=/path/to/dmsgwebsrv.conf skywire dmsg web srv
Flags:
-D, --dmsg-disc string dmsg discovery url (default "http://dmsgd.skywire.skycoin.com")
-d, --dport uint dmsg port to serve (default 80)
-e, --dsess int dmsg sessions (default 1)
-z, --envs show example .conf file
-l, --lport uint local application http interface port (default 8086)
-p, --port uint port to serve (default 8081)
-s, --sk cipher.SecKey a random key is generated if unspecified
(default 0000000000000000000000000000000000000000000000000000000000000000)
-w, --wl string whitelisted keys for dmsg authenticated routes
example conf file can be used instead of flags if preferred
$ skywire dmsg web srv -z
#########################################################################
#-- DMSGWEB SRV CONFIG TEMPLATE
#-- Defaults shown
#-- Uncomment to change default value
#########################################################################
#-- DMSG port to serve
#DMSGPORT=80
#-- Port for this application to serve http
#WEBPORT=8081
#-- Local Port to serve over dmsg
LOCALPORT=8086
#-- Number of dmsg servers to connect to (0 unlimits)
#DMSGSESSIONS=1
#-- Set secret key
#DMSGWEBSRV_SK=''
#-- Whitelisted keys to access the web interface
#WHITELISTPKS=('')
A server side implementation has been included as the srv subcommand of dmsgweb, which allows for easily serving http from a local port over dmsg.
As a test of serving and accessing so-called complex websites or web interfaces working over dmsg, accessing the skywire hypervisor makes a good demonstration
Run skywire with the hypervisor on one machine. On the same machine, then run:
$ skywire dmsg web srv -l 8000
[2024-05-20T15:00:57.724443172-05:00] DEBUG disc.NewHTTP [dmsgwebsrv]: Created HTTP client. addr="http://dmsgd.skywire.skycoin.com"
[2024-05-20T15:00:57.724535349-05:00] DEBUG [dmsg_client]: Discovering dmsg servers...
[2024-05-20T15:00:58.208964915-05:00] DEBUG [dmsg_client]: Dialing session... remote_pk=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-05-20T15:01:03.34028898-05:00] WARN [dmsg_client]: Failed to establish session. current_backoff="5s" error="deadline exceeded" remote_pk=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-05-20T15:01:08.344464474-05:00] DEBUG [dmsg_client]: Dialing session... remote_pk=0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb
[2024-05-20T15:01:13.483605911-05:00] WARN [dmsg_client]: Failed to establish session. current_backoff="6.5s" error="deadline exceeded" remote_pk=0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb
[2024-05-20T15:01:19.986939676-05:00] DEBUG [dmsg_client]: Dialing session... remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2024-05-20T15:01:24.99413617-05:00] WARN [dmsg_client]: Failed to establish session. current_backoff="8.45s" error="deadline exceeded" remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2024-05-20T15:01:33.449717262-05:00] DEBUG [dmsg_client]: Dialing session... remote_pk=03d5b55d1133b26485c664cf8b95cff6746d1e321c34e48c9fed293eff0d6d49e5
[2024-05-20T15:01:34.396529156-05:00] DEBUG [dmsg_client]: Serving session. remote_pk=03d5b55d1133b26485c664cf8b95cff6746d1e321c34e48c9fed293eff0d6d49e5
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] POST /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] PUT /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] PATCH /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] HEAD /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] OPTIONS /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] DELETE /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] CONNECT /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
[GIN-debug] TRACE /*path --> github.com/skycoin/dmsg/cmd/dmsgweb/commands.server.func3 (3 handlers)
listening on http://127.0.0.1:8081 using gin router
[GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value.
Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.
[2024-05-20T15:01:34.396851581-05:00] INFO [dmsgwebsrv]: Serving... dmsg_addr="0367b4bd78c53e6fc24f8f83cd9ad40ba09e62a950f0181a943a9f46e3110ae9bf:80"
[GIN-debug] Listening and serving HTTP on :8081
note the second to last line of logging. transfer the dmsg address which in the above is; 0367b4bd78c53e6fc24f8f83cd9ad40ba09e62a950f0181a943a9f46e3110ae9bf (yours will be different) to the remote machine you wish to access the interface from, and run
skywire dmsg web -t 0367b4bd78c53e6fc24f8f83cd9ad40ba09e62a950f0181a943a9f46e3110ae9bf -p 8000
then go to 127.0.0.1:8000 to access the hypervisor UI
In the above example the whitelist was not demonstrated. Setting no whitelisted keys by default makes the application accessible to any other key; conversely, setting keys in the whitelist effectively authenticates the clients which are accessing the interface. It is highly recommended to set a whitelist of keys or a key which may access the interface.