Configuration refactor proposal

[paspo]

The current way of configuring hbbs/hbbr is not clear.

This is a proposal to address some issues and this should be the start for a discussion, to better rationalize the configuration.

Environment variables

Main goals:

• different variables for different binary
• allow secrets to be but in a file, instead of an environment variable or a command line parameter
• command line parameters should override environment variables
• there's no need for a public key (file or environment variable): this can be derived from the secret key
• each binary allow environment to be automatically loaded (and overridden) from an .env file
• hbbr/hbbs should be able to run from ENV only (plus DB for hbbs), cli args only, .env file only, or a mix

hbbs

variable	previous	default value	cli arg	meaning
HBBS_SECRET_KEY	KEY	-	--key	the secret key
HBBS_SECRET_KEY_FILE	-	-		path to a file containing the secret key
HBBS_DB_FILE	DB_URL	db_v2.sqlite3		path to the sqlite3 database file
HBBS_ALLOW_UNENCRYPTED	-	N		if "Y" explicitly allow non encrypted connections
HBBS_ALLOW_ANY_KEY	-	N		if "Y" explicitly allow any encryption key
HBBS_ALWAYS_USE_RELAY	ALWAYS_USE_RELAY	N		If "Y" disallow direct peer connection
HBBS_RELAY_SERVERS	RELAY_SERVERS	-	--relay-servers	IP address/DNS name of the machines running hbbr (separated by comma)
HBBS_PORT	PORT	21116	--port	hbbs listening port
MAX_DATABASE_CONNECTIONS	MAX_DATABASE_CONNECTIONS	1
RUST_LOG	RUST_LOG	error		set debug level (error, warn, info, debug, trace)
			--config	custom config file
			--serial	update serial number
			--software-url	new release download url
			--rmem	Sets UDP recv buffer size
			--local-ip
			--mask	Determine if the connection comes from LAN, e.g. 192.168.0.0/16
			--rendezvous_servers	??

Notes:

• HBBS_SECRET_KEY and HBBS_SECRET_KEY_FILE are mutually exclusive. The program will stop for configuration error if both are specified.
• HBBS_SECRET_KEY (or HBBS_SECRET_KEY_FILE) is required unless HBBS_ALLOW_UNENCRYPTED=1 or HBBS_ALLOW_ANY_KEY=1 is specified
• HBBS_SECRET_KEY should be a valid key (- is not allowed, - is not allowed)
• HBBS_SECRET_KEY_FILE should refer to a file containing a valid key (- is not allowed, - is not allowed)
• KEY should be a valid key or - or -
• HBBS_DB_FILE is required. The destination file is created if not already present.

hbbr

variable	previous	default value	cli arg	meaning
HBBR_SECRET_KEY	KEY	-	--key	the secret key
HBBR_SECRET_KEY_FILE	-	-		path to a file containing the secret key
HBBR_ALLOW_UNENCRYPTED	-	N		if "Y" explicitly allow non encrypted connections
HBBR_ALLOW_ANY_KEY	-	N		if "Y" explicitly allow any encryption key
HBBR_PORT	PORT	21117	--port	hbbr listening port
HBBR_DOWNGRADE_DELAY	DOWNGRADE_START_CHECK	1800		delay (in seconds) before downgrade check
HBBR_DOWNGRADE_THRESHOLD	DOWNGRADE_THRESHOLD	0.66		threshold of downgrade check (bit/ms)
HBBR_BLACKLIST_BANDWIDTH	LIMIT_SPEED	4		speed limit (in Mb/s) for blacklisted IPs
HBBR_SINGLE_BANDWIDTH	SINGLE_BANDWIDTH	16		max bandwidth for a single connection (in mbps)
HBBR_TOTAL_BANDWIDTH	TOTAL_BANDWIDTH	1024		max total bandwidth (in mpbs) for hbbr process
HBBR_BLACKLIST_FILE		blacklist.txt		path to a file containing a blacklist (IPs bandwidth limited)
HBBR_BLOCKLIST_FILE		blocklist.txt		path to a file containing a blocklist (IPs not allowed to connect)
RUST_LOG	RUST_LOG	error		set debug level (off, error, warn, info, debug, trace)

Notes:

• HBBR_SECRET_KEY and HBBR_SECRET_KEY_FILE are mutually exclusive. The program will stop for configuration error if both are specified.
• HBBR_SECRET_KEY (or HBBR_SECRET_KEY_FILE) is required unless HBBR_ALLOW_UNENCRYPTED=1 or HBBR_ALLOW_ANY_KEY=1 is specified
• HBBR_SECRET_KEY should be a valid key (- is not allowed, - is not allowed)
• HBBR_SECRET_KEY_FILE should refer to a file containing a valid key (- is not allowed, - is not allowed)
• KEY should be a valid key or _ or –

error logging

It is possible to specify different logging level for both programs, by setting RUST_LOG in this way: RUST_LOG=hbbr=warn,hbbs=off.

some clarification needed

• (hbbr) What is the difference between blacklist.txt and blocklist.txt? And why is this not used in hbbs?
• (hbbs) What is the meaning of the environment variable TEST_HBBS? There's only a comment saying "temp solution to solve udp socket failure". I understand that this can be an IP address (maybe the external IP of the server when natted), but it keeps failing for me with "Error: invalid socket address syntax"
• (hbbs) What is the meaning of the command line argument "--software-url"? I understand that if I set like "--software-url=http://somedomain.com/something-44", then the calculated version is "44", but where is this information used?
• (hbbs) There are some other command line arguments that are unclear (to me): --local-ip, --mask.
• (hbbs) What is the meaning of the environment variables PORT_FOR_API and KEY_FOR_API?
• (hbbs) Is MAX_DATABASE_CONNECTIONS needed?
• (hbbs) What is the meaning of --rendezvous-servers?

other ideas

Previous environment variables should generate a deprecation warning message in the log.
It's also a good idea to indicate in which version the variable will be removed (next major release?).

There's no specific log configuration, the only option is setting the RUST_LOG variable; this has to be improved. One idea is to differentiate log levels for the two binaries, another is to add an option to log to a file; this should cover the vast majority of cases.

Instead of using blacklist.txt and blocklist.txt, we can use a database. Maybe the same database we use for hbbs.

Is the .env file really necessary? It can create confusion with docker-compose.

[rustdesk]

(hbbr) What is the difference between blacklist.txt and blocklist.txt? And why is this not used in hbbs?

blacklist, just limit its bandwidth

rustdesk-server/src/relay_server.rs

Line 527 in bac9548

blacked = BLACKLIST.read().await.get(&ip).is_some();

blocklist, will block its connection

rustdesk-server/src/relay_server.rs

Line 382 in bac9548

if BLOCKLIST.read().await.get(&ip).is_some() {

About why not in hbbs, there is no why, just not did it. :(

(hbbs) What is the meaning of the environment variable TEST_HBBS? There's only a comment saying "temp solution to solve udp socket failure". I understand that this can be an IP address (maybe the external IP of the server when natted), but it keeps failing for me with "Error: invalid socket address syntax"

It is used in case lo0 is not 127.0.0.1, I have never used it though I added it. :(

rustdesk-server/src/rendezvous_server.rs

Line 145 in bac9548

let test_addr = std::env::var("TEST_HBBS").unwrap_or_default();

test_hbbs is for health check, because I found our udp on public server failed when big udp traffic sometimes, it can only recover after restarting hbbs, I did not know the reason, so add this as the solution.

(hbbs) What is the meaning of the command line argument "--software-url"? I understand that if I set like "--software-url=http://somedomain.com/something-44", then the calculated version is "44", but where is this information used?

ignore this, useless for self-hosting. My bad, not clear code enough when publish this.

(hbbs) There are some other command line arguments that are unclear (to me): --local-ip, --mask.

I attempted to solve internet/intranet access issue before, but it did not work. please ignore this.

(hbbs) What is the meaning of the environment variables PORT_FOR_API and KEY_FOR_API?

ignore this, useless for self-hosting.

(hbbs) Is MAX_DATABASE_CONNECTIONS needed?

useless for sqlite I guess

(hbbs) What is the meaning of --rendezvous-servers?

ignore this, useless for self-hosting.

[paspo]

About why not in hbbs, there is no why, just not did it. :(

IMHO, this should be considered.

In any web application the bandwidth limit is usually not considered; that's because a webapp is put behind a reverse proxy to do SSL termination, some security and this includes bandwidth limitation. But rustdesk-server is very different from a web application and putting it behind a reverse proxy is most of the times a problem instead of a solution to a problem; this happens only in very specific edge cases which we should not consider.

So rustdesk-server (I'm talking about this open-source self-hosted one, I don't know about the pro version) is usually never assisted by a reverse proxy, and basic bandwidth limitation and security have to be managed by hbbr and hbbs themselves.

With more intensive logging of bad behavior (eg. wrong key connections) we can use tools like fail2ban or crowdsec to ban ips.

[paspo]

About all the "useless for self-hosting" things, we need to discuss.

On one side, this code is useless and confusing for the open-source version (and should not be present), on the other side, this is probably needed on the pro version. So now the issue is "code parity" between the two versions.

I imagine that (given all the differences in functionality) now the two rustdesk-server versions are basically two different projects, and porting features and modifications between the two is difficult.

Lots of other commercial projects are solving this by implementing some sort of "plugin logic": the open source version is kept as a common base and the pro functionalities are an extension of this base. This can be a good way to solve it. Just my 2 cents for @rustdesk .

[paspo]

(hbbs) Is MAX_DATABASE_CONNECTIONS needed?

useless for sqlite I guess

I suppose more database options are available in the pro version.
Can the blocklist and blacklist be moved into db?

[rustdesk]

.env

It is not a must, but it is added because I like to use .env. :(

rustdesk-server/src/common.rs

Line 64 in bac9548

if let Ok(v) = Ini::load_from_file(".env") {

[paspo]

More configuration are ok, we just need:

• a clear and consistent order of precedence (cli args, env variables, env files)
• a clear documentation

[xlionjuan]

What is HBBR_LIMIT_SPEED?

I can understand what is HBBR_SINGLE_BANDWIDTH and HBBR_TOTAL_BANDWIDTH, but another LIMIT SPEED is very confusing.

[paspo]

Only HBBR_LIMIT_SPEED is unclear, HBBR_TOTAL_BANDWIDTH is for entire HBBR process.

I'll update the list in the top of this discussion.

HBBR_SINGLE_BANDWIDTH for single connection, by default it is 16Mbps (Which is 2MB/s), it only bother me when doing file transfer, it will stay on 2MB/s, so I increase it manually to get higher transfer speed.

Do you think it'll be useful to differentiate the two cases, with different limits?
In fact, the limit is in place to protect the server from overload, so maybe this type of transfer should be "best effort, up to XXX mbps divided between all file transfers".

[xlionjuan]

Yes, both HBBR_TOTAL_BANDWIDTH and HBBR_SINGLE_BANDWIDTH is needed

Total is needed for protect the server or this server (or same network) has another service running, to prevent DDoS your other service by yourself.

Single is needed to

1. Prevent abuse (You wanna reduce it)
2. You wanna maximize the clients but your server bandiwidth isn't fast enough (You wanna reduce it)
3. You wanna transfer file at max speed as fast as possible (You wanna increase it)

---

Another suggestion but hard to do
You could let HBBR support PCQ (Per Connection Queue)+QoS
Just like the name it called, for example, HBBR_TOTAL_BANDWIDTH is set to 100 Mbps, when 1 user is transferring files, it can use all the speed that server allowed, if 2, each get 50 Mbps, if 4, each get 25 Mbps..etc.

And QoS for set the minimum (guarantee) speed for Remote Desktop, maybe 500Kbps per session, because Remote Desktop should get 1st priority than file transfer.

[rustdesk]

(hbbr) What is the difference between blacklist.txt and blocklist.txt? And why is this not used in hbbs?

blacklist, just limit its bandwidth

HBBR_LIMIT_SPEED is used here.

rustdesk-server/src/relay_server.rs

Line 469 in bac9548

let blacklist_limiter = <Limiter>::new(LIMIT_SPEED.load(Ordering::SeqCst) as _);

[paspo]

so a better name should be HBBR_BLACKLIST_BANDWIDTH to make clear its relationship with HBBR_BLACKLIST_FILE. Am I right?

[rustdesk]

I am ok with your new name.

[GuillaumeHullin]

Hi there, very interesting conversation that helped me a lot clarified the variable and what they are doing.

As @paspo mentioned,

With more intensive logging of bad behavior (eg. wrong key connections) we can use tools like fail2ban or crowdsec to ban ips.

That's exactly what I'm trying to achieve... a way to log and ban bad behaviours.
I want to use Zabbix and Fail2Ban for those respective purpose (I can write template/settings for both).

Here are my questions:
1- What triggers the addition of IP in blocklist?
2- What triggers the addition of IP in blacklist? is it related to those DOWNGRADE variables?
3- The variable RUST_LOG control the log level... but is this log saved in the filesystem somewhere?

The log data I'm the most interested to store would be:

INFO [src/relay_server.rs:441] New relay request xxxxxx from [::ffff:ip_address]:59246
DEBUG [src/rendezvous_server.rs:742] Punch hole "1234" [::ffff: ip_address]:64187 request from [::ffff: ip_address]:59233

[xlionjuan]

Is downgrade related to TCP congestion control?

[BlockheadVFX-IT]

@rustdesk Where does the environment variable file .env live?

Lets say if i have a hbbs/hbbr server , where should .env be placed?

or is it just a Global/System Env variable?

[xlionjuan]

.env should put on same directory of docker-compose.yml

But unless you need, or you won't need to separate a .env file.

[paspo]

In fact, we can have two different .env files.

If you're using docker-compose, you can have a .env file with the relevant environment variables, it's a way to keep all variables in the same place. This file is only used with docker-compose.

In addition to that, each binary (hbbr and hbbs) will check if an .env file exists in the working directory, if it exists, it'll be loaded and it'll be used to set the environment variables for that process.
See the previous comment #371 (comment) from @rustdesk

[dimejo]

This is looking great! Few questions/notes from me:

• Is there any information available for cli args like --config, --serial and --software-url?
• RUST_LOG: RUST_LOG_LEVEL would be more unambiguous as RUST_LOG could mean log location as well
• HBBR_DOWNGRADE_START_CHECK: IMHO HBBR_DOWNGRADE_DELAY or HBBR_DOWNGRADE_START_DELAY would be better
• HBBR_BLACKLIST_FILE: Only single IPs or is a CIDR notation allowed aswell?

[paspo]

--serial and --software-url should be removed from this version of rustdesk-server.

[paspo]

--config lets you specify an additional .env file from which you load some configuration parameters.

rustdesk-server/src/common.rs

Lines 71 to 79 in 4240c47

	if let Some(config) = matches.value_of("config") {
	if let Ok(v) = Ini::load_from_file(config) {
	if let Some(section) = v.section(None::<String>) {
	section
	.iter()
	.for_each(|(k, v)| std::env::set_var(arg_name(k), v));
	}
	}
	}

[paspo]

I agree with you, RUST_LOG_LEVEL is a better choice. RUST_LOG comes from a standard rust logging library and sadly we can't rename that without implementing a different logging approach.

[paspo]

HBBR_DOWNGRADE_DELAY is actually better because it refers to an amount of seconds.