Adding SRV Record Support

[Jedimaster4559]

Good Evening everyone!

Would there be any interested in adding support for SRV Records (for some background on SRV Records, see here). I might be interested in implementing this if the main contributors don't see a problem with me working on adding it!

For reference, the advantage (to me) for adding this is that it would allow for multiple private servers to be hosted behind one public IP without having to worry about users specifying the port for the server they would like to connect to. The SRV record setup with the DNS provider indicates the ports/protocols and will direct traffic properly (assuming the application supports it).

Let me know your thoughts! I'd love to look into implementing this if the dev team will let me!

[ghost]

Jamulus Directory Server is not Kerberos, or Active Directory, or LDAP. Thus, SRV is such a large hammer that it is most useful as a door-stop.

[Jedimaster4559]

I'm not suggesting making any changes to the Jamulus Directory Server. I'm suggesting allowing for running multiple servers behind one public IP address without requiring that users specify a port (which as mentioned below can get confusing, especially for musicians who may not be technically adept). This should be a change that only needs to be made to the client.

[hoffie]

I was thinking about exactly that when onboarding different ensembles on different Jamulus instances. Having a way for "virtual hosting" would make this easier, both for users and for hosters.
Using SRV records to do that is a really light-weight way without having to touch any protocol parts.

Implementation-wise, it will probably only affect client-related connection code, right?

The alternatives have downsides:

• Specifying the port is not user-friendly and quickly gets confusing when running many instances.
• Using different public IP addresses is wasteful in times where the number of IPv4 addresses is a bottle-neck.

I like the idea, but we should wait what other developers say.

[Jedimaster4559]

Yes. This would likely be only a change to how the client resolves it's DNS. There shouldn't need to be any changes to the server implementation.

[Jedimaster4559]

I just wanted to follow up on this post. Does anyone else have any thoughts on this? If not, I may see if I can implement this later this summer.

[gilgongo]

Seems like a good enhancement if those who need it can take advantage of if without confusing or disadvantaging those who can't.

[jugo6]

Perhaps not the solution what you are looking for, but my different ensembles gather on my private server which is a central server (or directory server now), and there they can choose from different rooms named after the ensemble. We did the same on a festival.

btw I came across this looking for the issue of letting 2 domainnames point to the same IP, so two different doors to enter my server. . might a SRV record solve that?

[gilgongo]

@pljones yes, but in the context of this discussion about SRV I'm not sure that answers @jugo6 question or not.

[pljones]

No, it's not clear what's being referred to.

[Jedimaster4559]

@jugo6 Yes, It's possible that a central server might be able to work as a solution to my original problem. It really depends on how the central server handles ports and/or it's connection to the client servers. The real question there would be is if the central server can properly direct traffic if both it and all the client servers are on the same local network. Might be worth some investigation.

That being said, I still think SRV records may be a batter option since most of the people who are using my jamulus servers lack technical skills. Having them just input a single domain name is much simpler than adding a new central server then finding the server they are supposed to go to. It's a good thought though. I may go try playing with it sometime to see if it might work.

Also, apologies on delays implementing this. I wanted to get it done last summer but other projects and life have taken priority. I'm hoping to eventually get around to this if my life ever clears up some.

[pljones]

The real question there would be is if the central server can properly direct traffic if both it and all the client servers are on the same local network. Might be worth some investigation.

That already works (note that I run 8 directories - Rock, Jazz, Classical and Choral directories are on the same box as four other of my directories and two of my jam servers, and one of my jam servers is registered with one of my directories) -- and I can happily connect to anything on my LAN.

[Jedimaster4559]

@pljones Awesome! Sounds like that may actually solve my use case. I'm looking forward to trying that out when I get home after the holidays.

[gene96817]

Today is a holiday. I finally had time to be curious about this topic. What problem are we trying to solve? Why doesn't the use of a domain name solve the problem? By using domain names:

• multiple domain names can point to the same IP address
• you can set up your local LAN to also use the domain name to reach a server (without the user knowing a local IP address)
• you can have backup servers that are transparent to your users.
• the client can connect to the server without using the directory server.

The main drawback is you need to create a domain name for your server.

[gene96817]

In the past 1.5 years that I have known Jamulus, my relationship has been primarily as a user. Since I am not a coder, I try to honor the coders and recognize their opinions count more than non-coders. Jamulus has some very clever "hacks", especially in the avoidance of using domain names and managing ports across the IPv4 (NAT/firewall) port mapping boundary. As the desire to add more features arises, we could reach the limits of the original tricks that up to now have kept things simple.

1. The SRV record is part of the DNS server functions. If we think of Jamulus as a service, then the SRV records can help (with some of the issues required to) manage port numbers. Can SRV be made to work without domain names? Is the title of this thread suggesting that we need to create an SRV-like function for Jamulus?
2. Is it time to consider using domain names with Jamulus? We would lose the ability to spin up a server without the delay of propagating server information across the Internet DNS infrastructure (could be 48 hours). We should think about how backward compatibility works. Without backward compatibility, everyone spinning up a server would need a domain name, it's not expensive but it is not free. We could create a convention to allow myUUID.jamulu.io, yourUUID.jamulus.io, anotherUUID.jamulus.io, etc.
3. To manage port numbers, look at some of the other services for example, e.g. FTP, telnet, etc. They all have a common port assignment and then the service manages ports as sessions are created. (That is, every new client starts with a connection to the common port and then gets an assignment to another port. This keeps the port assignments local, avoiding global issues.)
4. Most of our users have "simple" IPv4 NAT/firewalls. Are we about to invent a convention that requires extreme NAT tricks (i.e. difficult to teach and support)?
5. Would servers needing SRV record support benefit from using IPv6? There would need to be a design to support IPv4 clients.

@Jedimaster4559 Answers to your questions:

• multiple domain names -- this fails if it has to go through a NAT to reach servers on a private LAN. Then you need a router instead of a NAT/firewall.
• local LAN to also use the domain name to reach a server -- your gateway/NAT/firewall has a local host table. You need to put the Jamulus server info on this host table. Then local users can reach the server in the same way as the remote user. Using this method, a client on the local LAN can get the Jamulus server public IP address from the directory server and still reach a Jamulus server on the local LAN, without knowing the Jamulus server's local LAN address.
• connect to server without using the directory server. -- how many Jamulus servers do you run on your private LAN? Seems to me you can run a private directory server to accomplish your goals without SRV/DNS.

[rdica]

Do I understand this problem correctly:
Since users are going to be typing in a hostname or IP anyway (whether for a direct server connect, or to add a Directory server address in Advanced Setup), you don't want them typing in six more characters, ie; :22524?

From everything I've read here, it looks like you're making things far more complicated than they have to be, and the solution is a custom directory server.

[gene96817]

@rdica You are right... I took the bait on SRV... In fact, if Jamulus is not running with other services, there is no reason Jamulus cannot use port 666 or any other number. Port numbers are assigned as a matter of convention.

[dcorson-ticino-com]

Especially if viewing from an non-tech users standpoint.
Entering a custom directory in the settings once and then using that and choosing from a list of servers is really easy.
As long as the custom directory server doesn't constantly change your non-tech users will have no problems.

[Jedimaster4559]

@gene96817 If I were to implement this, it would by no means require servers to use domain names. In fact, everything should always be exactly the same unless you specifically want to use an SRV record. My goal here isn't to change how Jamulus handles DNS or IP addresses (either IPv4 or IPv6).

@rdica @gene96817 Right now, I just have users specify the port number and run on other ports. I just know that in the past I've had a couple users have trouble understanding that. For some reason, they have a habit of connecting to jam.example.com when I tell them to connect to jam.example.com:portnum.

@rdica

you don't want them typing in six more characters

You would be amazed how lazy I can be sometimes.

Sounds like this may be solved much simpler using the custom directory server. I can't wait to get home and test that out. I'll let you know the outcome of my testing when I'm able to get around to it!

[QJKX]

Honestly, IP addresses and ports are software jank. Users shouldn't see them, or be expected to type them ever (naturally they should still work). They're to be expected for the maturity and resources of the project - I'm grateful to all the contributors who have kindly built and shared what we have.

I like that this proposal creates a better way, but doesn't cause disruption.

Please consider the security implications of this change, both from malicious domain name strings, and malicious SRV responses.
Potentially the code could be called by a URL handler (that doesn't exist yet - #494).
We wouldn't want your SRV code to be exploitable through a hyperlink.

[pljones]

No one is expected to see IP numbers or port ever. Use a custom directory, set it up on the default port and use its name.

I've seen no explanation in clear, user-comprehensible terms, as to what SRV records add over the current solution, so I'd consider this issue closed.

[ginjo]

The OP's question and SRV solution is the only one that would make sense for my situation. I host a number of private Jamulus servers for a number of customers. I use AWS's Global Accelerator to allow Jamulus clients the best latency possible from anywhere in the world. The Global Accelerator provides a single anycast IP address for my entire infrastructure, and it routes the Jamulus traffic internally based on the external port number. So one IP address with many port numbers get routed to my Jamulus servers on my private AWS VPN.

The Jamulus servers are stock linux installs, listening on the standard Jamulus port with a private-only IP address. The Jamulus servers are completely unaware of the sophisticated routing happening in the AWS cloud. And the Jamulus servers have no interaction with domain names or a Central Registry Server.

This all works really well, except that I have to give my customers ugly connection URLS 3.43.56.78:54223. Even if I give customers a proper domain, it's still ugly jamulus.customer.com:36457.

If Jamulus CLIENT could resolve SRV records, this issue would be solved for me. No changes to Jamulus Server, no need for central registry servers, no need to waste IP address, and no need to change a singly byte of code in Jamulus Server.

[pljones]

Who would host the SRV records for the public servers?

How would servers with dynamic IP get their records updated quickly?

[QJKX]

Who would host the SRV records for the public servers?

Whoever hosts their DNS.

For casual use there are "Dynamic DNS" providers that support SRV records. eg afraid.org: https://freedns.afraid.org/signup/features/

How would servers with dynamic IP get their records updated quickly?

Use a low TTL on the DNS record, this is implicit when using a DDNS provider and just works.

DDNS records are typically updated with a scripted HTTP GET. Linux distos already have well established scripts, and most consumer routers do too.
https://openwrt.org/docs/guide-user/services/ddns/client

[pljones]

For most server users this is more set up overhead than they're going to want to get into. Currently, they just install and connect to a directory. Nothing else is needed, no account registration or anything.

For IPv6, I'd suggest you just take out a block of addresses, allocate each of your operators a sub-domainname of their choice and resolve that to a specific internal server. No worrying about port numbers as you can use the IPv6 address and the default port for each server. Multiple addresses can resolve to the same physical box with port mapping allowing multiple servers on that box.

Where IPv6 isn't available, yes, it's more of an issue currently.

[hoffie]

FWIW, I've hacked up a quick SRV parsing PoC here: hoffie@client-srv-support (required refactoring: hoffie@client-srv-support~1)
I've set up localhost-1234.srv.jamulus.hoffie.info and localhost-5678.srv.jamulus.hoffie.info for testing.

Connecting to either of those with the updated client will actually connect to localhost:1234 or localhost:5678 respectively.

$ host -t SRV _jamulus._udp.localhost-5678.srv.jamulus.hoffie.info 
_jamulus._udp.localhost-5678.srv.jamulus.hoffie.info has SRV record 0 0 5678 localhost.

$ host -t SRV _jamulus._udp.localhost-1234.srv.jamulus.hoffie.info 
_jamulus._udp.localhost-1234.srv.jamulus.hoffie.info has SRV record 0 0 1234 localhost.

Not sure how or when to continue though, considering that there is no consensus for this feature yet.

[pljones]

I've nothing against it other than not seeing that it brings significant benefit. If the added code is small and makes significant improvements to the client code, it may be worth having.

[hoffie]

other than not seeing that it brings significant benefit

I'm not sure how to quantify the benefit.

The feature is obviously not necessary for:

• Users of public directories
• Users of Jamulus servers which live on their own IP
• Users of private directories
• Users who are not confused by technical stuff such as port numbers

This feature certainly benefits admins and users of multiple Jamulus instances on a single IP (which might be used by different groups). In other words, it's similar a similar use case to the virtual host support in web servers.
The main selling point is usability. It will take some time to be effective though, as it only helps once a group of users can expect all of them to have the at least the first Jamulus version with this feature (maybe 3.10.0).

There's definitely demand, but it might be a niche.

If the added code is small and makes significant improvements to the client code, it may be worth having.

I've now refactored it according to your comments. The code link above is still valid.

[pljones]

Actually :)
master...hoffie:jamulus:client-srv-support

auto records = dns->serviceRecords();
auto record = records.first();

One or other of those must have a data type, otherwise I can't review the code... (I'm not a compiler, I don't read header files...)

dns->setName ( QString ( "_jamulus._udp.%1" ).arg ( strAddress ) );

So I'd add these to my drealm.info. DNS entries:

_jamulus._udp.in-house1.drealm.info. UDP 3600 IN SRV 10 5 56554 jamulusserver.drealm.info
_jamulus._udp.in-house2.drealm.info. UDP 3600 IN SRV 10 5 56555 jamulusserver.drealm.info
_jamulus._udp.in-house3.drealm.info. UDP 3600 IN SRV 10 5 56556 jamulusserver.drealm.info

(where I think the .drealm.info. after _jamulus._udp.in-houseX is redundant?), so long as jamulusserver.drealm.info is an A entry -- and then I tell people to connect to in-house1.drealm.info or whatever and that's easier than jamulusserver.drealm.info:56554? (Though they don't have to be on the same domain - so long as the SRV points to an A.)

Given that the target in the entry must be an A record, does the response return IP details directly, rather than the target name string? If so, the call to ParseNetworkAddressName in ParseNetworkAddressSrv should be redundant (or does that just take the "IP address as a string" and turn it into the parsed value?).

[QJKX]

Given that the target in the entry must be an A record, does the response return IP details directly, rather than the target name string?

No, but...

https://www.rfc-editor.org/rfc/rfc2782.html#page-4

[DNS Server] Implementors are urged, but not required, to return the address record(s) in the Additional Data section.

The qt docs talk about "records associated with this lookup" https://doc.qt.io/qt-5/qdnslookup.html#details
so presumably records in "additional data" sections of the response just get parsed in to the hostAddressRecords list of the dns object.
https://doc.qt.io/qt-5/qdnslookup.html#hostAddressRecords

Another question is the RFC says setting the RR target (hostname) to '.' indicates the resource explicitly doesn't exist. So to follow the RFC Jamulus should abort and not fall back to the non-SRV procedure.

[hoffie]

One or other of those must have a data type, otherwise I can't review the code... (I'm not a compiler, I don't read header files...)

Done.

(where I think the .drealm.info. after _jamulus._udp.in-houseX is redundant?)

I think it should be an FQDN (although DNS server zone information might auto-complete it?).

then I tell people to connect to in-house1.drealm.info or whatever and that's easier than jamulusserver.drealm.info:56554?

Yep.

Given that the target in the entry must be an A record, does the response return IP details directly, rather than the target name string?

It seems like one is only supposed to return names via this mechanism, although I don't see a reason why IPs would not work. I guess I wouldn't add any enforcement code for that. It's more about DNS correctness, IMO.

If so, the call to ParseNetworkAddressName in ParseNetworkAddressSrv should be redundant (or does that just take the "IP address as a string" and turn it into the parsed value?).

ParseNetworkAddressName performs a real DNS lookup on the name.

Another question is the RFC says setting the RR target (hostname) to '.' indicates the resource explicitly doesn't exist. So to follow the RFC Jamulus should abort and not fall back to the non-SRV procedure.

I've implemented that now. I don't see a way to fail explicitly from the place where the logic has to live. Therefore, the behavior now matches the behavior for other unsuccessful connection attempts.

I'll open this as a PR now to make further reviews simpler and provide have autobuilds at hand.

[ginjo]

Thanks everyone for the consideration and code workup. I know this is somewhat of a niche feature, but I think it's valuable in offsetting the relatively narrow reach of a single Jamulus server. Unlike the more common internet protocols (http, etc.) that serve thousands of users at once, Jamulus serves a relatively few users for a long time. So the slope of server requirement vs user demand is rather steep. This feature will help admins host a number of servers while keeping it "clean" from the user POV, similar to putting a bunch of http servers behind a common load balancer.

If someone posts a Mac binary of the above workup (my Mac build environment is not currently functional), I'd be happy to try it out with my own servers and DNS setup.

[ginjo]

I experimented with @hoffie 's PR, and it works exactly as I was hoping it would. This allows me to host a number of Jamulus servers behind a single-IP gateway that maps external ports to the internal machines' port 22124, AND I can give my clients friendly connection strings like client1.something.com, jazzjam.customer.com, secretstring.whatever.com.

I'm using AWS to deploy ec2 instances and route53 CNAME and SRV records on-the-fly, so I can bring up a fresh server on-demand and have clients start jamming in minutes. Of course, I could do all this without the SRV records, but the experience is so much nicer with the ports hidden from view.

Thanks again, I'll see what I can do for the documentation (user docs + server/admin docs). Any guidance or suggestions welcome.

[ann0see]

Any guidance or suggestions welcome.

Just open a pull request on the website repo to the next-release branch. You can edit the .md files in the wiki folder directly.

[ginjo]

Great, thanks!

[ginjo]

Ok, I've added a section for DNS SRV record support near the bottom of the Running-a-Server.md page. Feel free to give a quick look and feedback, before I issue a PR. Or I can just issue the PR.

I didn't create any documentation for the user side, as I don't feel there's anything the user needs to know about this feature. Kinda the whole point, and beauty, of it 😀

[hoffie]

@ginjo Thanks! Looks good at first sight, just go ahead and open the PR, it'll be easier to comment on individual parts there anyway. I think it makes sense to add a notice that this feature is only supported by Jamulus 3.10.0 onwards.

I didn't create any documentation for the user side, as I don't feel there's anything the user needs to know about this feature.

Yep.

[ginjo]

I've been experimenting with the SRV support, and it's working great with custom server addresses in the Server Address field of the Connect window.

I'm also trying to use this feature in conjunction with a Directory server, and I can't figure out how to pass the FQDN to the Directory server from an instance of Jamulus server. I haven't used Directory servers much, so maybe I'm missing something. How do we list a Jamulus server on a Directory server based on FQDN instead of an IP address?

[pljones]

You don't. The Directory looks at the source IP address and uses that. The --serverpublicip (resolved to an IP and then sent to the directory), if set, will be used to switch between LAN and WAN responses (also note that --port only sets the local LAN port - if your router maps that to a different source port, that's what the Directory will use; there's no equivalent of --serverpublicip (yet) for port, IIRC).

[ginjo]

Ah, ok. So if the Directory server is reachable by a Jamulus server via WAN, it should work with whatever public port Jamulus is reachable at. And if the Directory and Jamulus server are on the same Lan, we can use --serverpublicip but have to use 22124 as the public port (or at least serve Jamulus on the same port that is reachable publicly).

Would be nice to eventually have an option to pass --serverpublicaddress <FQDN[:port]> to the Directory server, but now I'm just being greedy 😁 . Meanwhile, super happy with the SRV support!

[pljones]

Yes, --serverpublicip should have been --serverpublicaddress from the start...