Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Confused about virtual interfaces #142

Closed
dark-swordsman opened this issue Feb 21, 2019 · 33 comments
Closed

Confused about virtual interfaces #142

dark-swordsman opened this issue Feb 21, 2019 · 33 comments

Comments

@dark-swordsman
Copy link

dark-swordsman commented Feb 21, 2019 •
edited
Loading

Are the interfaces in this file the "virtual interfaces" that are mentioned in step 5? https://github.com/bntjah/lancache/blob/master/interfaces

And are these interfaces where we need 18 open ip addresses on the subnet? I'm just a little confused why each platform needs it's own network interface.
@bntjah
Copy link
Owner

bntjah commented Feb 21, 2019 via email

@dark-swordsman
Copy link
Author

So I'm still missing the point of them. Like what devices/systems are using these virtual IPs and how?

Are the virtual IPs exposed to the network in such a way that, say a computer wants to download from steam, it would need to use the Steam virtual interface IP (say 10.1.0.120)? If this is the case, then assuming the actual lancache server is on 10.1.0.110 for it's physical IP, are the computers still only connecting to 10.1.0.110, 8.8.8.8, and 8.8.4.4 for DNS, or do we need to put the lancache IP, the 18 virtual IPs, and then google's IPs down for DNS?

@nexusofdoom
Copy link
Collaborator

This would be one to look at why it needs more then one IP address
lancachenet/lancache-dns#69

@dark-swordsman
Copy link
Author

dark-swordsman commented Feb 21, 2019 •
edited
Loading

So they're saying there that, for example, if 2-4 people on the network were to download from 10.1.0.110, it would probably download at max speed, but the moment you do 8, 10, 20 etc., each one progressively gets slower?

Therefore, if steam had, say, 10 IPs, then downloading to 20-40 PCs (obviously limited by harddisk read speeds), then it would work?

Sorry if I'm asking a dumb question but it's really unclear to me why each provider needs it's own IP. It's only for connections in the network (computers connecting to the lancache for downloads) and has nothing to do with the lancache downloading from the internet, right?

Edit: It's also unclear how this will be handled from the computers' POV. Are we adding all 20+ IP addresses to their DNS settings? How does that work?

@brgerig
Copy link
Collaborator

brgerig commented Feb 21, 2019 •
edited
Loading

It doesn't help with speed because all of the IPs are on the same network connection. It's because each provider has a slightly different configuration, and there needs to be some way to know what to use. Think of them as languages: say your battle.net client tried to talk to the virtual IP used by steam. Your client starts "talking battle.net," and lancache doesn't understand it, because it can only speak steam on that IP.

There is a way to have only a single IP address and have all of the providers share it, but you have to do custom stuff so that lancache knows which config to use for each request, and there's more that can go wrong.

Think of it like a call center. It's a lot easier to publish a phone number for English, and a phone number for Spanish, and a phone number for French. The alternative would be a single phone number, with a menu of "Press 1 for English," etc. (Not a perfect analogy, but hopefully it helps.)

@dark-swordsman
Copy link
Author

Okay, great, I understand that, but that still doesn't answer my question of: Do I need to add ALL of these IPs to the DNS settings of each PC? We can define them in the router and it'll automatically update, but how do we tell the computers, "Okay! Steam is on 10.1.0.120!"

I guess in a broader sense: I'm fine with assigning the IPs in the Interface, but is there any other configuration required?

@brgerig
Copy link
Collaborator

brgerig commented Feb 21, 2019

The only IP that you want to put into the DNS settings of each PC is the main lancache IP. (Or have DHCP hand it out.)

@dark-swordsman
Copy link
Author

The only IP that you want to put into the DNS settings of each PC is the main lancache IP. (Or have DHCP hand it out.)

So you're saying that when a computer hits the lancache IP, the lancache will automatically route the traffic to a different virtual interface, and that there's no other setup required. Yes or no?

@bntjah
Copy link
Owner

bntjah commented Feb 22, 2019 via email

@dark-swordsman
Copy link
Author

dark-swordsman commented Feb 23, 2019 •
edited
Loading

Sorry if I'm coming off rude, but you're not answering my question. I just want to make sure I know exactly what is going on before I go assign 18 random IPs on the network.

"...or manually set to route certain domains towards the correct lancache virtual interfaces..."

So by saying this, you're saying that I need to, either by DHCP or manually, set ALL the IPs to DHCP. So in other words, there will be:

  • lancache IP
  • 18 virtual IPs
  • google IP

So 20 IPs set in the PC's DNS settings.

Then you say, "...wich from there sent it to the right part of the cache. To either download it from cache or get it from the upstream and cache it at the same time..."

How does the computer, in this case, know where to request the right download from? brgerig said you only need to put in the lancache IP, but reading what you say makes me question that.

What I can only assume is the actual way it works is that when a PC hits the lancache IP, lancache will see what TLD the request is to and then somehow route the traffic to the correct virtual interface, which at that point, once the virtual interface receives that request, it will pipe the data to the PC. Is that right?

Edit: I also want to clarify, when you say computer, I assume you mean the gaming PC, NOT the lancache. The lancache I would say is the server.

@brgerig
Copy link
Collaborator

brgerig commented Feb 25, 2019

No, you only need the primary Lancache IP in the gaming PC's DNS. There isn't any IP routing going on, just DNS hijacking. Here's how it works:

Let's say you visit CNN on the PC. Computer needs to know what IP address to use for www.cnn.com, so it asks Lancache IP via DNS. Lancache looks at DNS config, doesn't have any info for www.cnn.com, so asks its upstream DNS (google) for the answer, then gives it to the PC. Then your PC happily goes and visits www.cnn.com on that address.

Now your PC wants to download a steam game. It asks Lancache IP what IP address to use for content2.steampowered.com. Lancache looks at its DNS config, and sees that content2.steampowered.com is in its special DNS config, so it tells the PC that domain name is at 10.1.0.120. Your PC then asks 10.1.0.120 for the files it wants to download from content2.steampowered.com. Lancache web server is listening there and proxying+caching any downloads that get requested.

Does that help clarify things?

@bntjah
Copy link
Owner

bntjah commented Feb 26, 2019 via email

@cyrianox
Copy link

cyrianox commented Feb 26, 2019 via email

@brgerig
Copy link
Collaborator

brgerig commented Feb 26, 2019

Happy to help!

@dark-swordsman
Copy link
Author

dark-swordsman commented Mar 24, 2019 •
edited
Loading

Hey again, sorry for coming off rude.

So I think I'm getting it. We only need to set 1 IP to our local network, and then using localhost (127.x.x.x) IPs on the actual server, it will act as a DNS server and route to the other localhost IPs?

In that case, what do I need to fill out for these entries?

# Regular host IP
auto lc-host-vint
    iface lc-host-vint inet static
    address lc-host-ip
    netmask lc-host-netmask
    gateway lc-host-gateway
    dns-nameservers 8.8.8.8 8.8.4.4

# IP used for Steam caching
auto lc-host-vint:1
    iface lc-host-vint:1 inet static
    address lc-host-steam
    netmask lc-host-netmask

# IP used for Riot caching
auto lc-host-vint:2
    iface lc-host-vint:2 inet static
    address lc-host-riot
    netmask lc-host-netmask

# and so on...

@dark-swordsman
Copy link
Author

Hey guys, back again.

I nuked the server and started from scratch. I was very deliberate and concise with the instructions and everything. It still isn't working.

It appears that having only our lancache @ 10.1.0.110 set as DNS for a PC, it is still able to download games, but it can't access the internet. This is expected, but it isn't actually sending traffic through the lancache to be cached. It looks like it's just hopping over it completely.

What do I do from here?

@dark-swordsman
Copy link
Author

dark-swordsman commented Apr 9, 2019 •
edited
Loading

Okay. so I was able to use this command and now I am at least able to redirect traffic using the lancache as a DNS server. I can get on this github, google, and download with steam:
./configure --with-http_slice_module --with-http_ssl_module --with-file-aio --with-threads --with-cc-opt="-m64 -march=native -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations" --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf

I got it from here: #89 (comment)

I guess in the install instructions, it doesn't set .../nginx/conf/nginx.conf as the default conf file. At some point, if I get this working, I will spend the time rewriting the install instructions so they are up to date and make sense.

So now we are making progress, but I am unable to still download through the lancache. I'll upload my configs shortly.

@dark-swordsman
Copy link
Author

Looks like there were some duplicate local-zones in unbound that prevented the localzones from mounting. So now my traffic is still redirected, and now steam is giving an error that it can't connect. So there's some progress. Now I guess it's something with the proxying/nginx?

@nexusofdoom
Copy link
Collaborator

nexusofdoom commented Apr 9, 2019 •
edited
Loading

if you can try this installer
https://github.com/nexusofdoom/lancache-installer
and let me know how it goes?

and if you still have some issues I may be able to use teamviewer to help out next week some time.

@dark-swordsman
Copy link
Author

dark-swordsman commented Apr 9, 2019 via email

@nexusofdoom
Copy link
Collaborator

nexusofdoom commented Apr 9, 2019 •
edited
Loading

I just redid my server this weekend and it worked with no issues.

Installer for the lancache by Nexusofdoom, Zero Ping Heroes and Geoffrey

Improve download speeds and reduce strain on your Internet connection at LAN parties. Locally cache game installs and updates from the largest distributors.

Ubuntu 18.04.1 with OpenSSH Server.
Download url for Ubuntu 18.04.1 Server
http://cdimage.ubuntu.com/releases/18.04.1/release/ubuntu-18.04.1-server-amd64.iso

You will need 18 avaliable IP's example 192.168.0.2 - 192.168.0.20 used for lancache
Clone the git repo
git clone -b master http://github.com/nexusofdoom/lancache-installer

cd lancache-installer

Run scripts with sudo
Run
sudo ./install-lancache.sh

##########################################################################

To access netdata open broswer and navigate to http://your-primary-ip:19999

##########################################################################

@dark-swordsman
Copy link
Author

Alright, I installed it with 18.04.1, but the IPs were what I wanted, so I modified the /etc/netplan/01-netcfg.yaml file (since it said that /etc/network/interfaces was deprecated), and it doesn't work.

What else do I need to modify to get this working.

@nexusofdoom
Copy link
Collaborator

Use your DHCP server to hand out the first IP you want to use to the MAC address of the network interface you are using. then when you run the installer it will auto setup the IP's starting from the first address your DHCP Server handed out. if you want to edit the IP's by hand you will need to edit more then the /etc/netplan/01-netcfg.yaml file.

@nexusofdoom
Copy link
Collaborator

example of the 01-netcfg.yaml file
network:
version: 2
renderer: networkd
ethernets:
eth0:
dhcp4: no
dhcp6: no
addresses: [10.0.4.30/23]
addresses: [10.0.4.40/23]
addresses: [10.0.4.39/23]
addresses: [10.0.4.33/23]
addresses: [10.0.4.34/23]
addresses: [10.0.4.38/23]
addresses: [10.0.4.41/23]
addresses: [10.0.4.37/23]
addresses: [10.0.4.42/23]
addresses: [10.0.4.35/23]
addresses: [10.0.4.31/23]
addresses: [10.0.4.32/23]
addresses: [10.0.4.43/23]
addresses: [10.0.4.44/23]
addresses: [10.0.4.36/23]
addresses: [10.0.4.45/23]
addresses: [10.0.4.46/23]
addresses: [10.0.4.47/23]
gateway4: 10.0.4.1
nameservers:
addresses: [8.8.8.8,4.2.2.2]

@nexusofdoom
Copy link
Collaborator

nexusofdoom commented Apr 9, 2019 •
edited
Loading

you would also need to edit the /etc/hosts file

127.0.0.1 localhost
127.0.1.1 lancache-server

127.1.0.1 lancache-origin-backend
127.1.0.2 lancache-blizzard-backend

10.0.4.30 lancache-eth

#The Following are Virtual IP's used by Lancache
10.0.4.40 lancache-steam
10.0.4.39 lancache-riot
10.0.4.33 lancache-blizzard
10.0.4.34 lancache-hirez
10.0.4.38 lancache-origin
10.0.4.41 lancache-sony
10.0.4.37 lancache-microsoft
10.0.4.42 lancache-enmasse
10.0.4.35 lancache-gog
10.0.4.31 lancache-arenanetworks
10.0.4.32 lancache-apple
10.0.4.43 lancache-wargaming
10.0.4.44 lancache-uplay
10.0.4.36 lancache-glyph
10.0.4.45 lancache-zenimax
10.0.4.46 lancache-digitalextremes
10.0.4.47 lancache-pearlabyss

@nexusofdoom
Copy link
Collaborator

you would also need to edit the unbound.conf also

@dark-swordsman
Copy link
Author

Cool. So I changed all three of those files. It is redirecting traffic like before, but now instead of (manifest missing) for steam, I get (no internet connection).

Here's the files:

01-netcfg.yaml

network:
 version: 2
 renderer: networkd
 ethernets:
   ens160:
     dhcp4: no
     dhcp6: no
     addresses: [10.1.0.110/20]
     addresses: [10.1.0.111/20]
     addresses: [10.1.0.112/20]
     addresses: [10.1.0.113/20]
     addresses: [10.1.0.114/20]
     addresses: [10.1.0.115/20]
     addresses: [10.1.0.116/20]
     addresses: [10.1.0.117/20]
     addresses: [10.1.0.118/20]
     addresses: [10.1.0.119/20]
     addresses: [10.1.0.120/20]
     addresses: [10.1.0.121/20]
     addresses: [10.1.0.122/20]
     addresses: [10.1.0.123/20]
     addresses: [10.1.0.124/20]
     addresses: [10.1.0.125/20]
     addresses: [10.1.0.126/20]
     addresses: [10.1.0.127/20]
     gateway4: 10.1.0.1
     nameservers:
       addresses: [8.8.8.8,4.2.2.2]

/etc/hosts

127.0.0.1       localhost
127.0.1.1       lhden-lancache

# The following lines are desirable for IPv6 capable hosts
# ::1     localhost ip6-localhost ip6-loopback
# ff02::1 ip6-allnodes
# ff02::2 ip6-allrouters

127.1.0.1       lancache-origin-backend
127.1.0.2       lancache-blizzard-backend

#This is the primary IP (of the network card / eth) that your Lancache is using
10.1.0.110 lancache-eth

#The Following are Virtual IP's used by Lancache
10.1.0.111 lancache-steam
10.1.0.112 lancache-riot
10.1.0.113 lancache-blizzard
10.1.0.114 lancache-hirez
10.1.0.115 lancache-origin
10.1.0.116 lancache-sony
10.1.0.117 lancache-microsoft
10.1.0.118 lancache-enmasse
10.1.0.119 lancache-gog
10.1.0.120 lancache-arenanetworks
10.1.0.121 lancache-apple
10.1.0.122 lancache-wargaming
10.1.0.123 lancache-uplay
10.1.0.124 lancache-glyph
10.1.0.125 lancache-zenimax
10.1.0.126 lancache-digitalextremes
10.1.0.127 lancache-pearlabyss

And then the unbound was updated accordingly.

Is there anything else I need to change?

@dark-swordsman
Copy link
Author

Ah my bad, forgot the vhosts. Just a moment

@nexusofdoom
Copy link
Collaborator

one thing on the windows client may have to run a admin command prompt and flush the dns "ipconfig /flushdns"

then you can also try and ping one of the lancache services from the windows client and see if it shows a reply from the local IPs you have setup.

@nexusofdoom
Copy link
Collaborator

should also see if all the services are running nginx, unbound, and sniproxy.

@dark-swordsman
Copy link
Author

Woohoo! I updated the vhosts and then the netdata.conf to the new 10.1.0.110 ip and it works!

Thanks @nexusofdoom.

Sorry if I was getting aggravated at some points. I just really needed this to work and it was a huge learning curve for me.

Is there any chance we can get a script to automatically update the IPs in the future?

@nexusofdoom
Copy link
Collaborator

I know at one point we did look at that.

@nexusofdoom
Copy link
Collaborator

Your welcome any time.
I will close this ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@bntjah @dark-swordsman @brgerig @nexusofdoom @cyrianox