Dev9ghzdrk: Add networking capability for Linux #2586
Conversation
|
The formatting is all over the place in some files between spaces and tabs, please keep it consistent, also there are many lines that shouldn't have been changed. |
|
@TheLastRar see anything you like |
|
I don't know specifically why you decided to get rid of bridged mode. While I never had luck with it I do recall that what worked varied user to user for some reason. A lot of the functional changes made seems to be specifically for removing all the code for bridged mode, Edit: I misread the PR, It was bridged mode that was removed not switched, I've amended the above. I don't know if WinPcap defines PCAP_NETMASK_UNKNOWN, someone would need to check that. XLink Kai will accept packets from the MAC address that the plugin already has, CLR_DEV9 uses the same prefix and changes just the last two bytes (on both WinPcap and tap). On linux testing I did in the past, I didn't need to run any specific commands to allow pcap to work, either they aren't needed for switched mode, or mono/wine runs them on install. |
|
Switched mode doesn't make much sense for this plugin; plugin only needs to provide Layer 2 processing and should let the PS2 CPU process Layer 3 natively at the OS and application layers. Performing layer 3 operations in the plugin is intrusive and may lead to unintended consequences. Looks like the code I removed was dealing with the complexity of having added switched mode in the first place. I take care of WinPCap absence of PCAP_NETMASK_UNKNOWN by simply defining it if it isn't already defined. I believe Mono runtime was granted access to network adapter promiscuous mode during installation, explaining why Pcap doesn't fail trying to open the devices. I'll address whitespace consistency. |
|
Hmm... First impression is that it could use some work. I tried to build it, throwing on the gtk 3 flag to make things interesting, noted that I needed dev versions of pcap and libxml2 installed, installed them,and built it a second time. Seems like a fair number of warnings on building, though a fair amount were similar and probably won't be too difficult to clean up. On going in the bin directory, I remembered that the inis folder that the *.ui file was being copied to doesn't exist until you run pcsx2 (maybe the plugin directory would work better?). I ran it and looked to see if the plugin was there anyways. Didn't show up in the list, but it was in the plugins directory. I ran the build again, and it didn't copy it the second time, either. I manually copied it from the source. Still not showing. I'm now doing a clean gtk 2 build, since I probably should've tried the normal version of gtk first. I do notice that while it found PCAP, it didn't find pcap_get_pfring_id, so there might also be a dependency issue it didn't catch... |
|
Regarding copying of ui and cfg files, the destination directories are created if not found. The INSTALL( CODE commands in CMakelists.txt are conditioned on the OS being reported as Linux. This is the desired behavior, but I've noticed the condition is not a reliable indicator of the OS actually being Linux. On some machines, the condition evaluates false when it should evaluate true. Can someone take a look and suggest a reliable solution? To understand correctly- "Didn't show up in the list" means PCSX2 doesn't recognize the library even though it's in the plugins directory? |
|
Ah, I had gotten confused with the relation between switch/bridged and promiscuous/non-promiscuous I don't know the reason why there is an option for non-promiscuous mode, given that it looks like it would be rather disruptive to unrelated network processes (possibly why I never got it to work, although it seemed to work for others). I can only assume it was for compatibility with drivers that didn't work with promiscuous mode, given the plugin also supports using tap this probably is less of a concern. The PS2 will read the mac address from eeprom, if you set that then you should be able to avoid replacing the mac address for every packet in and out. I note that you removed the code for re-sending broadcast packets to the host machine, was this causing duplicate responses for you? You've included a memory card file, assuming this contains a network config save, I was under the impression that network configs are tied to the BIOS of the system they where created on, is this not the case? |
|
The IP address, netmask, gateway are configured by the Network Disc DVD that shipped with the network adapter hardware. Or could be configured within certain games like SOCOM. Layer 3 addresses. However, layer 2 MAC addresses were presumably tied to what the network adapter reported to the BIOS. No need to re-send broadcast packets because the host machine gets them anyway. Memory card contains network config but this will be overwritten by users trying to connect to XLink. |
Yes. It isn't in the list, and prints that the file is not a PCSX2 plugin on the console. I will note that the CMake files are not used in windows, so any tests would mostly be if it was Linux, OS X, BSD, etc. I modified the CMakelists.txt file to print out the variable you were checking, and it was listed as Linux for me, though. However, looking closer, it's copying to ~/.config/pcsx2/inis, and I'm actually running directly from bin, with the inis and memcard folders directly in there. I also didn't note that it had a cfg file and memcard that needed to be copied in. Copying them in directly doesn't seem to have helped with it not being a valid plugin, though... |
Packets sent via winpcap are treated as outgoing by the OS. The code for duplicating the packet is presumably to allow the broadcast packet to bounce back so the host treats it as incoming, since a router might not send the broadcast packet back to its source. I tried to test this by connecting a PC running PCSX2 to a laptop, but neither the broadcasts packets or the duplicated to-host packets got returned, so it seems to have failed to achieve what it was attempting to do, The results may differ if you are using an actual router, but based on my results, I'm not going to argue against its removal. Xkink Kai won't care as it uses pcap, which can capture both incoming and outgoing packets.
I wasn't able to use the memory card file you provided (it showed up as 7MB, but did not show up in the BIOS browser menu after it was inserted),
Correct, this information is retrieved from eeprom and was updated in pcap_io_init(), however, you seem to have removed that code. |
Pushed changes to address this, please confirm that you can see the plugin and configure it to point to bridge: adapter from the dialog's Ethernet drop-down.
I have removed the conditions in my latest upload.
I did development against v1.3.1, where ~/.config/pcsx2 seemed to be the right path to place config files. Nowadays it seems the config directory is ~/PCSX2 so I've pushed changes to reflect this. Seems to work OK now.
Not entirely sure why this is the case. Then again, the memory card file is a convenience that probably doesn't serve anyone except me. Might be worth removing altogether, as long as someone can verify that they can setup network onto their own memcards
I would like to restore that logic. Will revisit down the road |
|
Can someone please help move this forward? |
|
It's also possible for pcsx2 to be running in portable mode, in which case the ini and plugin folders and such are directly in the bin directory. In cmake, there's a variable named "PLUGINS_DIRECTORY" that points to the plugins directory. I didn't see one of the ini directory, but presumably it's in the same folder as the PLUGINS directory. I'll also note that if you happen to do sudo sh build.sh instead of sh build.sh, the way you've got it written, no one will be able to run pcsx2 except root. You probably shouldn't be doing that, but it could happen... It did create the two directories in question and seemed to read out of them, showed up in the list, and had all the network devices available to configure, and let me pick out my ethernet port, pcap bridge: enp3s0. On then trying to run an iso, it gave me the following: Though, oddly, which the first error came up every time, the second crash didn't always happen. Just to note, I'm not sure if I actually have any games that use the network side of things. I'm just trying to make sure it compiles and doesn't crash. --arcum42 |
What is your environment? Are you cross-compiling in Windows with Cygwin? Have you tried this from a pure Linux environment? It looks like CMake linked against WinPcap instead of libpcap0.8
This error message is a bit cryptic. Do you only get it if you open the plugin configuration dialog? I don't get it when I run locally, so I need a way to reproduce it. |
|
It's a completely pure Linux environment, actually. It's the most recent 32 bit copy of Mint Cinnamon (so no cross compiling whatsoever, including the usual 64 bit to 32 bit cross compiling), installed on a computer without any other operating system on it, and it does, in fact, have libpcap-0.8 and libpcap-0.8-dev installed. The second error message happened moments after the first, and was what it printed when pcsx2 crashed. |
|
I'll add my test findings The plugin will crash is the configuration file is not found, when trying to enter the plugin's config menu (showing a popup that can't be closed), or when starting emulation (just crashes), Ideally we should be creating a new config file in such a situation. The config file is loaded from It fails to open the selected adapter for me, giving the same error message. On windows the plugin will show a popup informing you if it can't open the selected adapter, presumably your linux port is trying to do the same thing and runs into issues. |
plugins/dev9ghzdrk/pcap_io.cpp
Outdated
| )) == NULL) | ||
| { | ||
| fprintf(stderr, errbuf); | ||
| fprintf(stderr,"\nUnable to open the adapter. %s is not supported by WinPcap\n", adapter); |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
|
Thanks, I thought that message was being printed by the library itself- I'll fix the misleading WinPCap reference. I think you're right about the "failure to load adapter" error dialog producing the GLib-CRITICAL message, will look into that. I'll also use DEV9setSettingsDir()... hadn't noticed it back in 2014 when I actually did most of this so I set it to what seemed to be the default at the time. Could you guys please run "sudo getcap path_of_PCSX2_binary" to confirm that the CMake install set the right networking capabilities? If those cqpabilities are set I don't expect the message (socket: Operation not permitted) |
|
Shows nothing for me when I run it on the plugin file. When building I don't see a message that states "Enabling networking capability on Linux..." but I do see the message about copying the MC and ui files. Also, something I forgot to mention in my last post, The config panel still lists bridged mode adapters, if you are removing bridged mode you should also remove these entries. |
|
Agreed, the config panel should not list switched or loopback. I'll add the
filter.
I'm not sure why you don't see "Enabling networking capability on Linux..."
during the install, but it would certainly explain the error opening the
adapter.
…On Tue, Sep 18, 2018, 3:36 PM TheLastRar ***@***.***> wrote:
Shows nothing for me when I run it on the plugin file.
When building I don't see a message that states "Enabling networking
capability on Linux..." but I do see the message about copying the MC and
ui files.
Also, something I forgot to mention in my last post, The config panel
still lists bridged mode adapters, if you are removing bridged mode you
should also remove these entries.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#2586 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ANDUOnmpms0Lx12w0S5fii-yIpb2b828ks5ucUtSgaJpZM4WggqQ>
.
|
|
I've pushed changes:
Could you please try now? Thanks for continued feedback |
|
np. I figure I might not be actually using the plugin, and I'm not sure how far I can test it, but I can at least try to get it to the point where it installs and doesn't crash or give me error messages. ^_^ This time I basically wiped the directory and redownloaded from git. It copied the .ui file into the bin directory. The .cfg file didn't seem like it copied anywhere, and since it crashes without it, I had to copy that myself. Having the plugin write that file if it isn't there instead of crashing would be nice, since it looks pretty minimal. (I'm also curious if that ui file works in gtk3, but I just compiled using gtk2 this time.) On the good side, while I had to futz around copying things to get it not to crash, this time running an iso (Dark Cloud) didn't crash (and I opened it 2-3 times), and presumably it initialised correctly. I saw the following (ignore the ALSA messages). So we definitely have some progress, anyways. --arcum42 |
|
I had meant to add a check for existence of the cfg file before trying to load it. I've added the check now, so that if there is an existing config it's loaded. Otherwise it gets created soon after anyway. |
|
I thought we where keeping switch and removing bridge? The plugin currently looks for the .ui file in the working directory The error message is now miss-leading on windows. For some reason, the plugin crashes when used with the PCSX2 Daily Builds PPA (note, I've manually run the set networking caps so that shouldn't be the issue), When using a build of PCSX2 created from your branch the plugin works fine, unsure what is going on there. The plugin doesn't seem to be able to read the settings it has saved, closing and reopening the emulator appears to reset the settings. If I open, close, then reopen the dev9 config panel, I get duplicate Expansion and PC Card entries in the Type dropdown box I'm able to connect Xlink Kai, I was also able to connect to both the RE:Outbreak resurrection server or the .Hack//fragment Netslum server (after some difficulty), so it seems to be working correctly. I've also tested this on windows. I was aiming to check if the filtering works on the windows side, however, I ran into an issue specific to your branch, It builds correctly but will crash PCSX2 when it searches for plugins. I'm not entirely sure what change you did that is causing the issue. Attaching a debugger on a debug build gives me the following message
From here; |
I can probably re-use the cfg directory to know where to read UI at runtime but that info isn't available at compile time, when installing the file, is it? I could copy to that directory upon first launch but it's hacky. I am trying my best to integrate with the project but could use a hand with details like variables specifying directory structures. Don't want to waste time with the guess-and-check approach. Please feel free to commit if you see easily fixable directory-related issues that will work across build environments. I will just make the message say pcap instead of WinPCap or libpcap. Another easy fix. The failure to load on Windows must be due to the extern "C" EXPORT that I placed in front of the functions that need to be visible from the main executable. I added an #ifdef to mimic the normal Windows mechanism or Linux mechanism. I'll need to fix that for Windows case. Apparently there's a need to change calling convention. Or maybe I just need to use _Win32 on line 55 of PS2Edefs.h. Or, remove lines 57 and 58 since I just realized there's a .def file I know nothing about that Daily Build PPA or how it modifies the build environment. Why does the PPA not build the way a developer does? The results should be identical, except perhaps recent commits from master that have not been pulled into the branch. It would be a freak coincidence if the plugin API changed since this pull request was created, but it's possible. Is failure to save plugin cfg related to file permissions? Do you know if the cfg file is being written? What do the contents look like when you see doubles? |
|
I'm not sure what my "build" environment would have to be. And that commit didn't seem to fix the problem. |
|
@mirh I meant you're using Visual Studio? As opposed to MinGW or Cygwin or WSL. I don't understand where adapter named "eth0" could be coming from in a Windows environment. The only thing I can think of is if you are building in Cygwin or WSL. Do you see where the eth0 is being sourced from? Are you using the latest version of WinPCap from winpcap.org/install? Have you been using a Tap adapter? If not, does the name of your network adapter begin with T? |
|
The other user that was getting it was using one of the precompiled Windows builds from the build bot. Not building on their own. It therefore would have been with VS 2015. |
|
Yes, vs2015. |
|
Can you answer the questions above? I cannot recreate the error on my end so I need information to do anything about it. Were you using the networking plugin before? Are you even trying to use networking at all? Have you tried opening the dev9 plugin settings, picking your adapter, and clicking OK? |
|
No at all. |
|
So apparently you are not using the networking feature but seeing that error when you launch the game. That wasn't made clear before. I know it sounds like you're saying something simple but the details matter. I'll look at the code but without more information about what you have installed or configured I can only help so much. Is any dev able to see the same issue? What do I need to do to reproduce it? |
|
Well when opening the plugin selector all dlls in the ini are loaded at least once (that’s why we get those ‘some kinda plugin error’ for higher gsdx instruction sets when the supported set is lower. I assume during load it is just checked what kind of plugin this dll generates to put it into the appropriate section. So you could look into the plugin loader from pcsx2 core to check what happens upon loading all plugins from the plugins folder. However dll loading is a nasty thing (and I am not sure how it behaves in the unmanaged world) but typically all linked assemblies and their dependencies are loaded as well (even if they are not explicitly referenced during the load process). So your code might not even be the issue but it could be. And the issue could basically occur everywhere. So I guess there is no easy way around than asking mirh to attach a debugger. You could possibly run dependency walker on the dll to check whether that one shows some linking errors. |
|
|
@mirh So after a bit of digging, it turns out @TheLastRar added the logic to copy "eth0" into config.eth by default in commit dcd477f , which explains the mystery of how on Windows you get a message about a Linux adapter name. It looks like the changes to Win32/Config.ccp that he made circa 2015 were attempting to use INI files for storing settings instead of the registry. I know that before that commit 5f068ec, this plugin was working without error on Windows. I know this because I started developing the Linux version before then. I was considering just backing out Win32/Config.cpp to commit 86895cb to get it working again ASAP. Instead I've just made it the Windows equivalent of Linux/Config.cpp. See whatever commit is currently mentioned at #2619 , which should remove the unwelcome pop-up box you get even if not trying to use networking |
|
Yes, fixed. Thank you. |
|
I believe that eth0 was the default setting before, although I can't remember at this point. While annoying, you can dismiss the popup and emulation should continue, was it crashing for you? @mirh That being said, making it not be pop up if you've specificity disable Ethernet emulation is a good idea. The plugin will only be able to detect your network devices if the winpcap service is running (by default it should be), you may also need to restart for winpcap pickup any changes you made to them (i.e. installing drivers, enabling/disabling adapters). That might be what your issue was. |
|
No it was just a popup. And there's no problem now for people that don't care. |
|
The only thing I would do for that case is pop up "have you enabled NetGroup Packet Filter Driver" or something similar in the plugin configuration dialog when there are no devices detected |
|
I mean.. it shouldn't be rocket science to distinguish between "driver on, no devices" and "driver off, end" |
Follow up from #2586 which ensures that MAC address is unique on Xlink Kai when using Windows. Previous build was returning last two bytes as 00:00 rather than last two bytes of host adapter's MAC.
lightningterror
changed the title
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@lowlyocean @TheLastRar @arcum42 Sorry to say but we need both switch and bridge mode to get games properly working online. From my tests LAN mode games work properly only with bridge mode while games with actual Online mode work with Switch mode. I request you to please fix this and bring back option to choose between bridge and switch mode. Also CLR Dev9 isn't compatible enough yet to completely replace dev9ghz so this definitely needs to be fixed. |
|
Care to elaborate what issues LAN games encountered with switched mode that bridged mode resolves? Regarding CLR DEV9, I assume you are referring to the forum post you made yesterday (I don't check the thread as often as I should) I think I know the cause of that and should have a fix soon(tm) |
Does this mean there are games that have original servers still running? I was under the impression that most PS2 games are played "online" nowadays are really just using LAN mode over a VPN. Can someone explain why Switched mode is actually needed? When using bridged mode, the PS2 BIOS thinks your actual PC's network hardware is its own. It just calls it by a different MAC address than your PC does. Everything that happens at Layer 2 on your actual PC's hardware can be seen by the PS2 BIOS if you use bridged mode. This allows Link Layer broadcasting that's used for discovery in LAN. The BIOS/game performs any Layer 3 / IP configuration natively. If you use switched mode, you introduce another level of Layer 3 management by an external tool (the plugin), you lose ability to handle MAC broadcasts for LAN, and it's not clear what benefit you can get. |
|
@lowlyocean There are private servers running for most of the games now and online play has been revived. @TheLastRar The main issue happens when trying to join a game. The games can be discovered/created but you are unable to join. Believe me I have thoroughly tested this, both modes are needed. Also if someone is trying to fix clrdev9 then try lan mode in Splinter Cell games. |
Would be helpful if you can provide details of the testing you've done [like exact steps you take, then the difference in behavior at the final step for A vs B] because it's still not clear why switched mode is needed. |
|
Tested this again today using Socom 2 game lan mode. The created game is never visible using the new plugin (which has only one option after this commit). It works perfectly fine if using bridged mode in old plugin (the plugin had both bridge and switch mode). The old bridge mode works perfectly with all lan games while old switch mode works perfectly with online games. I guess this either needs a revert or a proper fix. |
|
This plugin uses bridged mode. Switched mode is considered obsolete and was removed to simplify the code. Anything that doesn't work for you in the new plugin would be a bug and we'd need evidence in the form of annotated packet captures to fix. Bridged mode was tested with Socom 2 LAN in the new plugin over XLink. Are you using XLink to host/join LAN games? Again, packet captures are worth a thousand words here |
|
No, I am not using Xlink. I am using real lan network at my home. Also used virtual lan network through hamachi. Infact even used two pcsx2 running on same pc. They all work with old plugin bridge mode. They also work with clr plugin. As I said earlier this plugin has problem in LAN mode. I have been playing PCSX2 online/lan since the first day this plugin was created and I know what I am talking about. Even @TheLastRar was confused earlier whether you deleted bridge mode or switch mode. I think some code of bridge mode has definitely been compromised here. |
|
|
The existing dev9ghzdrk plugin was designed to run on Windows only. I've implemented the Linux equivalent. This includes configuration dialog in GTK+, configuration storage and retrieval using LibXML2, and network adapter enumeration using LibPCAP.
I've moved some files from the Win32 sub-directory up one level and made slight capitalization changes to their #include statements so that a Linux build can also locate the headers.
I've updated the appropriate CMake files to conditionally include the plugin on Linux if necessary libraries are detected. I've also embedded post-install commands to copy necessary UI layout and device configuration files to user's .config directory, as well as to grant necessary privileges to PCSX2 executable. The exact commands are printed to install_log.txt toward completion of the build.
The networking code has been slightly modified; the MAC address of the virtual PS2 network adapter contains real Sony MAC address prefix followed by last two bytes of real adapter's MAC address. This allows detection of virtual PS2 console by the XLink Kai LAN-over-internet service while also allowing each user to appear unique on that service.