New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate LIVE/socket networking. #64

Closed
benvanik opened this Issue Jan 10, 2014 · 44 comments

Comments

Projects
None yet
10 participants
@benvanik
Copy link
Collaborator

benvanik commented Jan 10, 2014

This is a long way out, but it could be interesting to look at what's required and how complex the APIs are for LIVE and the network (above and beyond the Winsock functions). Allowing this to be stubbed out could enable networked play over a virtual LIVE service (some nodejs server performing auth/discovery/etc).

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 9, 2014

You can count with me.

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 9, 2014

Me too, I don't how to start, my idea is to use some wireshark and conect the xbox to my MacBook and analyze the network, got some experience on node.js. Count with me too :)

@sgtfrankieboy

This comment has been minimized.

Copy link

sgtfrankieboy commented Mar 9, 2014

The only possible way to have Xbox LIVE is to connect to the real service. There is absolutely no reason to have Xbox LIVE support if the game servers also have to be rewritten.

All the traffic from Xbox LIVE goes through https. Using WireShark wouldn't be too much of a help since everything is encrypted.

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 9, 2014

sad news :(

@gibbed

This comment has been minimized.

Copy link
Member

gibbed commented Mar 9, 2014

There is zero intent to have Xenia ever connect to real Xbox LIVE servers, and won't happen. The opened issue is about simulating the real service, not connecting to it.

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 9, 2014

I understod that. Right then.

@sgtfrankieboy

This comment has been minimized.

Copy link

sgtfrankieboy commented Mar 9, 2014

I know that. Even if tried, everybody connecting to it will get an Insta-Ban.

But how will the virtual Xbox LIVE service be useful when servers for the games still have to emulated? I only see P2P games working, since they require minimum efforts compared to other games.

@darkengine-io

This comment has been minimized.

Copy link

darkengine-io commented Mar 9, 2014

One solution might be several community-supported servers that connect to a central coordinating server. The "main" server would be simple enough to run on a cheap VPS, while the community servers can crop up around games that become popular on the platform.

As for decrypting the communication, there are two likely possibilities: each game has its own set of certificates, or the certificates are part of the Xbox main system. In the case of the former, once we get the emulated game working, we can feed it certificates for a server of our own, and receive those communications in the clear. In the case of the latter, we'd have to run a version of the Xbox system, modified with out own certs, either on the Xbox itself or emulated in some way. Either way, once we can send and receive HTTPS to and from the software, we can start reversing the protocol they're using.

@gibbed

This comment has been minimized.

Copy link
Member

gibbed commented Mar 9, 2014

There is no need for any of that, as the networking happens as part of the OS, not the libraries baked into a game, it'd be up to us to decide how to implement it entirely.

@darkengine-io

This comment has been minimized.

Copy link

darkengine-io commented Mar 9, 2014

Ah I see, I was assuming that the game handled interaction with LIVE on a lower level. Based on your comment, am I correct in assuming that the game talks to LIVE through high-level system calls, and never actually opens a socket to Live directly? That is, the OS provides functions that handle user accounts, passing multiplayer data to and from the game, etc?

@gibbed

This comment has been minimized.

Copy link
Member

gibbed commented Mar 9, 2014

Yes.

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 10, 2014

To do that make own pair of public and private keys, wireshark - long wiresharking and decrypting m$ packets and creating own server which will do the same as m$ servers. "Vy kokoti!!! Ak niekto z vas rozumie slovensky tak sorry :D :D" Also we can redirect all traffic directly to m$ but it will ne suspicous that everyone will have the same keyvaults...

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 10, 2014

So we should wait to networking libraries for developing the LIVE servers right?

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 10, 2014

why wait what means wait, its our thing to implement them not to wait...
I already tried to fix xam_net on my fork... Didnt tried it if it works.
Can i run devkit-compiled xex or i have to convert?
As for decrypting xbl packets, there is m$ public key integrated in cpu. I have xbox but unfortunately it is (most likely) xenon mobo so no rgh...

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 10, 2014

@alexperezpaya Yeah. Although if we have a decent idea of how networking will be implemented here, we can start on the LIVE servers.

Classical Chicken and Egg problem.

@darkengine-io

This comment has been minimized.

Copy link

darkengine-io commented Mar 10, 2014

Well in that case, perhaps the best option would be to dig through the networking symbols and start building a specification of what needs to be implemented on the server and client sides. As long as the protocol is documented, development can begin on coding the server and re-implementing the client simultaneously.

Though it might be ideal to get games booting and accessing those APIs before any significant work is put into their implementation.

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 10, 2014

If we can do the server in C, and esp. if we can write it for POSIX systems (OSX, Linux, *nix), I'm down to help out as much as I can with writing the LIVE servers.

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 10, 2014

Actually, is there anyone else who would also like to work on LIVE servers in C for POSIX platforms?

@XePeleato

This comment has been minimized.

Copy link

XePeleato commented Mar 10, 2014

I have a VPS if anyone wants to try something on it

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 11, 2014

Aww hell yeah @XePeleato now we're talking.

I think we could run a master-slave system with all of the slaves running in P2P. All of the slaves would be servers likely with lower bandwidths, also likely behind NAT or something, while the master would be a public facing server. If we support multiple masters, they would be running in P2P too, albeit with signed keys generated by us on our end.

To that end, I have a server that's running 24/7 behind NAT at my place, so I could perhaps configure port forwarding and be good to go.

@benvanik Perhaps we ought to make a GH org, w/ your permission and anyone coming up with an umbrella name I can do that.

Anyone else in?

Pretty much any of the following skills are useful:

  • Distributed Systems
  • P2P
  • C
  • Networking (esp. low bandwidth low latency)
  • *nix skills
  • service skills (init.d, etc.)
  • Reverse Engineering
  • Distributed Algorithms

And the following resources are useful too:

  • Public facing servers
  • Publicly accessibly secure persistent storage (for things like keys, passwords, etc)
  • Slave Servers (should be able to run 24/7, okay if behind NAT as long as you can configure port forwarding)

And one more thing: any have an idea for an umbrella name for the collective projects and a name for the server?

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 11, 2014

Omg writing network in C will be painful for me, but I will try to help as much as possible

@0xdeafcafe

This comment has been minimized.

Copy link
Contributor

0xdeafcafe commented Mar 11, 2014

Or write something quick in node.js, and not spend more time on the server than on the emulator.

@perezpaya

This comment has been minimized.

Copy link

perezpaya commented Mar 11, 2014

@Xerax i see that option ;) it would be easier to develop and to get maintainers

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 11, 2014

On the other hand, performance is gained by writing it in low level code.
Besides, I feel like there are probably a lot of people, like me, who are
better at server side things.

I mean, I could almost never hope to contribute to the emulator, but I can
definitely do things server side.

Writing it in a lower level language also makes it easily extendable in a
multitude of languages.

Lastly, I was thinking of having a separate project for this; that way
neither will impede each other's progress.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

Yeah i can help also i have windows azure 115euro every month until some 2016 so hosting will be no problem, please send me mail if you want linux, windows server or whatever. Please insert in Subject [XENIA], thanks.I think c++ will be best choice for doing the server. I would also use windows server.
We can do everything with the 115euro for this project. I will also help with coding server... Just making github repo...

@XePeleato

This comment has been minimized.

Copy link

XePeleato commented Mar 11, 2014

Xlink Kai can be a good example for our Emulated XBL

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

It isnt XBL but just system link...

@XePeleato

This comment has been minimized.

Copy link

XePeleato commented Mar 11, 2014

Yes but it is similar

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

yes for mp, but xbl does profile management and more then just xlink.

@XePeleato

This comment has been minimized.

Copy link

XePeleato commented Mar 11, 2014

Thats true, my fault

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 11, 2014

@Jan4V could we have a cooler name like

aixen

So we could shorten it to AXN or something?

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

ok :D AXN what does it stand for? (what does it mean)

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 11, 2014

AiXeN

On Tuesday, March 11, 2014, Jano Varga notifications@github.com wrote:

ok :D AXN what does it stand for?

Reply to this email directly or view it on GitHubhttps://github.com//issues/64#issuecomment-37328915
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

And what is AiXeN :D :D ? ok renaming

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 11, 2014

It's Xenia backwards.

On Tuesday, March 11, 2014, Jano Varga notifications@github.com wrote:

ok :D AXN what does it stand for?

Reply to this email directly or view it on GitHubhttps://github.com//issues/64#issuecomment-37328915
.

NOTICE: This e-mail (and any attachments) may contain PRIVILEGED OR
CONFIDENTIAL information and is intended only for the use of the specific
individual(s) to whom it is addressed. It may contain information that is
privileged and confidential under state and federal law. This information
may be used or disclosed only in accordance with law, and you may be
subject to penalties under law for improper use or further disclosure of
the information in this e-mail and its attachments. If you have received
this e-mail in error, please immediately notify the person named above by
reply e-mail, and then delete the original e-mail. Thank you.

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

GREAT

@Jan4V

This comment has been minimized.

Copy link

Jan4V commented Mar 11, 2014

AiNeX is Xenia backwards but dont matter...

Also I made a VS project... Please dont convert to vs 12 or 13 thanks.

EDIT: Or convert for me behind me but dont commit it

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Mar 11, 2014

alright repo is up at: https://github.com/haneefmubarak/Aixen

The following have been given push access:

@Jan4V @alexperezpaya @benvanik @XePeleato

If anyone else would like to help, feel free to fork and send a pull req.

@Rob-pw

This comment has been minimized.

Copy link

Rob-pw commented Aug 24, 2014

Maybe it's worth registering Xenia and Aixen as part of a separate organisation?

As far as the server goes, as mentioned earlier, it might be worth just knocking up a simple service which maintains a connection to the Xenia client. Node.js would allow such a service to be implemented in days, rather than weeks, leaving more development time for the emulator.

A simple mock-up LIVE service as mentioned above probably wouldn't be too difficult to implement.

Obviously there's no need to implement networking based off of LIVE, since you can use which ever protocol you deem appropriate. However, for those interested in seeing how the Xbox 360 currently negotiates connections:

Just a thought, clearly sniffing packets over the network is too difficult as HTTPS is being used. What if you were to setup a live environment on a devkit/rgh, pointing to a server, then log every request made to that server by that IP. Would you not then see what kind of data is being requested, and what responses should be given?

@gibbed

This comment has been minimized.

Copy link
Member

gibbed commented Aug 24, 2014

There is absolutely no reason to do anything with the real LIVE service. The implementation details of LIVE networking are left to the OS/kernel.

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Aug 24, 2014

Scalability and stability, my friend. Anyways, the time for when Xenia will need a networking service is relatively far off so there's that.

Also, Aixen is intended to provide a platform abstracted networking service. This would work well with emulators given that we have to implement whatever functionality as part of the emulator anyways. Because of this, we are able to create our networking service however we want, using whatever protocol we want.

The plus side of doing so is that we can write and maintain one service that runs with Xenia, RPCS3 (the ps3 emulator) and possibly even newer platforms.


As for the organization part, I speak as lead for the Aixen project when I say that I want it to remain independent of any related projects to prevent bias. If we were to create an emulator foundation or something, with the big emulator projects joining, then maybe I could reconsider, but until then, the answer from the Aixen project is no.

Also, there's just a big hassle that comes up with formal organizations and I think it would counterproductive at this early a stage.

However, I can only speak for the Aixen project. As for Xenia itself, that is a question that @benvanik will have to answer.

@haneefmubarak

This comment has been minimized.

Copy link

haneefmubarak commented Aug 24, 2014

@gibbed ya beat me to part of it! :D

@Rob-pw

This comment has been minimized.

Copy link

Rob-pw commented Aug 24, 2014

I didn't mean that you should base Aixen off of the existing LIVE service, since you can implement whichever protocols are most applicable, I was suggesting an alternative to previous suggestions of sniffing packets.

Understood, it would appear that I didn't fully understand Aixen's mission.

Thanks both for the response.

@benvanik

This comment has been minimized.

Copy link
Collaborator

benvanik commented Feb 10, 2015

NetDll is implemented (as games use it for IPC). Closing for now.

@benvanik benvanik closed this Feb 10, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment