New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example usage #25

Closed
mikesun opened this Issue Dec 5, 2016 · 8 comments

Comments

Projects
None yet
2 participants
@mikesun
Copy link

mikesun commented Dec 5, 2016

I'm trying to understand how to use the library, but have found the myself confused by the example client and server in tests/.

In the loop in tests/client.cpp (similar one in tests/server.cpp), the client establishes a connection to the server and continuously they then continuously exchange packets of type TEST_PACKET_CONNECTION. No user packets are exchanged however (type=TEST_USER_PACKET).

    while ( !quit )
    {
        client.SendPackets();

        clientTransport.WritePackets();

        clientTransport.ReadPackets();

        client.ReceivePackets();

        client.CheckForTimeOut();

        if ( client.IsDisconnected() )
            break;

        time += deltaTime;

        client.AdvanceTime( time );

        clientTransport.AdvanceTime( time );

        if ( client.ConnectionFailed() )
            break;

        platform_sleep( deltaTime );
    }

Conceptually, what functionality does the client perform compared to the transport? What does client.SendPackets()do exactly as compared toclientTransport.WritePackets()?

How can user packets (type=TEST_USER_PACKET) be exchanged once the connection is setup?

Also, my understanding is that various user packet formats are defined in shared.h and a factory for these types is generated with YOJIMBO_PACKET_FACTORY_START/STOP macro block?

It'd be great to have some kind of HOWTO on using the library!

@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 6, 2016

Hi Mike, thanks for your feedback. I agree, the library needs usage documentation badly. I've been working on some doxygen documentation for the past few weeks that should help as a reference, and after that I'd like to create some usage documentation that is much more high level, a getting started, concepts and so on. Basically pretty much what you are looking for.

Until then, here is the quick gist.

The core concepts are:

  1. Client
  2. Server
  3. Transport
  4. Connection
  5. Channels
  6. Packets
  7. Messages

The client and server are protocol level. They create the abstraction of the server with n slots for clients to connect to, and make unreliable UDP look like a connection based protocol by implementing a connection negotiation state machine and timeouts.

The transport is the actual, on the wire sending and receiving packets. The client and server work on top of the transport, and don't actually serialize (read or write) packets directly, they just create those packet objects, and add them to a queue to be sent later, vice-versa for receive packets are read in from the network and added to a queue as packet objects, so they can be read off that queue later.

The connection is the thing that is generating the connection packets you are seeing. Internally each client has one connection (to the server), and the server has one connection per-client. This is the layer that implements acks and reliability for messages. Connections may be configured to have multiple channels. Each channel has its own send and receive queue for messages, and may have different reliability and ordering guarantees. Currently, there are reliable-ordered channels and unreliable-unordered channels.

Reliable ordered channels are good for sending control messages and RPCs, as well as large blocks of data (larger than your max packet size) that get split up into fragments for you and reassembled automatically on the other side. This is good for stuff like, sending down the initial state of a level when a client joins, or the client sending up user profile data to the server on connect and so on.

Unreliable unordered channels are basically UDP, so messages are unreliable and may arrive out of order. Large blocks can still be sent over this channel, but in this case, there is no fragmentation support, so you have to configure your max packet size to be larger that the largest block you want to send. I'm planning on adding transport level packet fragmentation and reassembly shortly, so you have a way to send large blocks (say 2-4k) over this channel, while avoiding sending packets over the wire that are larger than MTU (~1500 bytes).

So your main choice right now is, do you extend the protocol by adding your own message types, or do you extend it by adding user packets? My recommendation is to go with messages, because that's the most fully featured path, and user packets are just meant for really low-level reasons when you really, really want your own specific packet types vs. messages that get aggregated into the connection packet that is generated each time you call Client|Server::SendPackets. You should be able to see client/server examples in test.cpp that send and receive messages, and for the user packet path look at test_client_server_user_packets example in test.cpp

Finally, the client and server are designed to be overridden to use, so take what is in shared.h and copy that to your own code and cut down, take the packet and message factories as well and cut and paste and create your own set of messages for stuff you want, and packets (if you want that), the implementation in shared.h is mostly there to support the tests in test.cpp and the testbed programs I use like client/server/secure_client/secure_server, but you should be able to quickly cut them down to do just what you need.

Transport is meant to be used as is, there is no need to subclass it. Also, the macros around the packet factory and message factory, are really just wrapping up a subclass of the PacketFactory and MessageFactory classes, to create factories customized to the set of packets and messages that you define.

cheers

  • Glenn
@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 6, 2016

Also, to answer your question specifically about the difference between the client/server and transport behavior, the Server::SendPackets, Client::SendPackets is really just generating the packets and queuing them on the transport. They don't actually get written to the network until Transport::WritePackets is called.

Similarly, Server::ReceivePackets, Client::ReceivePackets don't actually read packets from the network, they just pop packets off the transport receive queue, so you want to call Transport::ReadPackets (which does), before that, to make sure you have the most recent packets in the receive queue first.

The core reasoning behind this is that by putting the send and receive of packets behind queues, it's possible to implement multithreaded transports that work off main thread in the future, without changing how any of the client/server code works.

cheers

  • Glenn
@mikesun

This comment has been minimized.

Copy link

mikesun commented Dec 6, 2016

Thanks Glenn, your response is super helpful and clarifies quite a bit for me.

I took a closer look at test.cpp::test_client_server_messages(). The call to the PumpClientServerUpdate is basically equivalent to this block in client.cpp , yes?

        client.SendPackets();

        clientTransport.WritePackets();

        clientTransport.ReadPackets();

        client.ReceivePackets();

        client.CheckForTimeOut();

        if ( client.IsDisconnected() )
            break;

        time += deltaTime;

        client.AdvanceTime( time );

        clientTransport.AdvanceTime( time );

My understanding is that the client/server implementation is responsible for regularly calling SendPackets(),WritePackets() and ReadPackets(),ReceivePackets --- generating and reading the "wave" of packets on which messages/acks are piggybacked. Is that correct?

Given that the implementation is currently not multi-threaded, does that mean that any send/receive message functions must be called from within the same loop that generates the packet wave (as opposed to another thread apart from the one running the packet wave loop)?

Also, it seems network IO performed by WritePackets()/ReadPackets() is non-blocking, but not async (no use of something like libevent or select/epoll/kqueue). Is that choice made because: it's assumed socket reads will occur continuously, essentially polling at the application-level; and that transport level acks provide for guarantee of transmission?

Thanks again for the quick and informative response!

@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 7, 2016

Yes, the library is designed around common patterns used in dedicated server-based games like FPS. These games are simulation bound, not I/O bound, and completely stateful, so async IO doesn't make a lot of sense.

The basic pattern for these game is frame based:

  1. start of game frame on server
  2. read packets from network
  3. process packets from client (eg. receive input packets), and process them (potentially, advancing players forward with their inputs as received)
  4. step forward world simulation one frame (non-player objects like AIs, scripted objects, physically simulated objects).
  5. contruct packets to send out to clients that efficiently encode state of current world at time t (e.g. delta encode snapshot relative to last snapshot received by each client).
  6. flush packets to network
  7. Sleep until next, 10HZ, 20HZ, 30HZ or 60HZ server frame (repeat from 1)

As you can see, it's quite a different pattern compared with async io type approaches, where it's all about, I got a packet, I have to do a bunch of work on it, then respond with something. That's not what this library is really for, it's for the frame based approach described above.

Eventually, once I multithread the library you'll be able to create packets/messages on multiple threads and queue them up, but these queues won't flush to network until the next write packets call on transport. If you are looking for something that receives a packet and responds as quickly as possible with another packet, I don't think this library is really what you are looking for.

cheers

@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 7, 2016

My understanding is that the client/server implementation is responsible for regularly calling
SendPackets(),WritePackets() and ReadPackets(),ReceivePackets --- generating and reading the
"wave" of packets on which messages/acks are piggybacked. Is that correct?

Yes. You have to pump this no matter what, so the basic loop is going to look pretty similar for all examples. I mentioned that test because if you follow the code, you'll find the bits of code that send and receive user packets.

cheers

@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 7, 2016

Given that the implementation is currently not multi-threaded, does that mean that any
send/receive message functions must be called from within the same loop that generates the
packet wave (as opposed to another thread apart from the one running the packet wave loop)?

Yes.

@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 7, 2016

Also, it seems network IO performed by WritePackets()/ReadPackets() is non-blocking, but not
async (no use of something like libevent or select/epoll/kqueue). Is that choice made because:
it's assumed socket reads will occur continuously, essentially polling at the application-level;
and that transport level acks provide for guarantee of transmission?

It's because the application this library is designed for is frame based, so it's typical for you to want to pump all packets received since last frame, process them, do stuff, send packets out, sleep until next frame.

Contrast with an async loop where its like, setup a bunch of worker threads, sleep until packets come in, holy shit a packet has come in do some work on some thread, maybe reply with a packet. Rinse and repeat for each packet that comes in, hopefully distributing work nicely across all cores.

So basically, frame based vs. packet based is the reason.

cheers

  • Glenn
@gafferongames

This comment has been minimized.

Copy link
Member

gafferongames commented Dec 7, 2016

I think you have the information you need, so I'm closing this issue. Feel free to keep asking questions in here if you have more though.

cheers

  • Glenn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment