Gamecontroller can Bind to a Different Port Based on Command Line Argument #3207

Mr-Anyone · 2024-05-21T14:47:41Z

Description

Game controller can now bind on a different port

Testing Done

Ran the following commands:

./tbots.py test kickoff_play_test --gamecontroller_port 50000 --enable_thunderscope
./tbots.py run thunderscope --gamecontroller_port 50001

and when each of the command from above is running, I ran

ps aux | grep gamecontroller

Resolved Issues

resolves #3160

Length Justification and Key Files to Review

N/A

Review Checklist

It is the reviewers responsibility to also make sure every item here has been covered

Function & Class comments: All function definitions (usually in the .h file) should have a javadoc style comment at the start of them. For examples, see the functions defined in thunderbots/software/geom. Similarly, all classes should have an associated Javadoc comment explaining the purpose of the class.
Remove all commented out code
Remove extra print statements: for example, those just used for testing
Resolve all TODO's: All TODO (or similar) statements should either be completed or associated with a github issue

nimazareian

I just tried this, and I'm not sure if it works as expected. Trying to launch 2+ concurrent AI vs AIs causes a crash due to a port binding error.

In the example you've listed in your PR description, since one of the targets you're running is running with bazel test with the "exclusive" tag added to its build file, the test actually ends up finishing before thunderscope is launched. A longtime issue we've had is actually #2619 which is similar to the problem I've described.

This is somewhat of a different, but very related, problem to #3160. Above mainly discusses an issue where we all of our simulated pytests need to have the "exclusive" tag to avoid having multiple of them running concurrently and the gamecontroller port being busy (by default Bazel runs tests in parallel). Fixing this issue will likely help speed up our CI/CD workflow for simulated pytests.

On the other hand, #3160 discusses the problem we faced during Robocup where most teams connect their computer to the same network which has access to the internet. Furthermore, during a game or field testing, all teams on the field connect to the same LAN. Our goal is to basically almost always launch the gamecontroller on some non-official port/multicast IP address that other teams or other members of our team aren't listening to for gamecontroller commands. This will allow us to send gamecontroller commands that only effects the fullsystem being ran locally. This should be the case UNLESS we're playing an actual game where we're not launching our own gamecontroller locally and we're using the provided field gamecontroller. Hence #3161 discussing a new "robocup" mode used during actual games with a non-local gc. To achieve this, we will probably need to update RobotCommunication aswell to not always listen to the official gamecontroller/referee port/IP:

Software/src/software/thunderscope/robot_communication.py

Lines 114 to 119 in aaa16cb

    
           self.receive_ssl_referee_proto = tbots_cpp.SSLRefereeProtoListener( 
        
               SSL_REFEREE_ADDRESS, 
        
               SSL_REFEREE_PORT, 
        
               lambda data: self.current_proto_unix_io.send_proto(Referee, data), 
        
               True, 
        
           )

Let me know if this doesn't make sense or you want any clarifications. Tagging @itsarune in case I've missed anything.

On a separate note, could you also update this socket connection to have a more descriptive error message:

Software/src/software/networking/ssl_proto_communication.py

Lines 37 to 39 in 1528173

    
           # bind to all local interfaces, TCP 
        
           self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
        
           self.socket.connect(("", port))

For example:

        try:
            # bind to all local interfaces, TCP
            self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self.socket.connect(("", port))
        except ConnectionRefusedError:
            raise ConnectionRefusedError(
                f"SSL Socket connection refused on port {port}. Is binary already running in a separate process?"
            )

src/software/thunderscope/binary_context_managers/game_controller.py

…_Software into gamecontroller_port

itsarune · 2024-05-29T05:14:01Z

src/software/thunderscope/binary_context_managers/game_controller.py

+        self,
+        supress_logs: bool = False,
+        gamecontroller_port: int = None,
+        referee_addresse: int = None,


Suggested change

referee_addresse: int = None,

referee_address: int = None,

bump: looks like it didn't get committed

itsarune · 2024-05-29T05:16:32Z

@nimazareian covered everything, I don't have anything more to add onto nima's conversation

itsarune · 2024-05-29T05:54:57Z

src/software/thunderscope/binary_context_managers/game_controller.py

+        self.REFEREE_IP = referee_addresse
+        if referee_addresse is None:
+            self.REFEREE_IP = "224.5.23.1"
+


REFEREE_IP is supposed to be a constant, we shouldn't be modifying it inside here

On the other hand, #3160 discusses the problem we faced during Robocup where most teams connect their computer to the same network which has access to the internet. Furthermore, during a game or field testing, all teams on the field connect to the same LAN. Our goal is to basically almost always launch the gamecontroller on some non-official port/multicast IP address that other teams or other members of our team aren't listening to for gamecontroller commands. This will allow us to send gamecontroller commands that only effects the fullsystem being ran locally. This should be the case UNLESS we're playing an actual game where we're not launching our own gamecontroller locally and we're using the provided field gamecontroller. Hence #3161 discussing a new "robocup" mode used during actual games with a non-local gc. To achieve this, we will probably need to update RobotCommunication aswell to not always listen to the official gamecontroller/referee port/IP

Doesn't this contradicts what nima said? Am I misunderstanding something? Are we setting the port instead then but keeping the multicast address constant?

Nima's right but in the implementation, don't modify REFEREE_IP. That is a constant (which is why it's all capitalized) we use during real games for connecting to the Gamecontroller and we should leave it so that it always point to that ip address. By convention, we never modify constants during runtime.

Instead, use a new local variable:

Suggested change

self.REFEREE_IP = referee_addresse

if referee_addresse is None:

self.REFEREE_IP = "224.5.23.1"

multicast_address = referee_address if referee_address else Gamecontroller.REFEREE_IP

Shouldn't this be self.multicast_address instead of multicast_address, since we are accessing this variable in line 180?

Yeah, you're correct

src/software/thunderscope/thunderscope_main.py

src/software/field_tests/field_test_fixture.py

itsarune

good start! left a few comments to try to clean up the logic a bit

itsarune · 2024-06-08T05:21:54Z

src/software/thunderscope/thunderscope_main.py

-            supress_logs=(not args.verbose)
+            supress_logs=(not args.verbose),
+            gamecontroller_port=args.gamecontroller_port,
+            referee_addresse=args.sslreferee_multicast_address,


Suggested change

referee_addresse=args.sslreferee_multicast_address,

referee_address=args.sslreferee_multicast_address,

itsarune

Reviewing this code, I think we should reconsider this ticket and talk about this more.

We have two goals for this ticket:

During robocup, we want to be able to launch the gamecontroller locally and run our AI without listening to messages on the SSL_REFEREE_PORT
If we run field tests at Robocup, we want to our local gamecontroller

I don't think we need to be messing with the referee address here with the changes you've got. I think the approach is:

Add a new option to launch the gamecontroller in thunderscope_main if --run_blue is called (maybe --launch_gc)
Add a new getter for the self.referee_port in gamecontroller
Use the getter to pass in the referee_port so that the SSLRefereeProtoListener will bind to that port in the RobotCommunication module
We should replicate this logic in FieldTestFixture

This means that we don't need args for changing the SSL referee address nor the SSL gamecontroller port and leave all the existing logic alone

Mr-Anyone · 2024-06-11T10:22:41Z

Test New Changes

I am not sure if I quite understand. You want something like the following?

./tbots.py run thunderscope  --use_unconventional_port --not_launch_gc
./tbots.py test pivot_kick_field_test --keyboard_estop --use_unconventional_port

Currently, there is a bug in the toolbar. Before I fix that, I want to make sure I have correctly implemented the requested change.

nimazareian · 2024-06-13T18:52:22Z

Test New Changes

I am not sure if I quite understand. You want something like the following?
./tbots.py run thunderscope  --use_unconventional_port --not_launch_gc
./tbots.py test pivot_kick_field_test --keyboard_estop --use_unconventional_port 
Currently, there is a bug in the toolbar. Before I fix that, I want to make sure I have correctly implemented the requested change.

@itsarune can pitch in as well, but I think we should always used an "unconventional port" for the GC, unless we're running --run_blue/--run_yellow without --launch_gc (i.e. in every scenario we should use an unconventional port, unless we're in an actual real life game, hence the --run_blue/--run_yellow without the --launch_gc since we're just using the GC that is used on the field by both teams. So there shouldn't be a need for the arg --use_unconventional_port and only --launch_gc should be added. This new arg could be optionally paired with --run_blue/--run_yellow to run our own game controller (both in Thunderscope and in the field test fixture).

For your reference, right now when we play actual games, I often run Thunderscope with something like:
./tbots.py run thunderscope --interface wlp2s0 --run_blue
In the updated logic, running that command should NOT launch the game controller, and should listen on the official GC port.

nimazareian · 2024-06-13T18:55:28Z

src/software/thunderscope/thunderscope_main.py

@@ -317,6 +339,7 @@
            estop_mode=estop_mode,
            estop_path=estop_path,
            enable_radio=args.enable_radio,
+            sslvision_address=args.sslvision_multicast_address,
        ) as robot_communication:


As part of the --run_blue/--run_yellow paths here, we should optionally launch the GC context manager if --launch_gc is passed. Here's an example of how a context manager could be launched optionally:

Software/src/software/thunderscope/thunderscope_main.py

Lines 432 to 444 in 501df36

) as gamecontroller, (

# Here we only initialize autoref if the --enable_autoref flag is requested.

# To avoid nested Python withs, the autoref is initialized as None when this flag doesn't exist.

# All calls to autoref should be guarded with args.enable_autoref

TigersAutoref(

ci_mode=True,

gc=gamecontroller,

supress_logs=(not args.verbose),

tick_rate_ms=DEFAULT_SIMULATOR_TICK_RATE_MILLISECONDS_PER_TICK,

show_gui=args.show_autoref_gui,

)

if args.enable_autoref

else contextlib.nullcontext()

itsarune · 2024-06-15T06:19:48Z

src/software/field_tests/field_test_fixture.py

+    parser.add_argument(
+        "--sslvision_multicast_address",
+        type=str,
+        default=SSL_VISION_ADDRESS,
+        help="the multicast address for ssl vision",
+    )
+


we don't need this arg, we can default to SSL_VISION_ADDRESS and just change the port

itsarune · 2024-06-15T06:21:33Z

src/software/field_tests/field_test_fixture.py

+    parser.add_argument(
+        "--use_unconventional_port",
+        action="store_true",
+        default=False,
+        help="setting this option would cause gamecontroller to bind on an unconventional port, likely 12393",
+    )


we don't need this as @nimazareian suggested. If we run --launch_gc, then we can set the appropriate port in RobotCommunication

itsarune · 2024-06-15T06:23:19Z

src/software/field_tests/field_test_fixture.py

+    parser.add_argument(
+        "--not_launch_gc",
+        action="store_true",
+        default=False,
+        help="whether we are launching gamecontroller or not",
+    )


let's do the inverse --launch_gc

itsarune · 2024-06-15T06:25:29Z

src/software/field_tests/field_test_fixture.py

        current_proto_unix_io=friendly_proto_unix_io,
        multicast_channel=getRobotMulticastChannel(args.channel),
        interface=args.interface,
        estop_mode=estop_mode,
        estop_path=estop_path,
        enable_radio=args.enable_radio,
+        sslvision_address=args.sslvision_multicast_address,
+        referee_port=gamecontroller.get_referee_port(),


yup this is exactly right, but we don't need sslvision_address

itsarune · 2024-06-15T06:26:39Z

src/software/thunderscope/binary_context_managers/game_controller.py

+    def __init__(
+        self,
+        supress_logs: bool = False,
+        use_unconventional_port: bool = False,


we don't need use_conventional_port

itsarune · 2024-06-15T06:27:00Z

src/software/thunderscope/binary_context_managers/game_controller.py

+        self,
+        supress_logs: bool = False,
+        use_unconventional_port: bool = False,
+        not_launch_gc: bool = True,


this shouldn't be passed here, this should be used for conditional initialization in thunderscope_main

itsarune · 2024-06-15T06:29:49Z

src/software/thunderscope/binary_context_managers/game_controller.py

+            # the intention of setting this port is to ensure that we don't listen other people's
+            # gamecontroller. It is highly unlikely that other people would use port 12393 since
+            # this is a random number that was set by us!
+            self.referee_port = self.next_free_port(start_port=12393)


can you make this a random int above 1024 and below 65535? If multiple people are running thunderscope_main, we can conflict with ourselves

itsarune · 2024-06-15T06:30:22Z

src/software/thunderscope/binary_context_managers/game_controller.py

+        if use_unconventional_port:
+            # the intention of setting this port is to ensure that we don't listen other people's
+            # gamecontroller. It is highly unlikely that other people would use port 12393 since
+            # this is a random number that was set by us!
+            self.referee_port = self.next_free_port(start_port=12393)


you can keep the old code and not have use_unconventional_port

itsarune · 2024-06-15T06:31:34Z

src/software/thunderscope/binary_context_managers/game_controller.py

@@ -150,7 +193,7 @@ def __send_referee_command(data: Referee) -> None:
                autoref_proto_unix_io.send_proto(Referee, data)

        self.receive_referee_command = tbots_cpp.SSLRefereeProtoListener(
-            Gamecontroller.REFEREE_IP, self.referee_port, __send_referee_command, True,
+            self.REFEREE_IP, self.referee_port, __send_referee_command, True,


you can keep the old code

itsarune · 2024-06-15T06:32:19Z

src/software/thunderscope/robot_communication.py

@@ -27,6 +27,8 @@ def __init__(
        estop_path: os.PathLike = None,
        estop_baudrate: int = 115200,
        enable_radio: bool = False,
+        sslvision_address: str = None,


we don't need this param

itsarune · 2024-06-15T06:32:20Z

src/software/thunderscope/robot_communication.py

+        # setting up the ssl vision and referee addresses
+        self.ssl_vision_address = sslvision_address
+        if self.ssl_vision_address is None:
+            self.ssl_vision_address = SSL_VISION_ADDRESS


we don't need this code

itsarune · 2024-06-15T06:32:34Z

src/software/thunderscope/robot_communication.py

@@ -105,7 +114,7 @@ def setup_for_fullsystem(self) -> None:
        Sets up a listener for SSL vision and referee data, and connects all robots to fullsystem as default
        """
        self.receive_ssl_wrapper = tbots_cpp.SSLWrapperPacketProtoListener(
-            SSL_VISION_ADDRESS,
+            self.ssl_vision_address,


keep the old code

itsarune · 2024-06-15T06:33:11Z

src/software/thunderscope/robot_communication.py

@@ -105,7 +114,7 @@ def setup_for_fullsystem(self) -> None:
        Sets up a listener for SSL vision and referee data, and connects all robots to fullsystem as default
        """
        self.receive_ssl_wrapper = tbots_cpp.SSLWrapperPacketProtoListener(
-            SSL_VISION_ADDRESS,
+            self.ssl_vision_address,
            SSL_VISION_PORT,
            lambda data: self.__forward_to_proto_unix_io(SSL_WrapperPacket, data),
            True,


line 125 should be self.referee_port if it isn't None or SSL_REFEREE_PORT

itsarune · 2024-06-15T06:33:28Z

src/software/thunderscope/thunderscope_main.py

+    parser.add_argument(
+        "--sslvision_multicast_address",
+        type=str,
+        default=SSL_VISION_ADDRESS,
+        help="the multicast address for ssl vision",
+    )
+
+    parser.add_argument(
+        "--use_unconventional_port",
+        action="store_true",
+        default=False,
+        help="setting this option would cause gamecontroller to bind on an unconventional port, likely 12393",
+    )
+
+    parser.add_argument(
+        "--not_launch_gc",
+        action="store_true",
+        default=False,
+        help="whether we are launching gamecontroller or not",
+    )
+


we don't need these args

itsarune · 2024-06-15T06:34:27Z

src/software/thunderscope/thunderscope_main.py

@@ -317,6 +339,7 @@
            estop_mode=estop_mode,
            estop_path=estop_path,
            enable_radio=args.enable_radio,
+            sslvision_address=args.sslvision_multicast_address,


if args.launch_gc exists, then we should have a gamecontroller.get_referee_port() call somewhere here

itsarune · 2024-06-15T06:36:13Z

src/software/thunderscope/thunderscope_main.py

+            supress_logs=(not args.verbose),
+            use_unconventional_port=args.use_unconventional_port,
+            not_launch_gc=args.not_launch_gc,


we can keep the old code. In this mode (blue + yellow AIs + gamecontroller + simulator), we always expect to run the gamecontroller to play out AI against itself

In the other mode --run_blue or --run_yellow, we run one of the fullsystems without the Simulator or Gamecontroller (sometimes). This is the mode we use in our actual robocup games because vision and gamecontroller run on a separate referee machine

itsarune

good start! Still needs a bit more work

Mr-Anyone added 5 commits May 20, 2024 18:43

chnage port depending on argument

bacfacf

added docs

d620120

remove stuff

1557448

updated entry points

5478f15

fixed stuff

8d394cd

Mr-Anyone requested review from nimazareian and itsarune as code owners May 21, 2024 14:47

[pre-commit.ci lite] apply automatic fixes

0b5d2a6

nimazareian reviewed May 22, 2024

View reviewed changes

src/software/thunderscope/binary_context_managers/game_controller.py Outdated Show resolved Hide resolved

src/software/thunderscope/binary_context_managers/game_controller.py Outdated Show resolved Hide resolved

nimazareian requested review from PhilD71, williamckha and sauravbanna May 22, 2024 20:12

Mr-Anyone added 3 commits May 22, 2024 18:27

copied code from nima

5d5e130

added some new port to fix

53b3120

change bind address and stuff like that

c052a97

Mr-Anyone force-pushed the gamecontroller_port branch from db5f7e0 to c052a97 Compare May 23, 2024 21:09

Mr-Anyone and others added 5 commits May 23, 2024 17:18

added new stuff to robot communication

ace67d6

[pre-commit.ci lite] apply automatic fixes

730753a

added new stuff for robot communication

9b8d59d

Merge branch 'gamecontroller_port' of github.com:Mr-Anyone/Thunderbot…

9453411

…_Software into gamecontroller_port

[pre-commit.ci lite] apply automatic fixes

ebcc8d6

itsarune reviewed May 29, 2024

View reviewed changes

src/software/thunderscope/thunderscope_main.py Outdated Show resolved Hide resolved

itsarune reviewed May 29, 2024

View reviewed changes

src/software/thunderscope/thunderscope_main.py Outdated Show resolved Hide resolved

itsarune reviewed May 29, 2024

View reviewed changes

src/software/field_tests/field_test_fixture.py Outdated Show resolved Hide resolved

itsarune reviewed May 29, 2024

View reviewed changes

src/software/field_tests/field_test_fixture.py Outdated Show resolved Hide resolved

itsarune reviewed May 29, 2024

View reviewed changes

merged arune's changes

6ad2b47

Mr-Anyone force-pushed the gamecontroller_port branch from 0ab6bc4 to 6ad2b47 Compare May 31, 2024 12:57

itsarune reviewed Jun 8, 2024

View reviewed changes

itsarune requested changes Jun 8, 2024

View reviewed changes

added new feature that arune requested

e730860

[pre-commit.ci lite] apply automatic fixes

60dad81

Mr-Anyone requested review from nimazareian and itsarune June 11, 2024 12:44

nimazareian reviewed Jun 13, 2024

View reviewed changes

itsarune reviewed Jun 15, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gamecontroller can Bind to a Different Port Based on Command Line Argument #3207

Gamecontroller can Bind to a Different Port Based on Command Line Argument #3207

Mr-Anyone commented May 21, 2024

nimazareian left a comment

itsarune May 29, 2024

Mr-Anyone May 31, 2024

itsarune Jun 8, 2024

itsarune commented May 29, 2024

itsarune May 29, 2024

Mr-Anyone May 31, 2024 •

edited

itsarune Jun 1, 2024

Mr-Anyone Jun 1, 2024 •

edited

itsarune Jun 1, 2024

itsarune left a comment •

edited

itsarune Jun 8, 2024

itsarune left a comment •

edited

Mr-Anyone commented Jun 11, 2024

nimazareian commented Jun 13, 2024

Test New Changes

nimazareian Jun 13, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune Jun 15, 2024

itsarune left a comment

	self.receive_ssl_referee_proto = tbots_cpp.SSLRefereeProtoListener(
	SSL_REFEREE_ADDRESS,
	SSL_REFEREE_PORT,
	lambda data: self.current_proto_unix_io.send_proto(Referee, data),
	True,
	)

	# bind to all local interfaces, TCP
	self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	self.socket.connect(("", port))

	referee_addresse=args.sslreferee_multicast_address,
	referee_address=args.sslreferee_multicast_address,

	) as gamecontroller, (
	# Here we only initialize autoref if the --enable_autoref flag is requested.
	# To avoid nested Python withs, the autoref is initialized as None when this flag doesn't exist.
	# All calls to autoref should be guarded with args.enable_autoref
	TigersAutoref(
	ci_mode=True,
	gc=gamecontroller,
	supress_logs=(not args.verbose),
	tick_rate_ms=DEFAULT_SIMULATOR_TICK_RATE_MILLISECONDS_PER_TICK,
	show_gui=args.show_autoref_gui,
	)
	if args.enable_autoref
	else contextlib.nullcontext()

Gamecontroller can Bind to a Different Port Based on Command Line Argument #3207

Are you sure you want to change the base?

Gamecontroller can Bind to a Different Port Based on Command Line Argument #3207

Conversation

Mr-Anyone commented May 21, 2024

Description

Testing Done

Resolved Issues

Length Justification and Key Files to Review

Review Checklist

nimazareian left a comment

Choose a reason for hiding this comment

Let me know if this doesn't make sense or you want any clarifications. Tagging @itsarune in case I've missed anything.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsarune commented May 29, 2024

Choose a reason for hiding this comment

Mr-Anyone May 31, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mr-Anyone Jun 1, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsarune left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsarune left a comment • edited

Choose a reason for hiding this comment

Mr-Anyone commented Jun 11, 2024

Test New Changes

nimazareian commented Jun 13, 2024

Test New Changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsarune left a comment

Choose a reason for hiding this comment

Mr-Anyone May 31, 2024 •

edited

Mr-Anyone Jun 1, 2024 •

edited

itsarune left a comment •

edited

itsarune left a comment •

edited