Long Message Receive Time on Client Connect

[7ark]

Describe the bug
There's occasionally (not consistent enough for me to easily reproduce every time unfortunately) a weird scenario where when a player joins my host, some messages that are sent seem to take a very long time to send or receive, up to 5 - 10 seconds.

Repro Steps and Explanation
I will attempt to explain how I'm handling my logic as much as I can. Again reproducing this isn't incredibly consistent to begin with unfortunately.
I have a custom network manager. In here, I have an override in OnServerConnect. In this, I send a message using

conn.Send(new PlayerIndexMessage()
{
 Index = playerIndex
}

The intent of this is a sort of initialization for any clients who connect so they know which player index they're meant to be, which is used for lots of misc stuff unimportant here.

My custom network manager registers this message callback inside on OnStartClient, and receives it in a function SetPlayerIndex.

Inside of this function, the client then sends out two messages itself. These are the messages that are delayed (at least that's what it appears to be, it's possible the index message is delayed, but i dont have any visuals or debugs at the moment to tell, and given the randomness of the reproducibility of this, I haven't noticed anything yet).

I have a singleton called NetworkCommands that I use to send out commands from many misc places. I use this to send the two messages as so

NetworkCommands.Instance.SendPlayerInfoMessage(new PlayerDisplayInfoMessage()
{
    Index = LocalPlayerIndex,
    DisplayName = GetPlayerName(),
    Icon = GetPlayerIcon()
});
NetworkCommands.Instance.SendIconMessage(new ToastMessage()
{
    Text = GetPlayerName() + " has joined the lobby!",
    Index = LocalPlayerIndex,
    Texture = GetPlayerIcon(),
    SoundToPlay = Sound.JoinNotificationOnline
});

and the code for the NetworkCommands is very simple, an example here:

public void SendPlayerInfoMessage(PlayerDisplayInfoMessage msg)
{
    CmdSendPlayerInfoMessage(msg);
}
[Command(requiresAuthority = false)]
private void CmdSendPlayerInfoMessage(PlayerDisplayInfoMessage msg)
{
    NetworkServer.SendToAll(msg);
}

This does work, I use this NetworkCommands singleton many places without any issue. I simply have the command that called the NetworkServer.SendToAll().

Those two messages then are meant to show a toast notification and update some general information about the client that has connected. However, in these random instances it just seems to be delayed up to 5 - 10 seconds. This includes any additional messages that I attempt to send in this time. If I send any other messages, they also are not triggered until everything else catches up.

So for example, Player A is hosting, Player B is joining the session. The two connect (I can see that they've connected via misc UI etc), and I would expect Player A to get a notification on screen (from the toast) about Player B having connected. However nothing happens. Adding debugs, etc. confirm that the registered callback hasn't happened yet. Player Bs screen currently shows as being connected to Player A as well.
Then, in this 5 - 10 seconds, if Player A attempts to load into a new scene (which i do through a custom message as well), Player A starts loading, but Player B is stuck and nothing happens. After a few seconds, the toast notification shows up on Player As screen, and then Player B starts loading into the scene as they were meant to.

It's like everything is clogged up temporarily.
I am using the Epic Games Transport.