battle_tag is wrong for invite-only battles

[Jay2645]

My AI kept randomly not doing anything in some battles, so I did some investigating.

Turns out these errors were being put into the log:

INFO - >>> battle-gen8randombattle-1220084985|/timer on
DEBUG - <<< |pm|!Geniusect-Q|~|/error /timer - must be used in a chat room, not a console

The root cause:

2020-11-09 11:54:58,144 - Geniusect-Q - DEBUG - Received message to handle: >battle-gen8randombattle-1220084985-pgoxls8251t2qkotmilabhfhov3uhlepw
|init|battle
|title|Geniusect-Q vs. yaasgagaga
|raw|<div class="broadcast-red"><strong>This battle is invite-only!</strong><br />Users must be invited with <code>/invite</code> (or be staff) to join</div>
|j|☆Geniusect-Q

2020-11-09 11:54:58,145 - Geniusect-Q - INFO - New battle started: battle-gen8randombattle-1220084985

As you can see, the battle's "real" ID is battle-gen8randombattle-1220084985-pgoxls8251t2qkotmilabhfhov3uhlepw, but because of the way that the battle_tag is being split, the battle_tag is battle-gen8randombattle-1220084985.

Would it be possible to throw an exception when /error is received in a PM (to identify when this happens) and also to fix how the battle_tag is generated for invite-only battles? I think this is a recent change, since I wasn't seeing it a couple days ago.

[hsahovic]

Hey @Jay2645,

Thanks for opening this issue. I don't think the recent commits have changed anything in the way battle tags are parsed, since the battle tag extraction logic has not changed in the last yearr. It might be a showdown update though. Regardless, I'll try to push a patch to fix this by tomorrow :)

On the error management: it's the first time I see an error sent by PM; error messages are more or less handled correcty, and unknown errors are logged with CRITICAL level.

I've never tested invite-only battles - how do I start one, and what's the fastest way to experiment with them?

[Jay2645]

I've never done an invite-only battle myself, but looking at the Showdown UI there's a new "Don't allow spectators" box under the "Search for battles" button. I'd assume that's what's causing this, and that's why I've seen an uptick in this sort of thing recently. I've just been sending a bot out into the ladder; it happens every half-hour or so.

I agree that it's a little strange to get errors over PM -- my guess is that since Showdown doesn't know what room you're trying to send a message in, it defaults to sending you a PM instead.

[hsahovic]

Well, thanks for crash testing this :) This is super valuable feedback; I'll fix the private-battle thing asap and take a deeper look at the PM errors.

Out of curiosity, what kind of performance are you getting with your bots?

[Jay2645]

I've been using your Showdown fork to train using a very heavily modified version of the OpenAI Gym example.

I've updated the neural net a bit (not sure if these settings are optimal -- it's my first time using ML outside of basic "hello world" stuff):

    model = Sequential()
    model.add(Dense(input_layer_size, name="Input", input_shape=(1, input_layer_size)))
    model.add(LeakyReLU(alpha=0.1))
    model.add(Flatten())
    model.add(Dropout(get_input_drop_percent(), name="Input_Drop"))
    model.add(Dense(512, name="Hidden_Layer_1"))
    model.add(LeakyReLU(alpha=0.1))
    model.add(Dropout(get_hidden_drop_percent(), name = "Hidden_Drop"))
    model.add(Dense(output_layer_size, name="Inner_Output", activation="linear"))
    model.add(Dense(output_layer_size, name="Model_Output", activation="linear"))

The input has also been expanded from 10 inputs to about 1500 or so. It stores as much information as I can get about every known Pokemon, known move, and field condition (and I probably missed a few). I made a fork of this to turn the enums you used into IntEnums, mainly so I could encode conditions within a 0-1 range for learning:

def _side_condition_id(self, side_conditions : Set[SideCondition]) -> float:
        output = 0.0

        for condition in side_conditions:
            condition_bit = 1 << int(condition)
            output += condition_bit

        return output / (1 << len(SideCondition))

(Yes, the code is terrible -- this is mostly for my own playing around more than anything else!)

I've done similar things for weather, status, and pretty much any other enum -- just so I could get them within that magical 0-1 range. I'm also doing it for items/abilities (I added an Itemdex script to match the others):

        possible_abilities = -np.ones(3)
        pkm_possible_abilities = list(pkm.possible_abilities.values())
        for i in range(len(pkm_possible_abilities)):
            possible_abilities[i] = ABILITYDEX[to_id_str(pkm_possible_abilities[i])] / len(ABILITYDEX)

        try:
            item = ITEMS[to_id_str(pkm.item)]["num"] / len(ITEMS)
        except (AttributeError, KeyError, TypeError):
            item = -1

---

Against the random player agent, I can get about a 95% winrate. Against the example max damage agent (modified to take into account STAB, type resistances, etc.), I can get about a 60% winrate. I have a third "hybrid" agent, which is basically the max damage agent but has a percentage chance of doing something random instead. If the hybrid agent loses, that "random move" chance is decreased; if it wins, the "random move" chance is increased. There's also a setting to add a floor in order to make sure it doesn't just devolve into the random agent over time.

Training against the hybrid seems to help learning somewhat, since otherwise the Deep Q agent was just getting absolutely flattened and never seemed to learn anything.

This is its first real field test against humans -- I left it to train overnight and came back to find out it kept getting booted out of battles because of this ID mismatch. I did a hack on my end to just pass the entire Showdown-reported battle_tag to player._create_battle(), but that's not very clean and probably breaks something elsewhere.

[Jay2645]

I guess while we're on the topic of errors -- and this is more of a feature request (I can put it in another ticket if needed) -- would it be possible to throw an exception when we're being rate limited? This is the Showdown code:

if (this.chatQueue.length >= THROTTLE_BUFFER_LIMIT - 1) {
	connection.sendTo(
		room,
		`|raw|<strong class="message-throttle-notice">Your message was not sent because you've been typing too quickly.</strong>`
	);
	return false;

IIRC, right now that raw tag is being handled in battle.py, where it tries to parse it as a rating instead. I feel like something lower-level should be listening to this and throwing an exception or something to let you know you're being rate limited. Alternatively, if poke_env could handle the rate limiting itself (either by resending after a delay if it gets that message or keeping track on its own), that'd work too.

[hsahovic]

Great work on the DQN! You might be interested in using this agent during training, if you're looking for something stronger than the max damage player.

I can a look at the rate limiting issue too.

[hsahovic]

I haven't been able to reproduce the issue you encountered with invite only battles; there has been two commits to showdown which involve them since my local showdown fork was updated though, so it might come from there.
Regardless, #89 and version 0.3.10 should fix this issue as long as the battle tags have the same structure as you quoted above.

Let me know if it persists!

Regarding the other issues you mentioned (eg. pm errors and rate limiting), would you mind opening separate issues?

[Jay2645]

Done -- /error in PMs is #90; rate limits is #91.

[Jay2645]

Tried latest out and it seemed to work!

2020-11-09 17:47:32,973 - Geniusect-Q - INFO - >>> battle-gen8randombattle-1220297059-50fd41ikl94eawdkvzj7hfkc8ubi03mpw|/choose move leechseed
  Turn    7. | [⦻●●●●●][168/250hp] Ferrothorn -   Dusknoir [ 82%hp][●●]

Closing this ticket. 🥇