Fix Ernie Moe Test

[vasqu]

Flaky test:

• pytest tests/models/ernie4_5_moe/test_modeling_ernie4_5_moe.py -k test_load_balancing_loss --flake-finder --flake-runs=1000
• Encounter 0 failures

Hope it's more robust again

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ydshieh]

Thank you.

@is_flaky(max_attempts=2)

(already here before this PR)

not working well?

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: ernie4_5_moe

[vasqu]

I think the core changes are

• Properly propagate padding tokens within the ids

  transformers/tests/models/ernie4_5_moe/test_modeling_ernie4_5_moe.py

  Lines 115 to 120 in eb3c088

  	# Add padding tokens to input_ids
  	padding_block = config.pad_token_id * torch.ones(input_ids.shape[0], pad_length, dtype=torch.int32).to(
  	torch_device
  	)
  	padded_input_ids = torch.cat((padding_block, input_ids), dim=1) # this is to simulate padding to the left
  	padded_attention_mask = padded_input_ids.ne(config.pad_token_id).to(torch_device)

• And using the config pad token id, not some arbitrary number

I think either or both contributed to slightly different behavior compared to the other moe models