Attribute warnings are not emitted on subsequent compilations

[lukaszsamson]

Elixir and Erlang/OTP versions

Erlang/OTP 26 [erts-14.1.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]

Elixir 1.16.0-rc.0 (b199896) (compiled with Erlang/OTP 26)

Operating system

macOS

Current behavior

Given a simple module in lib/sample.ex

defmodule Sample do
  @foo
end

on first mix compile a warning is emitted

Compiling 1 file (.ex)
warning: undefined module attribute @foo, please remove access to @foo or explicitly set it before access
  lib/a.ex:2: Sample (module)

On subsequent compile there is no warning

Possibly related to #13162 as this affects other attribute warnings as well.

Expected behavior

All warnings are emitted unless --no-all-warnings flag is given