v0.16.14

Flux v0.16.14

Diff since v0.16.13

Merged pull requests:

• Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl) (#2754) (@CarloLucibello)
• Integer class labels in crossentropy and logitcrossentropy (#2758) (@CarloLucibello)
• Compute losses in Float32 for Float16/BFloat16 inputs (#2760) (@CarloLucibello)
• README: standard example first, install/GPU/feature overview; CI: nightly non-blocking (#2763) (@CarloLucibello)
• Fix the gradient rule of xlogx (#2764) (@CarloLucibello)
• Don't copy BFloat16 input of a BFloat16 layer in _match_eltype (#2769) (@CarloLucibello)
• Fused rule for logitcrossentropy with class labels (#2770) (@CarloLucibello)
• Use NNlib.layernorm in the LayerNorm forward pass (#2773) (@CarloLucibello)
• Fix a GC race in the Reactant compile-cache test (#2775) (@CarloLucibello)
• Remove the ignored Greek keywords of focal_loss and tversky_loss (#2779) (@parth-patel010)
• Build error messages lazily: first Conv 1.4 s → 0.4 s (#2783) (@CarloLucibello)
• Let layers customize f16/bf16/f32/f64 with a 2-argument method (#2784) (@CarloLucibello)
• Drop the Reactant override of logsumexp in the label loss (#2785) (@CarloLucibello)

Closed issues:

• Cross-entropy losses with integer class targets (#2756)
• Losses should compute in Float32 for Float16/BFloat16 inputs (#2757)
• xlogx rrule refers to an undefined y, so scalar gradients throw UndefVarError (#2762)
• _match_eltype copies the BFloat16 input of a BFloat16 layer (#2766)
• logitcrossentropy with integer labels: 15 ms of memory traffic on large bf16 logits (#2767)
• LayerNorm should use NNlib.layernorm in the forward (#2771)
• focal_loss and tversky_loss silently ignore the γ, ϵ and β keywords (#2778)
• First Conv construction spends ~1 s inferring repr(::Symbol) in _check_pad_mode (#2780)
• Public interface for layers to customize f16/bf16 conversion (#2782)