RelativePositionBias.mdx

import { Callout } from "nextra-theme-docs" import { Tab, Tabs } from "nextra-theme-docs"

RelativePositionBias (Attention)

Relative Position Bias for T5.

Attention(
    dim=768,
    num_heads=12,
    plugins=[
        RelativePositionBias(
            num_buckets=256,
            num_heads=12,
        )
    ],
)