Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic-Tokens: Generate from token JSON export #33931

Draft
wants to merge 37 commits into
base: master
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
7a48bb0
Stash initial for now
Mitch-At-Work Feb 11, 2025
13424bc
Add initial Link tokens as example
Mitch-At-Work Feb 12, 2025
5badbf8
Add tsconfig for semantic-tokens
Mitch-At-Work Feb 12, 2025
36cc985
Update indexing and add token vars as exports
Mitch-At-Work Feb 14, 2025
88cc6f4
Change files
Mitch-At-Work Feb 18, 2025
6b008bd
Fix version of react-tokens
Mitch-At-Work Feb 18, 2025
8dba624
Update and connect local build artifacts
Mitch-At-Work Feb 18, 2025
4a8db31
Lint and fix exports
Mitch-At-Work Feb 19, 2025
e2aa7f2
Update E2E for semantic tokens to a dummy test (for now)
Mitch-At-Work Feb 20, 2025
96ef49f
Fix up extra bracket and brand ref color
Mitch-At-Work Feb 20, 2025
1d6d672
remove console
Mitch-At-Work Feb 21, 2025
f478839
Alphebetize order
Mitch-At-Work Feb 24, 2025
bb81096
Update tokens so far
Mitch-At-Work Feb 28, 2025
940f605
Add script to generate
Mitch-At-Work Feb 28, 2025
44bfcdf
Initial script gen
Mitch-At-Work Feb 28, 2025
57c7867
Update tokens and script
Mitch-At-Work Feb 28, 2025
81f1e05
Update script to handle duplicated tokens
Mitch-At-Work Feb 28, 2025
6035592
Add strokeWidthThin fallback
Mitch-At-Work Feb 28, 2025
83f6a0a
Set variable path correctly
Mitch-At-Work Mar 1, 2025
e5671d7
Update latest
Mitch-At-Work Mar 3, 2025
0ed592f
Update with better fallback logic
Mitch-At-Work Mar 3, 2025
c81c862
Update script to handle fallbacks and f2 tokens better
Mitch-At-Work Mar 3, 2025
4d4bd3f
Update naming to handle brackets and spacings
Mitch-At-Work Mar 3, 2025
4609e99
Ensure fallbacks work correctly for FST
Mitch-At-Work Mar 3, 2025
9c3a7fe
Simplify script to match design and add custom fluent fallback map
Mitch-At-Work Mar 5, 2025
032d3b7
Add missing imports structure
Mitch-At-Work Mar 5, 2025
a750369
Fix imports
Mitch-At-Work Mar 5, 2025
a55568e
Update script to add exports
Mitch-At-Work Mar 5, 2025
5658e2d
Update script to handle imports/exports
Mitch-At-Work Mar 5, 2025
56e1102
Token script fixed
Mitch-At-Work Mar 5, 2025
e3a0bba
Update tokens and script
Mitch-At-Work Mar 6, 2025
58bac11
Fix token reference fallback
Mitch-At-Work Mar 6, 2025
791a838
Update api
Mitch-At-Work Mar 6, 2025
9c1c981
Fix port
Mitch-At-Work Mar 6, 2025
8884ea1
Lint all files
Mitch-At-Work Mar 6, 2025
a5bb059
Update and remove eslint rule
Mitch-At-Work Mar 6, 2025
c4719f4
Fix up the ESLint rule
Mitch-At-Work Mar 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update latest
  • Loading branch information
Mitch-At-Work committed Mar 3, 2025
commit e5671d7701bd706b624d94adc10fdba8faa08a88
112 changes: 86 additions & 26 deletions packages/semantic-tokens/scripts/tokenGen.ts
Original file line number Diff line number Diff line change
@@ -107,7 +107,48 @@ function toCamelCase(str: string) {
.join('');
}

function handleTokenTransforms(tokenString: string) {
// Base case
let formattedToken: string = toCamelCase(tokenString) + 'Raw';
// Token fallback may be a fluent 2 token or new FST, let's check known fluent 2 tokens
if (tokenString.length > 0) {
if (tokenString === 'Thin') {
formattedToken = `tokens.strokeWidthThin`;
} else if (tokenString.startsWith('Line-height')) {
const tokenFallbackArr = tokenString.split('/');
const tokenFallbackVal = tokenFallbackArr[tokenFallbackArr.length - 1];
formattedToken = `tokens.lineHeightBase${tokenFallbackVal}`;
} else if (tokenString.startsWith('Font-size')) {
const tokenFallbackArr = tokenString.split('/');
const tokenFallbackVal = tokenFallbackArr[tokenFallbackArr.length - 1];
formattedToken = `tokens.fontSizeBase${tokenFallbackVal}`;
} else if (tokenString.startsWith('Neutral/Stroke/Transparent')) {
formattedToken = `tokens.strokeNeutralTransparent`;
} else if (tokenString.startsWith('Neutral/Stroke/Disabled')) {
formattedToken = `tokens.strokeNeutralDisabled`;
} else if (tokenString.startsWith('Neutral/Stroke/')) {
const tokenFallbackArr = tokenString.split('/');
const tokenFallbackVal = tokenFallbackArr[2];
formattedToken = `tokens.colorNeutralStroke${tokenFallbackVal}`;
} else if (tokenString.startsWith('Neutral/Background/') || tokenString.startsWith('Neutral/Foreground/')) {
const tokenFallbackArr = tokenString.split('/');
const tokenFallbackArea = tokenFallbackArr[1];
const tokenFallbackVal = tokenFallbackArr[2];
const tokenFallbackType = tokenFallbackArr[tokenFallbackArr.length - 1];
if (tokenFallbackType === 'Rest') {
// Rest tokens don't have a qualifier
formattedToken = `tokens.colorNeutral${tokenFallbackArea}${tokenFallbackVal}`;
} else {
formattedToken = `tokens.colorNeutral${tokenFallbackArea}${tokenFallbackVal}${tokenFallbackType}`;
}
}
}

return formattedToken;
}

function generateTokenVariables() {
const foundTokens: { [tokenName: string]: string } = {};
// Default our files to token imports
// TODO: Add raw token imports
let optionalTokens = generateImportHeaders();
@@ -119,43 +160,59 @@ function generateTokenVariables() {
const tokenNameRaw = token + 'Raw';

let tokenFallback: null | string = null;
let tokenFallbackName: null | string = null;
let tokenSemanticRef: null | string = null;
let tokenSemanticName: null | string = null;

if (tokenData.fst_reference.length > 0) {
tokenSemanticRef = toCamelCase(cleanFSTTokenName(tokenData.fst_reference)) + 'Raw';
tokenSemanticName = toCamelCase(cleanFSTTokenName(tokenData.fst_reference));
tokenSemanticRef = tokenSemanticName + 'Raw';
}

// Token fallback may be a fluent 2 token or new FST, let's check known fluent 2 tokens
// Token fallback may be a fluent 2, we format these fallbacks manually
if (tokenData.fallback.length > 0) {
if (tokenData.fallback === 'Thin') {
tokenFallback = `tokens.strokeWidthThin`;
} else if (tokenData.fallback.startsWith('Line-height')) {
const tokenFallbackArr = tokenData.fallback.split('/');
const tokenFallbackVal = tokenFallbackArr[tokenFallbackArr.length - 1];
tokenFallback = `tokens.lineHeightBase${tokenFallbackVal}`;
} else if (tokenData.fallback.startsWith('Font-size')) {
const tokenFallbackArr = tokenData.fallback.split('/');
const tokenFallbackVal = tokenFallbackArr[tokenFallbackArr.length - 1];
tokenFallback = `tokens.fontSizeBase${tokenFallbackVal}`;
} else {
// handle base case
tokenFallback = toCamelCase(tokenData.fallback) + 'Raw';
}
tokenFallbackName = toCamelCase(cleanFSTTokenName(tokenData.fallback));
tokenFallback = handleTokenTransforms(tokenData.fallback);
}

let resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)})`;

// The semantic token was processed already, use the full semantic token CSS var.
if (tokenFallbackName && foundTokens[tokenFallbackName]) {
tokenFallback = foundTokens[tokenFallbackName];
} else if (tokenFallback) {
tokenFallback = escapeInlineToken(tokenFallback);
}

// The fallback token was processed already, use the full fallback CSS var.
if (tokenSemanticName && foundTokens[tokenSemanticName]) {
tokenSemanticRef = foundTokens[tokenSemanticName];
} else if (tokenSemanticRef) {
tokenSemanticRef = escapeInlineToken(tokenSemanticRef);
}

// TODO: Check if a token has a FST reference that falls back to another FST/fluent fallback?
if (tokenFallback && tokenSemanticRef && tokenFallback !== tokenSemanticRef) {
if (
tokenFallback &&
tokenSemanticRef &&
tokenFallback !== tokenSemanticRef &&
!tokenSemanticRef.includes(tokenFallback)
) {
// Token has both a FST fallback and a Fluent fallback
resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)}, var(${escapeInlineToken(
tokenSemanticRef,
)}, ${escapeInlineToken(tokenFallback)}))`;
resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)}, var(${tokenSemanticRef}, ${tokenFallback}))`;
} else if (tokenSemanticRef) {
// Token just has a FST reference fallback
resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)}, ${escapeInlineToken(tokenSemanticRef)})`;
resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)}, ${tokenSemanticRef})`;
} else if (tokenFallback) {
// Just in case a token falls back directly to a Fluent fallback
resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)}, ${escapeInlineToken(tokenFallback)})`;
resolvedTokenFallback = `var(${escapeInlineToken(tokenNameRaw)}, ${tokenFallback})`;
}

if (token === 'ctrlLinkForegroundNeutralHover') {
console.log('Found ctrl link token:', tokenData);
console.log('Token fallback:', tokenFallback);
console.log('Token semantic ref:', tokenSemanticRef);
console.log('Resolved token fallback:', resolvedTokenFallback);
}

if (tokenData.name.startsWith('CTRL/')) {
@@ -164,17 +221,20 @@ function generateTokenVariables() {
if (!componentTokens[component]) {
componentTokens[component] = generateImportHeaders();
}
componentTokens[component] += `export const ${token} = '${resolvedTokenFallback}';\n`;
componentTokens[component] += `export const ${token} = \`${resolvedTokenFallback}\`;\n`;
} else {
// We have a global token
if (tokenData.optional) {
optionalTokens += `export const ${token} = '${resolvedTokenFallback}';\n`;
optionalTokens += `export const ${token} = \`${resolvedTokenFallback}\`;\n`;
} else if (tokenData.nullable) {
nullableTokens += `export const ${token} = '${resolvedTokenFallback}';\n`;
nullableTokens += `export const ${token} = \`${resolvedTokenFallback}\`;\n`;
} else {
controlTokens += `export const ${token} = '${resolvedTokenFallback}';\n`;
controlTokens += `export const ${token} = \`${resolvedTokenFallback}\`;\n`;
}
}

// Track our tokens - we will use these to do fallback replacement for complex tokens as we find them.
foundTokens[token] = resolvedTokenFallback;
}

fs.writeFileSync('./src/optional/tokens.ts', optionalTokens);
34 changes: 17 additions & 17 deletions packages/semantic-tokens/src/components/avatar/tokens.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
// eslint-disable-next-line no-restricted-imports
import { tokens } from '@fluentui/tokens';

export const ctrlAvatarSize = 'var(${ctrlAvatarSizeRaw}, var(${sizeCtrlDefaultRaw}, ${📦 sizeAvatarSizeRaw}))';
export const ctrlAvatarCornerItem = 'var(${ctrlAvatarCornerItemRaw}, ${cornerCircularRaw})';
export const ctrlAvatarBackground = 'var(${ctrlAvatarBackgroundRaw}, ${neutralBackground6RestRaw})';
export const ctrlAvatarForeground = 'var(${ctrlAvatarForegroundRaw}, ${neutralForeground3RestRaw})';
export const ctrlAvatarIconSize = 'var(${ctrlAvatarIconSizeRaw}, ${sizeCtrlIconRaw})';
export const ctrlAvatarPresencebadgeSize = 'var(${ctrlAvatarPresencebadgeSizeRaw})';
export const ctrlAvatarActiveringSize = 'var(${ctrlAvatarActiveringSizeRaw})';
export const ctrlAvatarPresencebadgePaddingBottomrightoffset = 'var(${ctrlAvatarPresencebadgePaddingBottomrightoffsetRaw}, ${strokewidthDefaultRaw})';
export const ctrlAvatarCornerGroup = 'var(${ctrlAvatarCornerGroupRaw}, var(${cornerCtrlRestRaw}, ${📦 sizeAvatarCornerGroupRaw}))';
export const ctrlAvatarActiveringStrokewidth = 'var(${ctrlAvatarActiveringStrokewidthRaw})';
export const ctrlAvatarPresencebadgeStrokewidth = 'var(${ctrlAvatarPresencebadgeStrokewidthRaw}, ${strokewidthDefaultRaw})';
export const ctrlAvatarTextFontsize = 'var(${ctrlAvatarTextFontsizeRaw}, ${textGlobalBody3FontsizeRaw})';
export const ctrlAvatarTextLineheight = 'var(${ctrlAvatarTextLineheightRaw}, ${textGlobalBody3LineheightRaw})';
export const ctrlAvatarTextPaddingTopoffset = 'var(${ctrlAvatarTextPaddingTopoffsetRaw}, ${nullNumberRaw})';
export const ctrlAvatarActiveringStroke = 'var(${ctrlAvatarActiveringStrokeRaw}, var(${backgroundCtrlBrandRestRaw}, ${📦 colorAvatarActive ringRaw}))';
export const ctrlAvatarShowcutout = 'var(${ctrlAvatarShowcutoutRaw})';
export const ctrlAvatarPresencebadgeBackgroundBehindbadge = 'var(${ctrlAvatarPresencebadgeBackgroundBehindbadgeRaw}, ${backgroundLayerPrimary (solid)Raw})';
export const ctrlAvatarSize = `var(${ctrlAvatarSizeRaw}, var(var(${sizeCtrlDefaultRaw}), ${📦 sizeAvatarSizeRaw}))`;
export const ctrlAvatarCornerItem = `var(${ctrlAvatarCornerItemRaw}, var(${cornerCircularRaw}, ${circularRaw}))`;
export const ctrlAvatarBackground = `var(${ctrlAvatarBackgroundRaw}, ${tokens.colorNeutralBackground6})`;
export const ctrlAvatarForeground = `var(${ctrlAvatarForegroundRaw}, ${tokens.colorNeutralForeground3})`;
export const ctrlAvatarIconSize = `var(${ctrlAvatarIconSizeRaw}, var(${sizeCtrlIconRaw}))`;
export const ctrlAvatarPresencebadgeSize = `var(${ctrlAvatarPresencebadgeSizeRaw})`;
export const ctrlAvatarActiveringSize = `var(${ctrlAvatarActiveringSizeRaw})`;
export const ctrlAvatarPresencebadgePaddingBottomrightoffset = `var(${ctrlAvatarPresencebadgePaddingBottomrightoffsetRaw}, var(${strokewidthDefaultRaw}, ${tokens.strokeWidthThin}))`;
export const ctrlAvatarCornerGroup = `var(${ctrlAvatarCornerGroupRaw}, var(var(${cornerCtrlRestRaw}, ${mediumRaw}), ${📦 sizeAvatarCornerGroupRaw}))`;
export const ctrlAvatarActiveringStrokewidth = `var(${ctrlAvatarActiveringStrokewidthRaw})`;
export const ctrlAvatarPresencebadgeStrokewidth = `var(${ctrlAvatarPresencebadgeStrokewidthRaw}, var(${strokewidthDefaultRaw}, ${tokens.strokeWidthThin}))`;
export const ctrlAvatarTextFontsize = `var(${ctrlAvatarTextFontsizeRaw}, var(${textGlobalBody3FontsizeRaw}, ${tokens.fontSizeBase300}))`;
export const ctrlAvatarTextLineheight = `var(${ctrlAvatarTextLineheightRaw}, var(${textGlobalBody3LineheightRaw}, ${tokens.lineHeightBase300}))`;
export const ctrlAvatarTextPaddingTopoffset = `var(${ctrlAvatarTextPaddingTopoffsetRaw}, ${nullNumberRaw})`;
export const ctrlAvatarActiveringStroke = `var(${ctrlAvatarActiveringStrokeRaw}, var(var(${backgroundCtrlBrandRestRaw}, ${brandBackground1RestRaw}), ${📦 colorAvatarActive ringRaw}))`;
export const ctrlAvatarShowcutout = `var(${ctrlAvatarShowcutoutRaw})`;
export const ctrlAvatarPresencebadgeBackgroundBehindbadge = `var(${ctrlAvatarPresencebadgeBackgroundBehindbadgeRaw}, ${backgroundLayerPrimary (solid)Raw})`;
48 changes: 24 additions & 24 deletions packages/semantic-tokens/src/components/badge/tokens.ts
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
// eslint-disable-next-line no-restricted-imports
import { tokens } from '@fluentui/tokens';

export const ctrlBadgeTextPaddingTop = 'var(${ctrlBadgeTextPaddingTopRaw})';
export const ctrlBadgeTextPaddingBottom = 'var(${ctrlBadgeTextPaddingBottomRaw}, ${ctrlBadgeTextPaddingTopRaw})';
export const ctrlBadgeSmTextPaddingTop = 'var(${ctrlBadgeSmTextPaddingTopRaw})';
export const ctrlBadgeSmTextPaddingBottom = 'var(${ctrlBadgeSmTextPaddingBottomRaw}, ${ctrlBadgeSmTextPaddingTopRaw})';
export const ctrlBadgeLgTextPaddingTop = 'var(${ctrlBadgeLgTextPaddingTopRaw})';
export const ctrlBadgeLgTextPaddingBottom = 'var(${ctrlBadgeLgTextPaddingBottomRaw}, ${ctrlBadgeLgTextPaddingTopRaw})';
export const ctrlBadgeIconTheme = 'var(${ctrlBadgeIconThemeRaw}, ${iconthemeCtrlDefaultRestRaw})';
export const ctrlBadgeBeaconSize = 'var(${ctrlBadgeBeaconSizeRaw})';
export const ctrlBadgeSize = 'var(${ctrlBadgeSizeRaw})';
export const ctrlBadgeCorner = 'var(${ctrlBadgeCornerRaw})';
export const ctrlBadgeIconSizeFigmaOnly = 'var(${ctrlBadgeIconSizeFigmaOnlyRaw})';
export const ctrlBadgeGap = 'var(${ctrlBadgeGapRaw})';
export const ctrlBadgeIconSize = 'var(${ctrlBadgeIconSizeRaw})';
export const ctrlBadgePadding = 'var(${ctrlBadgePaddingRaw})';
export const ctrlBadgeSmSize = 'var(${ctrlBadgeSmSizeRaw})';
export const ctrlBadgeSmCorner = 'var(${ctrlBadgeSmCornerRaw})';
export const ctrlBadgeSmIconSizeFigmaOnly = 'var(${ctrlBadgeSmIconSizeFigmaOnlyRaw})';
export const ctrlBadgeSmIconSize = 'var(${ctrlBadgeSmIconSizeRaw})';
export const ctrlBadgeSmPadding = 'var(${ctrlBadgeSmPaddingRaw})';
export const ctrlBadgeLgSize = 'var(${ctrlBadgeLgSizeRaw})';
export const ctrlBadgeLgIconSizeFigmaOnly = 'var(${ctrlBadgeLgIconSizeFigmaOnlyRaw})';
export const ctrlBadgeLgIconSize = 'var(${ctrlBadgeLgIconSizeRaw})';
export const ctrlBadgeLgCorner = 'var(${ctrlBadgeLgCornerRaw})';
export const ctrlBadgeLgPadding = 'var(${ctrlBadgeLgPaddingRaw})';
export const ctrlBadgeTextPaddingTop = `var(${ctrlBadgeTextPaddingTopRaw})`;
export const ctrlBadgeTextPaddingBottom = `var(${ctrlBadgeTextPaddingBottomRaw}, var(${ctrlBadgeTextPaddingTopRaw}))`;
export const ctrlBadgeSmTextPaddingTop = `var(${ctrlBadgeSmTextPaddingTopRaw})`;
export const ctrlBadgeSmTextPaddingBottom = `var(${ctrlBadgeSmTextPaddingBottomRaw}, var(${ctrlBadgeSmTextPaddingTopRaw}))`;
export const ctrlBadgeLgTextPaddingTop = `var(${ctrlBadgeLgTextPaddingTopRaw})`;
export const ctrlBadgeLgTextPaddingBottom = `var(${ctrlBadgeLgTextPaddingBottomRaw}, var(${ctrlBadgeLgTextPaddingTopRaw}))`;
export const ctrlBadgeIconTheme = `var(${ctrlBadgeIconThemeRaw}, var(${iconthemeCtrlDefaultRestRaw}))`;
export const ctrlBadgeBeaconSize = `var(${ctrlBadgeBeaconSizeRaw})`;
export const ctrlBadgeSize = `var(${ctrlBadgeSizeRaw})`;
export const ctrlBadgeCorner = `var(${ctrlBadgeCornerRaw})`;
export const ctrlBadgeIconSizeFigmaOnly = `var(${ctrlBadgeIconSizeFigmaOnlyRaw})`;
export const ctrlBadgeGap = `var(${ctrlBadgeGapRaw})`;
export const ctrlBadgeIconSize = `var(${ctrlBadgeIconSizeRaw})`;
export const ctrlBadgePadding = `var(${ctrlBadgePaddingRaw})`;
export const ctrlBadgeSmSize = `var(${ctrlBadgeSmSizeRaw})`;
export const ctrlBadgeSmCorner = `var(${ctrlBadgeSmCornerRaw})`;
export const ctrlBadgeSmIconSizeFigmaOnly = `var(${ctrlBadgeSmIconSizeFigmaOnlyRaw})`;
export const ctrlBadgeSmIconSize = `var(${ctrlBadgeSmIconSizeRaw})`;
export const ctrlBadgeSmPadding = `var(${ctrlBadgeSmPaddingRaw})`;
export const ctrlBadgeLgSize = `var(${ctrlBadgeLgSizeRaw})`;
export const ctrlBadgeLgIconSizeFigmaOnly = `var(${ctrlBadgeLgIconSizeFigmaOnlyRaw})`;
export const ctrlBadgeLgIconSize = `var(${ctrlBadgeLgIconSizeRaw})`;
export const ctrlBadgeLgCorner = `var(${ctrlBadgeLgCornerRaw})`;
export const ctrlBadgeLgPadding = `var(${ctrlBadgeLgPaddingRaw})`;
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// eslint-disable-next-line no-restricted-imports
import { tokens } from '@fluentui/tokens';

export const ctrlBooleanSelectionhint = 'var(${ctrlBooleanSelectionhintRaw})';
export const ctrlBooleanSelectionhint = `var(${ctrlBooleanSelectionhintRaw})`;
Loading
Oops, something went wrong.
Loading
Oops, something went wrong.