Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/orchestrator integration #1191

Merged
merged 143 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
98d4d85
removed some dead code
bkdiehl Apr 30, 2024
fb071b7
remove unhelpful code
bkdiehl May 1, 2024
aa5616d
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 1, 2024
071839a
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 1, 2024
6c99631
test civitai client with new textToImage service
bkdiehl May 3, 2024
1ff8cf1
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 3, 2024
b3c22a4
refine textToImage caller
bkdiehl May 3, 2024
519f800
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 3, 2024
af2a462
begin work on formatting textToImage responses
bkdiehl May 3, 2024
92afd8a
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 3, 2024
67f1fa8
work on formatting textToImage response
bkdiehl May 6, 2024
73220ef
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 6, 2024
8e4ad73
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 6, 2024
49f3cc9
add services methods to integratoe orchestrator sdk
bkdiehl May 6, 2024
7a961c5
continue to work on textToImage orchestrator integration
bkdiehl May 7, 2024
c20dd56
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 20, 2024
6935ddc
update to use latest version of orchestrator
bkdiehl May 20, 2024
7eb97b4
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 21, 2024
f3edeb0
commit
bkdiehl May 21, 2024
c5b84ac
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 22, 2024
0a9affe
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 22, 2024
90bbf40
added workflow schemas in separate file
bkdiehl May 22, 2024
f1b79c2
Transitioned most of the old scheduler trpc endpoints to orchestrator…
bkdiehl May 23, 2024
c7dc51e
commit
bkdiehl May 24, 2024
4d2c61d
Merge branch 'main' into feature/orchestrator-integration
bkdiehl May 24, 2024
be1b3c6
created persistent form hook
bkdiehl May 29, 2024
6ef49d3
completed new generation form
bkdiehl May 29, 2024
9993f75
re-added remix button functionality to generationFormProvider
bkdiehl May 30, 2024
e48d204
hooks and endpoints for updating textToImage workflow metadata
bkdiehl May 31, 2024
dc03822
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 3, 2024
b0c7080
fixes after merge
bkdiehl Jun 3, 2024
0b78685
when creating textToImage workflows, report probihited requests
bkdiehl Jun 3, 2024
3efdabc
orchestrator service for models api
bkdiehl Jun 3, 2024
0108c25
added cookie encryption utils
bkdiehl Jun 3, 2024
770c6b7
added migration to support ApiKey type and expiresAt
bkdiehl Jun 4, 2024
eaa7406
validate session user with api token taking expiresAt into account
bkdiehl Jun 4, 2024
1aaa4ef
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 4, 2024
fd265cb
added npm package for @civitai/client
bkdiehl Jun 4, 2024
8431e00
update generation service cookie max age
bkdiehl Jun 4, 2024
d3fcda2
fix issue causing generation sidebar to always open on page load
bkdiehl Jun 4, 2024
2af1960
abstract civitaiClient to simplify instantiating new instances
bkdiehl Jun 4, 2024
1763dd4
InputQuantity now uses correct form context
bkdiehl Jun 4, 2024
a081913
added ORCHESTRATOR_API_TOKEN to env schema checks
bkdiehl Jun 4, 2024
0ed39f8
sort textToImage images by completedAt first, then by id alphabetically
bkdiehl Jun 4, 2024
c13baf1
fix issues with draft quantity accuracy with returned data
bkdiehl Jun 4, 2024
398c96f
remove reference to old Generation.Params type
bkdiehl Jun 4, 2024
9f39ded
fixed issue where persistent form errors didn't appear
bkdiehl Jun 5, 2024
8442e46
workaround fix to persistent form validation
bkdiehl Jun 5, 2024
9b1ad14
fix submit button rerendering
bkdiehl Jun 5, 2024
228d23b
common method to get resource data with airs for orchestrator
bkdiehl Jun 5, 2024
30867f1
stringifyAir no longer nullable
bkdiehl Jun 5, 2024
f8b8c18
added provider for textToImage whatIf queries
bkdiehl Jun 5, 2024
c0076db
cancelling a textToImage request will now update the workflow/image s…
bkdiehl Jun 5, 2024
b5c6869
updated to work with common utility that fetches resources and sets airs
bkdiehl Jun 5, 2024
2283678
removed/commented out old code
bkdiehl Jun 5, 2024
6f72c3b
update to accept zod schema with effects
bkdiehl Jun 5, 2024
86c6c49
cancelling workflow request only updates workflow status if any image…
bkdiehl Jun 5, 2024
68e9745
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 7, 2024
6c8d59c
removed unused methods from useGenerationStore
bkdiehl Jun 7, 2024
2e98077
show workflows with missing images
bkdiehl Jun 7, 2024
b13c3a9
use env var for textToImage callback url
bkdiehl Jun 7, 2024
4096474
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 7, 2024
71e8aa6
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 10, 2024
0a96b68
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 11, 2024
1d28c9f
fixed bad component keys
bkdiehl Jun 11, 2024
ae4ee32
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 17, 2024
d75cb9e
fix type issues after merge with main
bkdiehl Jun 17, 2024
1ba8251
updated OrchestratorClient to connect to orchestrator based on env vars
bkdiehl Jun 17, 2024
57d1c0d
updated @civitai/client
bkdiehl Jun 17, 2024
8d70f39
removed scheduler env vars
bkdiehl Jun 17, 2024
a35dfa5
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 17, 2024
470ab38
simpified generation store
bkdiehl Jun 17, 2024
7b59db2
added to generation constants
bkdiehl Jun 18, 2024
7a2bf1f
checking in with type errors
bkdiehl Jun 19, 2024
87275dd
fixed type issues
bkdiehl Jun 20, 2024
a2d20db
added trpc helper hooks file
bkdiehl Jun 20, 2024
1228ed6
added workflow step hooks
bkdiehl Jun 20, 2024
7046ba8
fixed various type issues
bkdiehl Jun 20, 2024
336e11d
remixing a queueItem now includes the step metadata
bkdiehl Jun 20, 2024
5446cce
remove old code
bkdiehl Jun 20, 2024
b8e6585
add notes to TextToImageStepMetadataSchema
bkdiehl Jun 20, 2024
5bd48d1
fixed issues with hiding multiple images
bkdiehl Jun 20, 2024
4462b4c
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 21, 2024
3256f97
fix workflow cost not showing correctly
bkdiehl Jun 21, 2024
ea5748a
fix bad merge
bkdiehl Jun 21, 2024
1716628
add endpoint to download historic scheduler images
bkdiehl Jun 24, 2024
494f896
rate limiting scheduler image downloads with signals callback
bkdiehl Jun 24, 2024
664ba51
scheduler downloader rate limiting with signals callback
bkdiehl Jun 24, 2024
cfd8ef1
added ability to download generated images as zip
bkdiehl Jun 24, 2024
28af106
hide sidebar image actions on "generate" page
bkdiehl Jun 25, 2024
6e4f90b
move constants variable to shared file
bkdiehl Jun 25, 2024
bd00f31
move file
bkdiehl Jun 26, 2024
bf75dd8
check in
bkdiehl Jun 26, 2024
9d065fc
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 26, 2024
e826e33
don't sort generated images by completed time
bkdiehl Jun 26, 2024
4726bcc
infer draft and nsfw input if not saved in step metadata
bkdiehl Jun 26, 2024
650ba62
reinstalled @civitai/client
bkdiehl Jun 26, 2024
d7f3305
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jun 27, 2024
5941a28
fix select all bug
bkdiehl Jun 27, 2024
078fc00
added some basic comfy types
bkdiehl Jun 28, 2024
c6e6182
fixed type errors
bkdiehl Jun 28, 2024
191527e
fixed remix not setting correct aspect ratio
bkdiehl Jun 28, 2024
21e129f
fixed remix not setting sampler
bkdiehl Jun 28, 2024
80b1216
start refactored ResourceSelectModal
bkdiehl Jun 28, 2024
3935314
fix type errors
bkdiehl Jun 29, 2024
e7635b2
check in work
bkdiehl Jul 1, 2024
2100d50
additional support for comfy nodes
bkdiehl Jul 2, 2024
b51320e
refactor to account for different types of generation workflows
bkdiehl Jul 9, 2024
66baae2
commit before joint session
bkdiehl Jul 9, 2024
37ad843
added upscale image modal
bkdiehl Jul 10, 2024
b27929b
Add lyco and dora loading to workflow prep
JustMaier Jul 10, 2024
1951d3c
additional workflow support
bkdiehl Jul 10, 2024
c774495
disable remix for upscaled images
bkdiehl Jul 10, 2024
ba37493
remove old code
bkdiehl Jul 10, 2024
0b8eeaa
update generation defaults
bkdiehl Jul 11, 2024
fa29463
update @civitai/client
bkdiehl Jul 11, 2024
3c5dffb
Fix workflow resource population
JustMaier Jul 11, 2024
abff1dc
Don't charge when running whatif
JustMaier Jul 11, 2024
a00eb2a
Adjust image-to-image workflow label position
JustMaier Jul 11, 2024
bf9117c
fixes after review
bkdiehl Jul 11, 2024
0325915
logging
bkdiehl Jul 11, 2024
537da9e
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jul 11, 2024
2c5691b
better labels for workflow definitions
bkdiehl Jul 11, 2024
bd1db00
fix type errors
bkdiehl Jul 11, 2024
3cffe3d
fix download dates
bkdiehl Jul 11, 2024
5b8e813
don't save unnecessary metadata to text to image workflow steps
bkdiehl Jul 11, 2024
51ab1a5
remove button
bkdiehl Jul 11, 2024
50b3c65
bug fixes
bkdiehl Jul 11, 2024
b67eef2
set workflowDefinitions button available to Justin and Briant
bkdiehl Jul 11, 2024
5c39b5f
Fix workflows
JustMaier Jul 11, 2024
22e4b04
Merge branch 'feature/orchestrator-integration' of https://github.com…
JustMaier Jul 11, 2024
2b1f8d7
fixed strength not setting to correct value on remix
bkdiehl Jul 12, 2024
a5b731a
Workflow fixes and additions
JustMaier Jul 12, 2024
b6eb9c9
upscale to use image url instead of params image url
bkdiehl Jul 12, 2024
c501e21
Adjust description display for image to image
JustMaier Jul 12, 2024
f7663ac
Adjust refining design
JustMaier Jul 12, 2024
e964475
Update release date of workflows
JustMaier Jul 12, 2024
db33a23
Add dummy checkpoint loader
JustMaier Jul 12, 2024
74dd07a
using height/width instead of relying so much on aspect ratio
bkdiehl Jul 12, 2024
b7545c2
Merge branch 'feature/orchestrator-integration' of https://github.com…
bkdiehl Jul 12, 2024
a371e46
remove console logs
bkdiehl Jul 12, 2024
6a7d955
Merge branch 'main' into feature/orchestrator-integration
bkdiehl Jul 12, 2024
a3f4657
sort generated images by completed date
bkdiehl Jul 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .env-example
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,6 @@ NEXT_PUBLIC_SEARCH_HOST=https://localhost:7700
NEXT_PUBLIC_SEARCH_CLIENT_KEY=aSampleKey

# Scheduler endpoint
SCHEDULER_ENDPOINT=url
ORCHESTRATOR_ENDPOINT=url
ORCHESTRATOR_ACCESS_TOKEN=cooltoken

Expand Down
642 changes: 377 additions & 265 deletions package-lock.json

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
"@aws-sdk/lib-storage": "^3.490.0",
"@aws-sdk/s3-request-presigner": "^3.490.0",
"@axiomhq/axiom-node": "^0.12.0",
"@civitai/client": "^0.1.9-beta.0",
"@clickhouse/client": "^0.2.2",
"@cloudamqp/amqp-client": "^2.1.1",
"@dnd-kit/core": "^6.1.0",
Expand Down Expand Up @@ -113,6 +114,7 @@
"cloudflare": "^2.9.1",
"cookies-next": "^2.1.1",
"cron-parser": "^4.6.0",
"crypto": "^1.0.1",
"dayjs": "^1.11.6",
"discord.js": "^14.7.1",
"embla-carousel-react": "^7.0.3",
Expand Down Expand Up @@ -222,7 +224,7 @@
"@types/xml2js": "^0.4.14",
"@typescript-eslint/eslint-plugin": "^5.33.0",
"@typescript-eslint/parser": "^5.33.0",
"autoprefixer": "^10.4.17",
"autoprefixer": "^10.4.19",
"cssnano": "^7.0.1",
"eslint": "8.22.0",
"eslint-config-airbnb": "^19.0.4",
Expand All @@ -237,7 +239,7 @@
"postcss": "^8.4.38",
"prettier": "^2.8.8",
"prisma": "^5.9.1",
"tailwindcss": "^3.4.1",
"tailwindcss": "^3.4.3",
"ts-node": "^10.9.1",
"typescript": "^5.5.3"
},
Expand All @@ -247,4 +249,4 @@
"overrides": {
"@react-aria/interactions": "3.16.0"
}
}
}
9 changes: 9 additions & 0 deletions prisma/migrations/20240604172025_api_key_type/migration.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@

-- CreateEnum
CREATE TYPE "ApiKeyType" AS ENUM ('System', 'User');

ALTER TYPE "KeyScope" ADD VALUE 'Generate';

-- AlterTable
ALTER TABLE "ApiKey" ADD COLUMN "expiresAt" TIMESTAMP(3),
ADD COLUMN "type" "ApiKeyType" NOT NULL DEFAULT 'User';
7 changes: 7 additions & 0 deletions prisma/schema.prisma
Original file line number Diff line number Diff line change
Expand Up @@ -1620,6 +1620,11 @@ model KeyValue {
value Json
}

enum ApiKeyType {
System
User
}

model ApiKey {
id Int @id @default(autoincrement())
key String @unique
Expand All @@ -1628,6 +1633,8 @@ model ApiKey {
userId Int
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
createdAt DateTime @default(now())
type ApiKeyType @default(User)
expiresAt DateTime?
}

enum KeyScope {
Expand Down
2 changes: 1 addition & 1 deletion src/components/Buzz/BuzzTransactionButton.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import { useBuzzTransaction } from './buzz.utils';
type Props = ButtonProps & {
buzzAmount: number;
message?: string | ((requiredBalance: number) => string);
label: string;
label: React.ReactNode;
onPerformTransaction?: () => void;
purchaseSuccessMessage?: (purchasedBalance: number) => React.ReactNode;
size?: MantineSize;
Expand Down
5 changes: 5 additions & 0 deletions src/components/Cards/ModelCard.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import { IconNose } from '~/components/SVG/IconNose';
import { UserAvatarSimple } from '~/components/UserAvatar/UserAvatarSimple';
import { VideoMetadata } from '~/server/schema/media.schema';
import { getSkipValue, shouldAnimateByDefault } from '~/components/EdgeMedia/EdgeMedia.util';
import { getIsSdxl } from '~/shared/constants/generation.constants';

const IMAGE_CARD_WIDTH = 450;

Expand Down Expand Up @@ -183,6 +184,10 @@ export function ModelCard({ data, forceInView }: Props) {
data.publishedAt &&
data.lastVersionAt > aDayAgo &&
data.lastVersionAt.getTime() - data.publishedAt.getTime() > constants.timeCutOffs.updatedModel;
const isSDXL = getIsSdxl(data.version?.baseModel);

const isPony = data.version?.baseModel === 'Pony';
const isOdor = data.version?.baseModel === 'ODOR';
const isArchived = data.mode === ModelModifier.Archived;
const onSite = !!data.version.trainingStatus;
const baseModelIndicator = BaseModelIndicator[data.version.baseModel as BaseModel];
Expand Down
1 change: 0 additions & 1 deletion src/components/Chopped/chopped.components.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ import {
} from '@tabler/icons-react';
import React, { Component, ErrorInfo, useState } from 'react';
import { useBuzzTransaction } from '~/components/Buzz/buzz.utils';
import { BuzzTransactionButton } from '~/components/Buzz/BuzzTransactionButton';
import {
GameState,
GlobalState,
Expand Down
1 change: 0 additions & 1 deletion src/components/Chopped/states/complete.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import { Carousel } from '@mantine/carousel';
import { IconArrowLeft, IconCheck, IconPlus, IconX } from '@tabler/icons-react';
import { useState } from 'react';
import { useBuzzTransaction } from '~/components/Buzz/buzz.utils';
import { BuzzTransactionButton } from '~/components/Buzz/BuzzTransactionButton';
import {
GameState,
GlobalState,
Expand Down
1 change: 0 additions & 1 deletion src/components/Chopped/states/joining.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import {
import { IconArrowLeft, IconCheck, IconPlus, IconX } from '@tabler/icons-react';
import { useState } from 'react';
import { useBuzzTransaction } from '~/components/Buzz/buzz.utils';
import { BuzzTransactionButton } from '~/components/Buzz/BuzzTransactionButton';
import {
GameState,
GlobalState,
Expand Down
1 change: 0 additions & 1 deletion src/components/Chopped/states/landing.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import {
import { IconCheck, IconPlus, IconX } from '@tabler/icons-react';
import { useState } from 'react';
import { useBuzzTransaction } from '~/components/Buzz/buzz.utils';
import { BuzzTransactionButton } from '~/components/Buzz/BuzzTransactionButton';
import { GameState, GlobalState, NewGame } from '~/components/Chopped/chopped.shared-types';
import { ComputeCost, defaultGameState, useChoppedStore } from '~/components/Chopped/chopped.utils';
import { EdgeMedia } from '~/components/EdgeMedia/EdgeMedia';
Expand Down
3 changes: 2 additions & 1 deletion src/components/CivitaiWrapped/AccountProvider.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { signIn, signOut, useSession } from 'next-auth/react';
import { useRouter } from 'next/router';
import { createContext, ReactNode, useContext, useEffect } from 'react';
import { civTokenEndpoint, EncryptedDataSchema } from '~/server/schema/civToken.schema';
import { generationServiceCookie } from '~/shared/constants/generation.constants';
import { deleteCookies } from '~/utils/cookies-helpers';

export type CivitaiAccount = {
Expand All @@ -22,7 +23,7 @@ type ogAccountType = {
} | null;
const ogAccountKey = 'civitai-og-account';

const deleteCookieList = ['ref_code', 'ref_source'];
const deleteCookieList = ['ref_code', 'ref_source', generationServiceCookie.name];

type AccountState = {
accounts: CivitaiAccounts;
Expand Down
33 changes: 20 additions & 13 deletions src/components/ImageGeneration/Feed.tsx
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
import { Center, Loader, createStyles, Stack, Alert, Text } from '@mantine/core';
import { IconInbox } from '@tabler/icons-react';
import { GeneratedImage } from '~/components/ImageGeneration/GeneratedImage';
import { useGetGenerationRequests } from '~/components/ImageGeneration/utils/generationRequestHooks';
import { useGetTextToImageRequestsImages } from '~/components/ImageGeneration/utils/generationRequestHooks';
import { InViewLoader } from '~/components/InView/InViewLoader';
import { generationPanel } from '~/store/generation.store';
import { isDefined } from '~/utils/type-guards';
import { ScrollArea } from '~/components/ScrollArea/ScrollArea';

export function Feed() {
const { classes } = useStyles();
const { requests, images, isLoading, fetchNextPage, hasNextPage, isRefetching, isError } =
useGetGenerationRequests();
const { requests, steps, isLoading, fetchNextPage, hasNextPage, isRefetching, isError } =
useGetTextToImageRequestsImages();

if (isError)
return (
Expand All @@ -26,7 +26,7 @@ export function Feed() {
</Center>
);

if (!images.length)
if (!steps.flatMap((x) => x.images).length)
return (
<Center h="100%">
<Stack spacing="xs" align="center" py="16">
Expand Down Expand Up @@ -56,16 +56,23 @@ export function Feed() {
<ScrollArea scrollRestore={{ key: 'feed' }} className="flex flex-col gap-2 px-3">
{/* <GeneratedImagesBuzzPrompt /> */}
<div className={classes.grid}>
{images
.map((image) => {
const request = requests.find((request) =>
request.images?.some((x) => x.id === image.id)
);
if (!request) return null;
{steps.map((step) =>
step.images
.map((image) => {
const request = requests.find((request) => request.id === image.workflowId);
if (!request) return null;

return <GeneratedImage key={image.id} request={request} image={image} />;
})
.filter(isDefined)}
return (
<GeneratedImage
key={`${image.workflowId}_${image.id}`}
request={request}
step={step}
image={image}
/>
);
})
.filter(isDefined)
)}
</div>
{hasNextPage && (
<InViewLoader loadFn={fetchNextPage} loadCondition={!isRefetching}>
Expand Down
Loading