Final phase of source tree restructure #2833

lstein · 2023-02-28T04:54:51Z

All python code has been moved under `invokeai`. All vestiges of `ldm` and `ldm.invoke` are now gone.

You will need to run pip install -e . before the code will work again!

Everything seems to be functional, but extensive testing is advised.

A guide to where the files have gone is forthcoming.

This is the first phase of a big shifting of files and directories in the source tree. You will need to run `pip install -e .` before the code will work again! Here's what's in the current commit: 1) Remove a lot of dead code that dealt with checkpoint and safetensor loading. 2) Entire ckpt_generator hierarchy is now gone! 3) ldm.invoke.generator.* => invokeai.generator.* 4) ldm.model.* => invokeai.model.* 5) ldm.invoke.model_manager => invokeai.model.model_manager 6) In addition, a number of frequently-accessed classes can be imported from the invokeai.model and invokeai.generator modules: from invokeai.generator import ( Generator, PipelineIntermediateState, StableDiffusionGeneratorPipeline, infill_methods) from invokeai.models import ( ModelManager, SDLegacyType InvokeAIDiffuserComponent, AttentionMapSaver, DDIMSampler, KSampler, PLMSSampler, PostprocessingSettings )

psychedelicious · 2023-02-28T22:54:23Z

is git history/blame retained through the reorganisation?
i have a large PR [ui]: migrate all styling to chakra-ui theme #2814 WIP for the frontend. i'll probably be finished by friday if not sooner. i would really like to get that into main and then have the reorganisation be after that. rebasing this PR on a different folder structure will be very tedious.

Kyle0654 · 2023-02-28T23:40:30Z

is git history/blame retained through the reorganisation?

i have a large PR [ui]: migrate all styling to chakra-ui theme #2814 WIP for the frontend. i'll probably be finished by friday if not sooner. i would really like to get that into main and then have the reorganisation be after that. rebasing this PR on a different folder structure will be very tedious.

Sounds like frontend files may not be moving?

psychedelicious · 2023-03-01T00:31:10Z

Sounds like frontend files may not be moving?

oh, you're right. all good

lstein · 2023-03-01T03:42:48Z

is git history/blame retained through the reorganisation?

i have a large PR [ui]: migrate all styling to chakra-ui theme #2814 WIP for the frontend. i'll probably be finished by friday if not sooner. i would really like to get that into main and then have the reorganisation be after that. rebasing this PR on a different folder structure will be very tedious.

The identity of the files remains intact through organization and git blame stays the same. If a file has been moved you can still often, but not always, merge onto it without conflicts. That being said, the frontend and backend/webui files have not moved at all, and aren't going to. Everything else is moving to join them!

Kyle0654 · 2023-03-01T04:04:11Z

Looks like some __pycache__ snuck in.

Kyle0654 · 2023-03-01T04:11:42Z

It feels like the Stable Diffusion code should probably go in its own module under the backend folder, to keep it separate from app code (e.g. CLI or API). I noticed this most in the /invokeai/backend/models folder, where models here is referring to diffusion models, but models in an API usually refers to the data models used for serializing.

Then you could have /invokeai/backend/app with ~/cli, ~/api, and ~/core (or whatever makes sense).

Two popular templates are roughly organized this way:
https://github.com/Buuntu/fastapi-react
https://github.com/arthurhenrique/cookiecutter-fastapi

lstein · 2023-03-01T04:26:09Z

Looks like some __pycache__ snuck in.

Removed

lstein · 2023-03-01T04:27:27Z

It feels like the Stable Diffusion code should probably go in its own module under the backend folder, to keep it separate from app code (e.g. CLI or API). I noticed this most in the /invokeai/backend/models folder, where models here is referring to diffusion models, but models in an API usually refers to the data models used for serializing.

Then you could have /invokeai/backend/app with ~/cli, ~/api, and ~/core (or whatever makes sense).

Two popular templates are roughly organized this way: https://github.com/Buuntu/fastapi-react https://github.com/arthurhenrique/cookiecutter-fastapi

How about invokeai/backend/ldm/models ? I don't want to confuse people too much by renaming things too radically. Then the modules would move to invokeai/backend/ldm/modules and the utilities to invokeai/backend/ldm/util.

Kyle0654 · 2023-03-01T04:32:43Z

How about invokeai/backend/ldm/models ? I don't want to confuse people too much by renaming things too radically. Then the modules would move to invokeai/backend/ldm/modules and the utilities to invokeai/backend/ldm/util.

Things are moving around a bunch already. No better time to change names. Might be a good idea to separate from the ldm name too? Didn't diffusers supersede that?

lstein · 2023-03-02T01:58:32Z

How about invokeai/backend/ldm/models ? I don't want to confuse people too much by renaming things too radically. Then the modules would move to invokeai/backend/ldm/modules and the utilities to invokeai/backend/ldm/util.

Things are moving around a bunch already. No better time to change names. Might be a good idea to separate from the ldm name too? Didn't diffusers supersede that?

Diffusers are implementing latent diffusion models, so I think we can legitimately keep the ldm name. Structure is now invokeai/backend/ldm/{models, modules}

.gitignore

invokeai/backend/ldm/models/model_manager.py

damian0815 · 2023-03-02T09:31:22Z

as i said in discord, just gonna hit "approve" on this

damian0815

👍

ebr

new structure makes a lot of sense 💯 👍

Kyle0654 · 2023-03-03T17:30:54Z

Could you undo the auto code formatting on at least the invocations (and add whatever needs to be added to avoid it)? It changed this:

    prompt: Optional[str]     = Field(description="The prompt to generate an image from")
    seed: int                 = Field(default=-1, ge=-1, le=np.iinfo(np.uint32).max, description="The seed to use (-1 for a random seed)")
    steps: int                = Field(default=10, gt=0, description="The number of steps to use to generate the image")
    width: int                = Field(default=512, multiple_of=64, gt=0, description="The width of the resulting image")
    height: int               = Field(default=512, multiple_of=64, gt=0, description="The height of the resulting image")
    cfg_scale: float          = Field(default=7.5, gt=0, description="The Classifier-Free Guidance, higher values may result in a result closer to the prompt")
    sampler_name: SAMPLER_NAME_VALUES = Field(default="k_lms", description="The sampler to use")
    seamless: bool            = Field(default=False, description="Whether or not to generate an image that can tile without seams")
    model: str                = Field(default='', description="The model to use (currently ignored)")
    progress_images: bool     = Field(default=False, description="Whether or not to produce progress images during generation")

Into this:

    prompt: Optional[str] = Field(description="The prompt to generate an image from")
    seed: int = Field(
        default=-1,
        ge=-1,
        le=np.iinfo(np.uint32).max,
        description="The seed to use (-1 for a random seed)",
    )
    steps: int = Field(
        default=10, gt=0, description="The number of steps to use to generate the image"
    )
    width: int = Field(
        default=512,
        multiple_of=64,
        gt=0,
        description="The width of the resulting image",
    )
    height: int = Field(
        default=512,
        multiple_of=64,
        gt=0,
        description="The height of the resulting image",
    )
    cfg_scale: float = Field(
        default=7.5,
        gt=0,
        description="The Classifier-Free Guidance, higher values may result in a result closer to the prompt",
    )
    sampler_name: SAMPLER_NAME_VALUES = Field(
        default="k_lms", description="The sampler to use"
    )
    seamless: bool = Field(
        default=False,
        description="Whether or not to generate an image that can tile without seams",
    )
    model: str = Field(default="", description="The model to use (currently ignored)")
    progress_images: bool = Field(
        default=False,
        description="Whether or not to produce progress images during generation",
    )

IMO I prefer the original formatting. Especially since the latter isn't even consistent (e.g. steps has been consolidated to a single line, whereas cfg_scale splits each parameter to a separate line).

Kyle0654 · 2023-03-03T17:34:01Z

It's also a nitpick, but a lot of what's in app would typically go in backend in a client-server application (with the client living in frontend). We have a CLI as well (as opposed to the traditional API and web client), which complicates things some, but I do feel like having all the Python code in the same folder (to the extent that you could potentially even split off backend into its own repo) would make sense.

lstein · 2023-03-03T19:42:37Z

It's also a nitpick, but a lot of what's in app would typically go in backend in a client-server application (with the client living in frontend). We have a CLI as well (as opposed to the traditional API and web client), which complicates things some, but I do feel like having all the Python code in the same folder (to the extent that you could potentially even split off backend into its own repo) would make sense.

My thinking on this was to do the generate refactor (which means reimplementing nodes to run off of generator classes directly) and to integrate the node modules into the frontend/backend directories as the migration is finished.

lstein · 2023-03-03T20:00:12Z

Done. Note that the way to disable autoformatting is like this:

#fmt: off
code code code
#fmt: on

Kyle0654 · 2023-03-03T20:06:04Z

Do you mind hitting the rest of the invocations as well? E.g. the Img2Img one.

Ack looks like I was too late x.x

lstein · 2023-03-03T20:18:33Z

Do you mind hitting the rest of the invocations as well? E.g. the Img2Img one.

Ack looks like I was too late x.x

Oh shoot. I thought all the invocations were at the bottom of the files. Will make a PR.

mickr777 · 2023-03-03T23:11:51Z

@lstein fresh install on main when creating a image in txt2image (works fine on unified canvas and img2img)

Traceback (most recent call last):
  File "/home/invokeuser/InvokeAI/invokeai/backend/web/invoke_ai_web_server.py", line 1313, in generate_images
    self.generate.prompt2image(
  File "/home/invokeuser/InvokeAI/invokeai/backend/generate.py", line 539, in prompt2image
    generator = self.select_generator(
  File "/home/invokeuser/InvokeAI/invokeai/backend/generate.py", line 818, in select_generator
    return self._make_txt2img2img()
  File "/home/invokeuser/InvokeAI/invokeai/backend/generate.py", line 894, in _make_txt2img2img
    return self._load_generator(".txt2img2img", "Txt2Img2Img")
  File "/home/invokeuser/InvokeAI/invokeai/backend/generate.py", line 902, in _load_generator
    module = importlib.import_module(mn)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/invokeuser/InvokeAI/invokeai/backend/generator/txt2img2img.py", line 11, in <module>
    from ..models import PostprocessingSettings
ModuleNotFoundError: No module named 'invokeai.backend.models'

#2856

lstein requested review from blessedcoolant, damian0815, ebr, keturn, mauwii and psychedelicious as code owners February 28, 2023 04:54

lstein added 3 commits February 28, 2023 00:31

add missing files

d334f7f

add more missing files

3b921cf

restore behavior of !import_model; fix initial models bug

1d77581

lstein marked this pull request as draft February 28, 2023 05:46

lstein added 2 commits February 28, 2023 08:32

move models and generator into backend

5b6c61f

fix import of moved model_manager module

f2ce2f1

lstein marked this pull request as ready for review February 28, 2023 13:38

Merge branch 'main' into refactor/move-models-and-generators

87d1ec6

remove pycaches from repo

2c7928b

move models and modules under invokeai/backend/ldm

850d1ee

remove unused embeddings code

1a7371e

Kyle0654 approved these changes Mar 2, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

invokeai/backend/ldm/models/model_manager.py Show resolved Hide resolved

damian0815 approved these changes Mar 2, 2023

View reviewed changes

almost all of backend migrated; restoration next

3f0b0f3

lstein requested review from CapableWeb and tildebyte as code owners March 2, 2023 18:28

lstein added 2 commits March 3, 2023 00:02

all files migrated; tweaks needed

6a99056

all vestiges of ldm.invoke removed

60a98ca

lstein changed the title ~~first phase of source tree restructure~~ Final phase of source tree restructure Mar 3, 2023

lstein added 5 commits March 3, 2023 01:07

fix incorrect import of merge code

44400d2

fix weblint call

d606abc

fix issue with invokeai.version

9559005

fix ldm->invokeai pathnames in workflows

c0aff39

move diagnostic message to stderr; was confusing CI

3f53f11

damian0815 approved these changes Mar 3, 2023

View reviewed changes

ebr approved these changes Mar 3, 2023

View reviewed changes

CODEOWNERS coarse draft

70203e6

lstein added 2 commits March 3, 2023 14:47

merge localisation files that were added in main

dd4a1c9

revert and disable auto-formatting of invocations

6477e31

lstein merged commit b3dccfa into main Mar 3, 2023

lstein deleted the refactor/move-models-and-generators branch March 3, 2023 20:05

lstein restored the refactor/move-models-and-generators branch March 3, 2023 22:18

lstein deleted the refactor/move-models-and-generators branch April 11, 2023 15:14

Final phase of source tree restructure #2833

Final phase of source tree restructure #2833

Uh oh!

Conversation

lstein commented Feb 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

All python code has been moved under invokeai. All vestiges of ldm and ldm.invoke are now gone.

Uh oh!

psychedelicious commented Feb 28, 2023

Uh oh!

Kyle0654 commented Feb 28, 2023

Uh oh!

psychedelicious commented Mar 1, 2023

Uh oh!

lstein commented Mar 1, 2023

Uh oh!

Kyle0654 commented Mar 1, 2023

Uh oh!

Kyle0654 commented Mar 1, 2023

Uh oh!

lstein commented Mar 1, 2023

Uh oh!

lstein commented Mar 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kyle0654 commented Mar 1, 2023

Uh oh!

lstein commented Mar 2, 2023

Uh oh!

Uh oh!

Uh oh!

damian0815 commented Mar 2, 2023

Uh oh!

damian0815 left a comment

Choose a reason for hiding this comment

Uh oh!

ebr left a comment

Choose a reason for hiding this comment

Uh oh!

Kyle0654 commented Mar 3, 2023

Uh oh!

Kyle0654 commented Mar 3, 2023

Uh oh!

lstein commented Mar 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lstein commented Mar 3, 2023

Uh oh!

Kyle0654 commented Mar 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lstein commented Mar 3, 2023

Uh oh!

mickr777 commented Mar 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

lstein commented Feb 28, 2023 •

edited

Loading

All python code has been moved under `invokeai`. All vestiges of `ldm` and `ldm.invoke` are now gone.

lstein commented Mar 1, 2023 •

edited

Loading

lstein commented Mar 3, 2023 •

edited

Loading

Kyle0654 commented Mar 3, 2023 •

edited

Loading

mickr777 commented Mar 3, 2023 •

edited

Loading