Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add TencentARC PhotoMaker support #179

Merged
merged 65 commits into from
Mar 12, 2024
Merged

Add TencentARC PhotoMaker support #179

merged 65 commits into from
Mar 12, 2024

Commits on Feb 2, 2024

  1. first efforts at implementing photomaker; lots more to do

    bssrdf committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    5dbbea0 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2024

  1. added PhotoMakerIDEncoder model in SD

    bssrdf committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    cbfa702 View commit details
    Browse the repository at this point in the history

  2. fixed soem bugs; now photomaker model weights can be loaded into thei… 

    …r tensor buffers
    bssrdf committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    78651f2 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. added input id image loading

    bssrdf committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    7da51ad View commit details
    Browse the repository at this point in the history

  2. added preprocessing inpit id images

    bssrdf committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    702a732 View commit details
    Browse the repository at this point in the history

  3. finished get_num_tensors

    bssrdf committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    7a7baef View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. fixed a bug in remove_duplicates

    bssrdf committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    f16b4da View commit details
    Browse the repository at this point in the history

  2. add a get_learned_condition_with_trigger function to do photomaker stuff

    bssrdf committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    df7f642 View commit details
    Browse the repository at this point in the history

  3. add a convert_token_to_id function for photomaker to extract trigger … 

    …word's token id
    bssrdf committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    ad7ec45 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. making progress; need to implement tokenizer decoder

    bssrdf committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    87fcee0 View commit details
    Browse the repository at this point in the history

  2. making more progress; finishing vision model forward

    bssrdf committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    7f5f580 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. debugging vision_model outputs

    bssrdf committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    0a38d84 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. corrected clip vision model output

    bssrdf committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    304704f View commit details
    Browse the repository at this point in the history

  2. continue making progress in id fusion process

    bssrdf committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    184e4b8 View commit details
    Browse the repository at this point in the history

  3. finished stacked id embedding; to be tested

    bssrdf committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    024c187 View commit details
    Browse the repository at this point in the history

  4. remove garbage file

    bssrdf committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    a584147 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2024

  1. debuging graph compute

    bssrdf committed Feb 10, 2024
    Configuration menu
    Copy the full SHA
    d939514 View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2024

  1. more progress; now alloc buffer failed

    bssrdf committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    807c340 View commit details
    Browse the repository at this point in the history

  2. fixed wtype issue; input images can only be 1 because issue with tran… 

    …sformer when batch size > 1 (to be investigated)
    bssrdf committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    5a13b48 View commit details
    Browse the repository at this point in the history

  3. added delayed subject conditioning; now photomaker runs and generates… 

    … images
    bssrdf committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    f4bf8e0 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. fixed stat_merge_step

    bssrdf committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    857af48 View commit details
    Browse the repository at this point in the history

  2. added photomaker lora model (to be tested)

    bssrdf committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    b0579ec View commit details
    Browse the repository at this point in the history

  3. reworked pmid lora

    bssrdf committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    753231a View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2024

  1. finished applying pmid lora; to be tested

    bssrdf committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    44052e1 View commit details
    Browse the repository at this point in the history

  2. finalized pmid lora

    bssrdf committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    c122507 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. add a few print tensor; tweak in sample again

    bssrdf committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    539a94a View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. small tweak; still not getting ID faces

    bssrdf committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    26c5cca View commit details
    Browse the repository at this point in the history

  2. fixed a bug in FuseBlock forward; also remove diag_mask op in for vis… 

    …ion transformer; getting better results
    bssrdf committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    191c5ec View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. disable pmid lora apply for now; 1 input image seems working; > 1 not… 

    … working
    bssrdf committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    26f591d View commit details
    Browse the repository at this point in the history

  2. turn pmid lora apply back on

    bssrdf committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    62b0a9b View commit details
    Browse the repository at this point in the history

  3. fixed a decode bug

    bssrdf committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6064b0e View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2024

  1. fixed a bug in ggml's conv_2d, and now > 1 input images working

    bssrdf committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    d38b85d View commit details
    Browse the repository at this point in the history

  2. add style_ratio as a cli param; reworked encode with trigger for atte… 

    …ntion weights
    bssrdf committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    a6a676b View commit details
    Browse the repository at this point in the history

  3. merge commit fixing lora free param buffer error

    bssrdf committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    aa21577 View commit details
    Browse the repository at this point in the history

  4. change default style ratio to 10%

    bssrdf committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    78ec6f7 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. added an option to offload vae decoder to CPU for mem-limited gpus

    bssrdf committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    8053aef View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. removing image normalization step seems making ID fidelity much higher

    bssrdf committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    dbe74df View commit details
    Browse the repository at this point in the history

  2. revert default style ratio back ro 20%

    bssrdf committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    e32462d View commit details
    Browse the repository at this point in the history

  3. added an option for normalizing input ID images; cleaned up debugging… 

    … code
    bssrdf committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    8b89e0e View commit details
    Browse the repository at this point in the history

  4. more clean up

    bssrdf committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    b7a4540 View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2024

  1. merged with master and resolved conflicts; adapted to GGMLBlock API

    bssrdf committed Feb 25, 2024
    Configuration menu
    Copy the full SHA
    53beb92 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. merge with a master again; build ok

    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    d35861b View commit details
    Browse the repository at this point in the history

  2. fixed bugs; now failed with cuda error; likely out-of-mem on GPU

    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    b556d44 View commit details
    Browse the repository at this point in the history

  3. free pmid model params when required

    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    4ac3df1 View commit details
    Browse the repository at this point in the history

  4. photomaker working properly now after merging and adapting to GGMLBlo… 

    …ck API
    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    77b6a42 View commit details
    Browse the repository at this point in the history

  5. remove tensor renaming; fixing names in the photomaker model file

    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    464f1f8 View commit details
    Browse the repository at this point in the history

  6. updated README.md to include instructions and notes for running Photo… 

    …Maker
    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    4b31b75 View commit details
    Browse the repository at this point in the history

  7. a bit clean up

    bssrdf committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    7b72e17 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. remove -DGGML_CUDA_FORCE_MMQ; more clean up and README update

    bssrdf committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    adbb9a1 View commit details
    Browse the repository at this point in the history

  2. add input image requirement in README

    bssrdf committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    6079615 View commit details
    Browse the repository at this point in the history

  3. bring back freeing pmid lora params buffer; simply pooled output of C… 

    …LIPvision
    bssrdf committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    fd098af View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2024

  1. remove MultiheadAttention2; customized MultiheadAttention

    bssrdf committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    70c3397 View commit details
    Browse the repository at this point in the history

  2. added a WIN32 get_files_from_dir; turn off Photomakder if receiving n… 

    …o input images
    bssrdf committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    16c9da0 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2024

  1. update docs

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    41f20e6 View commit details
    Browse the repository at this point in the history

  2. fix ci error

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    27887b6 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into add-photomaker-support

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    983e552 View commit details
    Browse the repository at this point in the history

  4. make stable-diffusion.h a pure c header file 

    This reverts commit 27887b6.
    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    b0940f0 View commit details
    Browse the repository at this point in the history

  5. fix ci error

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    f8c0831 View commit details
    Browse the repository at this point in the history

  6. format code

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    745ed8f View commit details
    Browse the repository at this point in the history

  7. reuse get_learned_condition

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    6bb87cf View commit details
    Browse the repository at this point in the history

  8. reuse pad_tokens

    @leejet
    leejet committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    7e2c796 View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2024

  1. reuse CLIPVisionModel

    @leejet
    leejet committed Mar 9, 2024
    Configuration menu
    Copy the full SHA
    9b3c8d8 View commit details
    Browse the repository at this point in the history

  2. reuse LoraModel

    @leejet
    leejet committed Mar 9, 2024
    Configuration menu
    Copy the full SHA
    df28af9 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2024

  1. add --clip-on-cpu

    @leejet
    leejet committed Mar 10, 2024
    Configuration menu
    Copy the full SHA
    6727d1c View commit details
    Browse the repository at this point in the history

  2. fix lora name conversion for SDXL

    @leejet
    leejet committed Mar 10, 2024
    Configuration menu
    Copy the full SHA
    df2afd8 View commit details
    Browse the repository at this point in the history