confusion

[skywo1f]

Forgive my confusion. Is there a guide on how to use this? I am used to systems like darknet where I have images and annotations, then I train on them. Finally I have a cfg and weights file which I can use to find examples in an image.

I ran both scripts (train DALL-E and train VAE) but nothing has changed in my folder. How do I get from here to the point where I can give it a sentence and have it produce an image for me?

[powderblock]

+1

Am also confused on how to use this. Even with existing data... How do I enter input and get a visual output?

[powderblock]

@lucidrains any help on this one? Would love to get started but confused just like this other guy :)

[johndpope]

It seems @deepglugs has got somewhere with his own training
https://github.com/deepglugs/dalle / there's specific helpful training instructions (requires tags / image foo.txt foo.png ) tags file format. (should be a pull request) maybe @lucidrains this is helpful?

python3 dalle.py --source path/to/images/and/tags/ \
				 --vocab curated_512.vocab \
				 --vae vae.pt \
				 --train_dalle \
				 --dalle dalle.pt \
				 --batch_size=16 \
				 --samples_out samples/dalle/ \
				 --epochs=2

There's a ticket related to training sets -
#7

Looking at @josephcappadona sample 50mb file - there's the following format.
1166-004-E9B61A49.jpg

1166-004-E9B61A49.txt
The basic design of Hawaii's state seal has been in use since 1894, but the legend now reads “State of Hawaii” rather than “Republic of Hawaii.” The Hawaiian coat of arms is supported by Kamehameha I and the goddess of liberty, with a rising sun behind. The motto “Ua Mau ke Ea o ka Aina i ka Pono” (The Life of the Land Is Perpetuated in Righteousness) is along the bottom edge. Below the shield are various symbols: a phoenix rising from flames, taro leaves, banana foliage, and maidenhair ferns.

I'm thinking the tag format by @deepglugs may yield better results initially on smaller datasets??
1girl, white_swimsuit, red_hair
There seems to be some guidance around vocab tokens (.vocab file) ~ 512 tags which would help things converge, right?

I'd also consider we could use an google /aws image recognition software to find tags on a large image dataset.