Text to image with AI experimentation
DreamBooth
LoRA
AI has been the hot topic this year and I have started to dip my toes into the Koolaid to feel it out for myself. I learn best by taking on projects that interest me and use the technologies I want to learn. The "project" asspect provides the stickiness factor that I need to power through some of the more difficult and boring parts of learning something new.
I ran across a blog post where the author was making self portraits with Stable Diffusion an LoRA. This led me to make a series of images using the same technique outlined in the blog post. The base models were various versions of models that can be found in Replicate.
I only had moderate success with the technique and eventually dropped the project due to unsatisfactory results.
Just recently I got a link to training SDXL with your own images and decided to give it another try. I had a lot more success this time around and was able to generate some pretty good images.
Along the way I followed a link to training and deploying a DreamBooth model and that is when things really got interesting. The instructions are quite simple and I was able to train the model and start producing images pretty quickly.
Using DreamBooth I was able to create images that most closely matched the idea that I had in my head. More importantly, when trying to generate images that look like the same person in the training set, I found DreamBooth to be much more successful than LoRA.
LoRA seems to be better at generating images that are more "imaginative" but not quite as recognizable.
Prompt Inspiration
To create the prompts I went to a variety of sources to use as inspriation. I like the sites which have the prompt that was used to generate the image so that I can start with it and tweak it to my liking.
Style Guides
I find the I generally have an idea of some kind of style for the image in my head but, lacking an kind of art training, I have a hard time describing it. There are a few sites that I have found where the authors have made comprehensive lists of artists and artistic styles which you add to your prompt to help guide the AI.