Stable Diffusion

44 readers

1 users here now

Welcome to the Stable Diffusion community, dedicated to the exploration and discussion of the open source deep learning model known as Stable Diffusion.

Introduced in 2022, Stable Diffusion uses a latent diffusion model to generate detailed images based on text descriptions and can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by text prompts. The model was developed by the startup Stability AI, in collaboration with a number of academic researchers and non-profit organizations, marking a significant shift from previous proprietary models that were accessible only via cloud services.

founded 2 years ago

MODERATORS

buda@lemmy.ml

Creating somewhat consistent Characters as Illustrations for TTRPG (self.stablediffusion)

submitted 2 years ago by Nicktar to c/stablediffusion@lemmy.ml

4 comments fedilink hide all child comments

I don't know if this community is intened for posts like this, if not, I'm sorry and I'll delete this post ASAP....

So, I play TTRPG (mostly online) and I'm a big fan of visual aids, so I wanted to create some chahrcter images for my charakter in the new campaign I'm playing in. I don't need perfect consistency as humans usually change a little over time and I only needed the character to be recognizable on a couple of images that are usually viewed on their own and not side by side, so nothing like the consistency you'd need for a comic book or something similar. So I decided to create a Textual Inversion following this tutorial and it worked way better than expected. After less than 6 epochs I had a consistency that was enough for my usecase and it didn't start to overfit when I stopped the training around epoch 50.

Generated image of a character wearing a black hoodie standing in a rundown neighborhood at night Generated image of the character wearing a black hoodie standing on a street Gerneated image of the character cosplaying as Ironman Generater image of the character cosplaying as Amos from the Expanse

Then my SO, who's playing in the same campaign asked me to do the same for their character. So we went through the motions and created and filtered the images. A first training attempt had the TI starting to overfit halfway through the second epoch, so I lowered the learning rate by factor five and started another round. This time the TI started overfitting somewhere around epoch 8 without reaching consistency before. The generated images alternate between a couple of similar yet distinguishable faces. To my eye the training images seem to have a simliar or higher quality than the images I used in the first set. Was I just lucky with my first TI and unlucky with the other two and simply should keep on trying or is there something I should change (like the learningrate that still seems high to me with 0.0002 judging from other machine learning topics)?

you are viewing a single comment's thread
view the rest of the comments

[–] thebardingreen@lemmy.starlightkel.xyz 1 points 2 years ago

I tried, about 3 months ago, to get consistent characters working using that exact same tutorial. I had a vague notion of maybe actually building a video game... Talentless as I am at visual arts.

Long story short, I got it to work well enough to temp myself into a false sense of potential success, only to spend hours and hours generating images that were so frustratingly, tantalizingly close to what I wanted!

If I actually was a talented, skilled visual artist, I think I could use this as a starting place to do really cool work much faster... But, I just eventually got frustrated with the minor flaws and inconsistencies between the images I was spending so much time generating.