Popular AI image creation service Midjourney has rolled out one of its most frequently requested features. It debuts the ability to consistently reproduce characters across multiple new-gen AI images.
This has been a big hurdle for AI image models to date, by their very nature.
This is because most AI image generators rely on “diffusion models,” tools similar to or based on Stability AI’s open-source image generation algorithm, Stable Diffusion, which works roughly by taking text entered by a user and trying to assemble a pixel-by-pixel image that matches that description, as learned from similar images and text labels in the massive (and controversial) training dataset of millions of images created by humans.
See also: Salesforce announces new AI tools for doctors
Why consistent characters are so powerful – and make generative AI imagery difficult
However, as with text-based large language models (LLMs) like OpenAI's ChatGPT or Cohere's new Command-R, the problem with all generative AI is the inadequacy of their responses: the AI generates something new for every unique prompt you feed it, even if the prompt is repeated or some of the same key words are used.
This is great for creating brand new content – in Midjourney’s case, new images. But what if you’re designing a film, a novel, a graphic novel or comic book, or some other visual medium where you want the same character or characters to move through it and appear in different scenes, environments, with different facial expressions and accessories?
This particular scene, which is usually essential to narrative continuity, has been very difficult to achieve with generative AI – until now. But Midjourney is now trying the trick, introducing a new tag, “–cref” (short for “character reference”) that users can add to the end of their prompts on the Midjourney Discord, and it will attempt to match the character’s features, body type, and even clothing from a URL that the user pastes after the tag.
As the feature progresses and improves, it could take Midjourney further from an interesting “toy” or source of ideas to a more professional tool.
See also: Adobe Express: New mobile application with Firefly generative AI features
How to use Midjourney's new consistent character feature
The tag works best with images that have been previously created by Midjourney. So, for example, the workflow for a user would be to create or retrieve the URL of a previously created character.
Let's start from the beginning and say we're creating a new character with this prompt: "a muscular bald man with an earring and an eye patch."
We'll zoom in on the image we like the most, then control-click on it on the Midjourney Discord server to find the "copy link" option.
We can then type a new prompt “wearing a white tuxedo standing in a mansion–cref [URL]” and paste the URL of the image we just created, and Midjourney will try to create that same character from before in our new context that we typed.
As you will see, the results are far from the exact nature of the original (or even our initial prompt), but certainly encouraging.
See also: ChatGPT Read Aloud: OpenAI Offers Voice Reading
Additionally, the user can somewhat control the “weight” of how closely the new image replicates the original character by applying the “–cw” tag followed by a number from 1 to 100 at the end of their new prompt (after the “–cref [URL]” string, like this: “–cref [URL] –cw 100.” The lower the “cw” number, the more deviation the new image will have. The higher the “cw” number, the more closely the new image will follow the original reference.
As you can see in our example, entering a very low “cw 8” does indeed return what we wanted: the white tuxedo. However, it has now removed our feature example.
But what can we do, nothing can fix a small “variety area” – right?
Okay, so the cover is on the wrong eye… but we’re getting closer!
You can also combine multiple characters into one by using two “–cref” tags side by side with their corresponding URLs.
See also: Apple: The new MacBook Air is the best laptop for AI
The feature just launched earlier tonight, but artists and creators are already testing it out. Try it out for yourself if you have Midjourney. And read the full note from founder David Holz below:
Hi everyone, we are testing a new “Character Reference” feature today. This is similar to the “Style Reference” feature, except it maps to a character image.
How it works
- Type –cref URL after your prompt with a URL for an image of a character.
- You can use –cw to modify the “strength” of the report from 100 to 0.
- Power 100 (–cw 100) is the default and uses the face, hair, and clothing.
- At power 0 (–cw 0) it will focus only on the face (good for changing clothes/hair etc.).
What is it intended for?
This feature works best when using characters created from Midjourney images. It is not designed for real people/photos (and will likely distort them like regular virtual prompts do).
Cref works in a similar way to regular virtual prompts, except that it “focuses” on the character’s attributes.
The accuracy of this technique is limited, it will not copy exact details such as wrinkles/pimples/or t-shirt logos.
Cref works for both Niji and regular MJ models and can also be combined with –sref.
Advanced functions
You can use more than one URL to mix the information/characters from multiple images like this cref URL1 URL2 (this is similar to multiple virtual or style prompts).
How does it work on the alpha web?
Drag or paste an image onto the imaginary line so that it has three icons. Selecting these determines whether it is a virtual prompt, a style reference, or a character reference. Press Shift+select an option to use an image for multiple categories.
Remember, while MJ V6 is in alpha this and other features may change suddenly, but the official beta of V6 is coming soon. We would like everyone's thoughts on ideas-and-features. We hope you enjoy this early version and hope it helps you play around with building stories and worlds.
Source: venturebeat
