AI Empowers Impressive Crowd Simulation in iClone

Simulating realistic crowds has long been a challenge in game design and character animation. Fortunately, Reallusion has created iClone Crowd Sim, an advanced tool that can handle these complex simulations in a way that is intelligent and consistent with the environment.

Adding to this great new development are advances in generative artificial intelligence technology, resulting in a truly dynamic solution that, until now, has involved purchasing several additional programs at a very high cost. This tutorial takes you step by step through the creation of crowds using AI to generate textures as well as the new features in iClone Crowd Sim. 

Step 1: Prepare your base characters 

The first step is to choose the base characters you will use to generate variations. You can use hyper-realistic character packs offered by Reallusion for Character Creator (CC) or create your own. It will be enough if you have a few to simulate a large population, but to work properly, each one must have unique elements.

For this, you can use the “Hair Builder 2 - eGirl” package to increase the attractiveness of your characters with a wide variety of high-quality hairstyles. On the other hand, for clothing selection, you can purchase the “Jackets and coats” package to apply directly to your characters. This pack contains up to 8 types of jackets and coats scanned in high quality 3D with a low poly methodology designed for video games and film productions. 

Step 2: Generate AI facial textures 

Export the facial texture of the base character you want to modify from Character Creator. To generate realistic facial textures from these textures we will use Stable Diffusion. An open-source diffusion model in which we will install “Mitsua Diffusion One,” an ethically developed model. Find and download the Mitsua Diffusion One model from its official website and place it in the corresponding folder of your Stable Diffusion model.

Then load the base texture in “Picture to Picture” mode and create a prompt such as: “hyper-realistic facial texture, highly detailed photograph, with very diffuse light.” Generate several images by adjusting the model settings without giving the AI too much creativity until you get satisfactory results.

Currently, it will not be easy for the AI to generate the texture at the resolution and hyper-realistic quality in the details we require. To make up for this we can use Photoaid.com or other similar software to increase the resolution of the generated texture and improve the overall definition. 

Before applying the texture, I recommend taking it to an image editor in which we overlay the original texture and the generated texture. This way we can replace the parts that the AI has not recreated well (such as ears or nostrils) and that might not put the texture back on the 3D model. 

When we apply this texture to the face, we will see that the character does not look the same as we are looking for, but there is still one more step to generate a face different from the original; the texture of normals. To create a normal map taking as a reference our generated diffuse texture, we must first convert it to a black and white image. Applying a contrast filter will further improve the result of the reliefs of the face and its definition. Finally, you can use free tools such as SmartNormaps to perform this conversion quickly and at the resolution you want. 

Step 3: Generate textures for clothing

You can use AI to generate cool designs and patterns for your characters' clothes. It is best to start with a basic sketch in a square format, in high resolution and light color so that the texture is easy to multiply on top of the clothing texture. It doesn't have to be very concrete; the important thing is the silhouette and the space it occupies. Create a simple sketch of the design you want to generate. 

Generate the image in the same way you generated the face texture and make final adjustments in image editing software if necessary. You can even generate patterns to apply to clothing if your version of Stable Diffusion is available. 

AI generates seamlessly tileable texture patterns for clothing texturing.

Step 4: iClone Crowd Sim 

After preparing the characters for our simulation, we'll proceed to import the scenario from our Unreal Engine project into iClone, where the crowd will be positioned.

Select the “Generate NavMesh” tool in iClone to define the traffic zones automatically. This intelligent tool delimits the areas where the characters can walk on the 3D mesh and generates the margins to avoid collisions. 

After doing this we can also create paths for the characters to follow using the “Path” tool. This will allow you to control where you want a group of characters to move.

It deploys the “Generate Crowd” menu to populate specific areas with animated characters. The menu allows us to enter characters from our “Content” menu, on the one hand, and movements for the characters on the other. 

When introducing several groups of people on stage, it may be necessary to use Navmesh barriers. They are useful because they prevent characters from overlapping during their animations and pedestrians can realistically dodge and avoid obstacles. 

Step 5: Export your crowd to Unreal Engine 

Finally, we are going to export, in a simple way, the crowd simulation we just created in iClone to Unreal Engine. My aim here is to render a beautiful animation. To do this you need to have the latest version of Unreal Live Link installed so that importing characters into Unreal Engine is faster and more efficient than ever. Remember that you have several tutorials about this process published by Reallusion.

I utilize the Unreal Blueprint to implement the random material feature, allowing for the application of AI-generated textures to both clothing and faces in a randomized manner.

To avoid the repetitive appearance of cloned characters in the imported Unreal Engine scene, we can utilize randomized materials in Unreal 5. These materials not only enable us to change clothing colors but also incorporate AI-generated facial and clothing textures.

While creating these textures involves multiple steps too complex to summarize here, I recommend watching the associated tutorial. It provides detailed explanations of the process, explores texture generation methods, and introduces a technique for creating textures tailored for CC characters, utilizing Reallusion's hyper-realistic materials upon character import.

That is all! I hope you like the final result, and that the tutorial has helped you. 

--

José Tijerín is a digital illustrator, 3D sculptor, and creator of video games such as Dear Althea available on Steam. His content packs “We’re Besties” and “We’re Homies” are currently for sale in the Reallusion content store.

>> Learn more about iClone Crowd Sim:

• Explore iClone for creating 3D character animation
• Create crowd animation with iClone Crowd Sim features
• Try iClone Crowd Sim for 30 days free
• All-in-one solution for crowd animation with Crowd Simulation Bundle
• Create cinematic 3D animation with iClone Unreal Live Link

José is a digital illustrator, 3D sculptor, and creator of video games like Dear Althea; his content packs We’re Besties and We’re Homies are sold in the Reallusion content store.