To meet the huge demand for marketing content, digital creative agency Mediaman has developed its own visualisation tools that harness the benefits of AI. Stephen Holmes discovers a workflow that offers users improved control and better results
By now, many readers will have had the chance to try out some of the AI-enabled visualisation tools offered online – and you don’t need to be an experienced visualisation specialist to quickly spot their flaws. Weirdly merged bodies, gobbledy-gook text and unrealistic lighting are just some of the immediate issues, not to mention the questions they raise about security and intellectual property protection.
At the same time, many would agree that the speed at which these products can build something useful from relatively little is impressive. It’s an ability that most creatives would like to harness.
That’s certainly true of the creatives at Chinese digital marketing agency Mediaman, which has recently added its in-house developed Jester3D software to its list of achievements. While the software is not yet openly available, it’s one of the first to target some of the most acute pain points of the AI visualisation process and give users some control, while still tapping into the benefits of speed and ease of use.
Steeped in digital, interactive content production for brands including Mercedes-Benz, Porsche, Bosch, UnderArmor and the South Korean cosmetics giant Amore Pacific, Mediaman knows how to transform workflows, both for its own internal teams and for its clients.
Much of this is about controlling scenes and keeping rendered products as close to the real thing as possible. The company already develops databases of hiresolution 3D digital twins for its clients using a mix of CAD data, 3D scanning and human artistry, and these assets form the key data for Jester3D.
Controlled, colour-accurate, and offering perfect life-like detail, these assets can be taken by a user and embedded in scenes and scenarios quickly and with no need for an extensive knowledge of visualisation software.
But in an age of throwaway content, Mediaman knows that the lifetime of a marketing image can be brutally short. Brands can’t be seen to be reusing the same image across social media, multiple times a day – but tight budgets and schedules can make it a real challenge to outsource content creation.
In response, Mediaman has developed Jester3D, which combines tools that already exist on the market, but brings them together in a way that makes them more accessible to a broader audience.
Speedy results
Web-based Jester3D utilises Three.js on WebGL technology, allowing it to be run on any modern laptop with speedy results. The Jester3D back end uses Stable Diffusion to power its AI elements, while the front end UI is skinned for each of Mediaman’s unique client dashboards.
This manages users and their permissions, gives an overview of all their brands, products and marketing campaigns, and is designed for ease of use by those not versed in 3D CAD and rendering software.
One key element is that Jester3D allows for the creation of projects. The agency’s own experience of working with clients is embedded deep in the product, enabling users not only to create content, but also edit or adjust it based on feedback.
To create a new project, you simply name it and choose one of the brands from the client, which then gives you access to its product assets, all professionally produced digital twins. Using the 3D-to-image workflow, you can add multiple products to your scene, where they form key building blocks.
You’re then ready to add in other premade assets from the Mediaman library; for example, flowers and foliage for use in a perfume advertisement. There are some pre-sets available, which add to the intuitive nature of the toolset, helping someone completely new to all this get to grips with the process quickly.
The clever bit is the ability to add in shapes – the greyscale spheres, prisms and cuboids used to guide the AI in the prompt stages. These can act as plinths for a perfume bottle, for example, or the positioning of buildings surrounding a car.
By using these 3D elements, you immediately gain a lot more control of a scene. Equally, the lighting and perspective can be altered as you’re not simply working with a static 2D PNG file, but 3D entities.
The next step is for Jester3D to take all this data – the type and position of objects, perspective and lighting requirements – and bring it all together on the back-end render server in a minute or so, creating a high-resolution, ray traced, quality render to use as the base for the AI generation.
From there, it comes down to the AI inputs. They are: Positive Prompt, Negative Prompt, Pick an AI Framework, Prompt Strength, Style Templates. (The Mediaman team, like others we’ve spoken to working in this field, recommend using an AI such as ChatGPT or similar to write prompts.)
In our example, a fragrance scene, the prompt for natural stone material takes its position and rough volume from the cube we placed in the scene. The perspective is matched and the dappled light takes its direction from the input, with realistic reflections and shadows.
The digital twins remain photorealistic and perfect in the final scenes, maintaining fine details such as text and labels, and this carries over into the finished scenes. Surrounding features respond well to the scene-setting procedures used.
Some AI outcomes still lack the required finesse of commercially usable content. Some need some postproduction work, while others may fall wide of the mark and require further prompt changes.
However, what Jester3D is good for is churning out lower res web content – for social media posts, images for e-commerce sites and similar – and feeding back ideas and concepts to the professional designers at Mediaman.
This is evident in the export settings at present, capable of 2k output at 2048 x 2048 pixels. The sizing allows greater output speed, and while the versatile square format proves ideal for social media, a key reason is that most AI tools are trained on square images, so this format usually gives the best results.
Meeting demand
Jester3D is already being used by Mediaman clients, with one cosmetics company using it to produce several hundred images on a weekly basis for its social media and e-commerce platforms in China, with an ever-improving success rate.
While the immediate focus for Jester3D is clearly Mediaman’s own clients, this toolset has great potential as a licensable software for in-house marketing teams in the future as companies look to boost digital creation skills.
With few marketing teams having the skills to develop 3D visualisation work from scratch, let alone the budget to spend on multiple seats of 3D design software, a co-creation mode, for designers to work alongside marketing teams and play around with different camera perspectives, prompt moods and styles, is something Mediaman is developing.
In a crowded market, the offered experience of greater client involvement and even co-creation definitely helps with buy-in on ideas.
There’s also a potential use case here for product designers, helping them to increase their render output, by moving beyond their existing toolset for complex and timeconsuming hi-res workflows, to more agile AI-enabled output tools for faster, simpler jobs. That could free up visualisation specialists, or simply reduce the cost associated with visualisation tool spend.
The team at Mediaman says that an enhanced version is in the works, capable of 4k output with greater reflections and refractions and able to be used in print or on billboards. And that could spell even more value for end users.
In its current form, Jester3D allows clients to communicate with agency teams with more ease and greater clarity. In best case scenarios, they can even create some content themselves. It’s an interesting way to package a toolset and provides a glimpse of a future in which companies develop software themselves using AI to increase productivity and build stronger relationships with customers.
This article first appeared in DEVELOP3D Magazine
DEVELOP3D is a publication dedicated to product design + development, from concept to manufacture and the technologies behind it all.
To receive the physical publication or digital issue free, as well as exclusive news and offers, subscribe to DEVELOP3D Magazine here