3D can be a powerful tool for brands and creatives, providing immersive and engaging experiences while enhancing the design process.

However, it can also be expensive, time-consuming, and difficult to execute effectively, making it impractical for everyday business operations.

But generative artificial intelligence (AI) is once again tackling this challenge - today, Nvidia is attempting to carve out a place in this new field. At the GTC 2024 conference, the company announced that its Nvidia Edify multimodal generative AI model can now generate 3D content and has established partnerships with Shutterstock and Getty Images for Edify-powered tools.

Shutterstock is offering early access to the application programming interface (API) built on Edify, which can create 3D objects for virtual scenes through text prompts and images.

Meanwhile, Getty is adding custom fine-tuning capabilities to its generative AI service, allowing enterprise customers to generate visual content that aligns with brand guidelines and styles.

Developers will soon be able to test these models through Nvidia NIM, a new collection of inference microservices announced at GTC.

"3D asset generation is one of the latest features Edify offers to developers and visual content providers, who will also be able to exert more creative control over AI image generation," wrote Gerardo Delgado, Nvidia product manager, in a blog post about this new feature.

Getty fine-tunes Edify for specific brands (including Sam's Club, Mucinex, and Coca-Cola)

One of the biggest challenges faced by generative AI is finer control over AI image outputs.

To help address this issue, Getty announced the launch of its repair and expand API based on Edify at the Consumer Electronics Show (CES) in January. The repair feature allows objects to be added, removed, or replaced in images, while the expand feature can enlarge the canvas. Both features are now available on the Getty website and iStock.com.

Starting in May, the company will also offer new services that allow enterprises to customize fine-tuning of Edify based on specific brands and styles. This will be achieved through a no-code self-service approach, where brands can upload proprietary datasets, review automatic tags, submit fine-tuning parameters, and review results before deployment.

In addition, developers will soon have access to Sketch, Depth, and Segmentation features. These features allow users to submit sketches to guide image generation, replicate the composition of reference images through depth maps, and segment various parts of an image to add, remove, or enhance characters and objects.

"Getty Images continues to expand the capabilities of its commercially safe generative AI service, which provides compensation protection for users' generated content," wrote Delgado.

According to the company, Getty's generative AI tools are being used by "leading creatives and advertisers," including:

• Dentsu: The Japanese PR agency is using Nvidia Picasso to fine-tune Getty's models for Sam's Club, a membership-based retail giant. The company also uses Getty to support the "Everyone Anime" project, which generates comic and animation-style content for marketing use cases.

• McCann: The creative agency has used generative AI to create a game for the over-the-counter medicine brand Mucinex; this interactive feature allows users to interact with their grumpy mascot, Mr. Mucus.

• WPP: The marketing and communications company is collaborating with Coca-Cola to fine-tune Getty's models for building custom visual content for this iconic soda brand.

Shutterstock speeds up prototype design

With Shutterstock's new 3D AI service, users can quickly generate virtual objects for creative and scene setups. According to Nvidia, users can input text prompts or reference images and choose from a range of popular 3D formats.

"This capability can significantly reduce the time required for designing scene prototypes, allowing artists to focus more on main characters and objects," wrote Delgado.

This Edify-based tool is commercially safe and trained on Shutterstock-licensed data. The stock photography company has already paid royalties to hundreds of thousands of artists and "expects to pay royalties to millions of artists to recognize their content IP's role in training generative technology," Nvidia said.

Shutterstock is also developing Edify-based tools that use text or image prompts to create 360 HDRi scenes to illuminate 3D scenes.

In addition, Shutterstock and HP showcased their collaboration in 3D custom printing at this week's GTC. With Shutterstock's 3D AI generator, designers can create digital content that HP can convert into 3D printable models.

"HP's 3D printers subsequently transform these models into physical prototypes to inspire product design," wrote Delgado.

Leading companies, including Dassault Systèmes and CGI studio Katana, are already using Shutterstock's 3D tools. These companies are integrating the generative 360 HDRi API based on Nvidia Omniverse for developing 3D scenes using the Universal Scene Description (OpenUSD).

Accenture Song is also leveraging Omniverse and Edify microservices based on generative AI to develop the Defender vehicle.

According to Delgado, this "enables the creation of cinematic, interactive 3D environments through dialogue prompts. The result is a fully immersive 3D scene that harmoniously unifies realistic generated environments with the digital twin of the Defender vehicle."