Copyright Fast Company
Most immersive experiences today may feel stale in retrospect. Brands have invested heavily in creating spaces meant to captivate, yet these experiences all replicate the same visual and audio cues, making it increasingly difficult for brands to differentiate. The underlying issue is a technological design constraint: You can either create something highly personalized or something that scales to hundreds of people simultaneously, but rarely both. A seismic change is afoot that will dwarf the previous chasm, like the shift from black and white film to color cinema. Multimodal AI is poised to eliminate the joint scaling and personalization limitation, enabling truly multidimensional, adaptive experiences where each person experiences something completely unique, all generated in real time. Multimodal AI—machine learning models that can process and integrate information from multiple modalities, like text, images, audio, and video—will fundamentally reshape not just the types of experiences designers create, but how they work. Designers who can orchestrate these AI systems will create the future of multidimensional experiences, realizing true personalization at scale. HOW MULTIMODAL AI WILL REVOLUTIONIZE DESIGN Close your eyes and imagine two people walking through the same physical space—an immersive entertainment activation—and they are each having a unique, hyper-personalized visit across every dimension. Through interfaces like smartphones, wearable devices, and embedded sensors throughout the space, the environment adapts in real time to each individual. That includes the visuals, sounds, narrative, and digital interactions. Subscribe to the Daily newsletter.Fast Company's trending stories delivered to you every day Privacy Policy | Fast Company Newsletters Multimodal AI can simultaneously “see” your facial expressions, “hear” your voice tone, “read” your text inputs, and “observe” your movement patterns. It weaves all this information together to make intelligent decisions about how to personalize your experience in real time. Las Vegas Sphere demonstrates early-stage capabilities with its 170,000-speaker Holoplot audio system that creates distinct sonic zones with surgical precision. Visitors standing just feet apart can experience completely different sounds, tones, intensities, or narrative perspectives of the same content. Multimodal AI will take this capability a step further by enabling even more distinct, individualized sonic experiences based on person, as opposed to a zone. The level of personalization sophistication will ultimately depend on the available interface capabilities. Achieve basic personalization through smartphone apps and existing displays, much like current museum audio guides that offer different language options. More immersive personalization may require wearable tech like alternative reality glasses or advanced earbuds that can overlay completely different visual and audio experiences for each user. The future promises even more seamless interfaces, with the rumored Jony Ive-Sam Altman device potentially enabling contextually aware, screenless interactions that respond to gesture, voice, and environmental cues with minimal technology barriers. THE RISE OF THE “UBER DESIGNER” Creating these AI-powered ultra-personalized immersive experiences requires designers to fundamentally change how they work. This evolution creates what I call the “uber designer,” creative professionals who direct AI systems across multiple modalities to craft unified, adaptive experiences. The uber designer becomes the conductor enabling experiences that account for every element while AI, alongside specialized design teams, handles the execution of countless personalized versions. This technological shift will represent an elevation into higher-order creative leadership. AI manages routine execution and personalization at scale, while humans focus on strategic vision, storytelling, creative judgment, and orchestrating the overall experience architecture. STAYING AHEAD: A DESIGNER’S SURVIVAL GUIDE This isn’t some distant future. Designers need to adapt now. The designers who position themselves now as AI orchestrators for immersive experiences will define the next generation of physical spaces, from retail environments that adapt to each shopper to museums where exhibits personalize to visitor interests. Some industry leaders have begun integrating AI within elements of their brand concepts. Beauty brands like L’Oreal and Sephora have released versions of an AI assistant allowing customers to “try on” beauty products before they make a final purchase or to analyze their skin. Bloomberg Connects has leveraged AI to enhance museum accessibility for visually impaired visitors within an immersive audio guide accessible through a digital app. “The Sphere Experience” enables guests to converse at length with an AI humanoid robot. Leveraging multimodal AI, designers will be able to expand these experiences even further into multiple dimensions, impacting sound, sight, touch, and smell all at once. advertisement So how do you become an uber designer? Designers can strengthen their toolkit in various ways, but here’s my advice: Start integrating AI into workflows now. Begin incorporating AI tools into daily practice. There are various administrative tasks that AI can handle with minimal oversight. Learn how to effectively prompt, direct, and refine AI-generated content. Develop fluency in multiple AI platforms to understand their strengths and limitations. Develop cross-disciplinary thinking. The most valuable designers will orchestrate experiences across every single dimension and not just specialize in one. Move from a “maker” mindset to that of an “experience conductor.” Emphasize the modernization of existing spaces. The biggest opportunities lie in reimagining stagnant industries like retail stores, museums, and entertainment venues, with AI-powered personalization that creates the ultimate multidimensional experiences. Multimodal AI will enable designers to envision even more impactful spaces and experiences that move, inspire, and connect with people. Those who start experimenting now and make an emphasis to revitalize stagnant industries will find themselves at the forefront of a creative renaissance. Humans will be directing machines to create immersive experiences we never thought possible. Andrew Zimmerman is CEO and cofounder of Journey.
