OpenAI, a research organization with some of the most advanced artificial intelligence models in the world, has released a new ChatGPT application for Apple's augmented reality headwear device, Vision Pro.

The new application allows users to chat with OpenAI's GPT-4 Turbo model, which is their latest and most powerful natural language processing system. Users can ask questions, get answers, receive suggestions, learn about new topics, and even generate images and text directly within the application.

A Glimpse into the Future of Human-Computer Interaction

The release of ChatGPT for Vision Pro marks a significant milestone for OpenAI in their mission to create beneficial artificial intelligence for humans. It provides a glimpse into the future of human-AI interaction: more natural, intuitive, and immersive.

In the future, we will interact with AI assistants by speaking our needs to the application, rather than typing in a chat interface. We may even be able to solve problems by simply looking at them in the real world, such as placing ingredients on a table, looking at them, and asking the AI assistant to suggest a recipe using the available ingredients.

One of the First Applications for visionOS

This released ChatGPT application is one of over 600 new applications developed for visionOS, the operating system powering Apple's Vision Pro. visionOS aims to provide a seamless and immersive experience for users who want to interact with digital content in the real world.

According to Apple, visionOS applications can leverage Vision Pro's features, such as Optic ID, a biometric authentication system using eye tracking and iris recognition; Spatial Audio, a sound system that creates realistic and directional audio effects; and VisionKit, a framework that enables developers to create applications that can see, hear, and speak. It is currently unclear whether the new visionOS ChatGPT application will utilize these advanced features.

The new ChatGPT application is also part of a broader industry push for multimodal AI, which is artificial intelligence capable of processing and understanding inputs from different modes or media, such as text, speech, images, and videos. This means it can recognize and interpret various forms of data, not just a single type, making it more versatile and adaptable to different contexts.

New Ways to Chat with AI using Text, Images, and Speech

The existing versions of ChatGPT already allow chatting using text, images, and speech. On Vision Pro, this could become even easier and more intuitive, enabling users to easily display ChatGPT images for complex problem-solving, such as troubleshooting a car engine that won't start or exploring the food in a refrigerator to plan a meal, or analyzing a math problem by looking at it.

Users can still generate content by describing it to ChatGPT, such as inventing new logos, comics, and realistic scenes. All of this will be done in a more intuitive way, simply by dictating or looking at objects in the real world while wearing the headwear device.

Currently, visionOS users can use ChatGPT for free and have the option to subscribe to ChatGPT Plus for access to GPT-4's features and faster response times. The application can be downloaded from the visionOS App Store, where users can also find hundreds of other customized applications for various purposes and interests, such as creative writing, marathon training, or math tutoring.