Since the launch of ChatGPT by OpenAI in late November 2022, its free version has remained relatively unchanged, using the same large language model (GPT-3.5) and user interface, as well as the same limitations. However, a series of ChatGPT upgrades by OpenAI will significantly improve the user experience with the chatbot.
In a live event on Monday, OpenAI announced major updates to the free version of ChatGPT, unlocking a range of new multimodal features for users, including new and improved voice and image assistance.
1. GPT-4o
The biggest news of the event is the launch of a new flagship large language model (LLM) by OpenAI called GPT-4o. As the name suggests, this model offers GPT-4 level intelligence; however, it comes with several upgrades compared to its predecessor, such as faster performance and multimodal capabilities across text, voice, and vision.
This upgrade not only unlocks cutting-edge features, including new voice modes and expanded language support, but also allows free users to access most of the features that were previously limited to ChatGPT Plus subscribers, who paid $20 per month.
For example, free users can now experience GPT-4 level intelligence, retrieve responses from the web using Bing's ChatGPT Browse, analyze data, chat around photos and documents, use GPTs, and access the GPT Store. Previously, all these features were only available to ChatGPT Plus subscribers.
ChatGPT Plus users continue to enjoy advantages such as five times more capacity than free users, priority access to GPT-4o, and upgrades like the new macOS application. In contrast, when the capacity of GPT-4o reaches its limit, free users will be reverted to GPT-3.5.
According to the company, GPT-4o has been rolled out to ChatGPT Plus and team users, and enterprise users will also be able to use it soon. Free users will also get access to the LLM launch; however, these users will have some usage limitations.
2. New Voice Modes
A notable highlight in the live demo is the upgraded voice mode, which has been enhanced with new video and audio capabilities of GPT-4o. Unlike the previous voice modes, users will now experience more natural conversations. For example, users can interrupt the voice assistant and make it speak in different voices. In the live demo, users made ChatGPT respond in a singing manner or with a dramatic robot voice.
The AI assistant will also be able to provide voice responses using contextual information about the environment. In the demo below, you can hear ChatGPT's voice mode chatting with the user and describing in detail the user's surroundings, such as what clothes they are wearing and where they are sitting.
This can be useful if you need assistance with what you are watching. For example, if you want to know the name of a beautiful building you recently encountered in a new city, or if you want to translate a menu. The live demo includes a section where two people have a conversation in two different languages, and ChatGPT's improved voice mode translates the entire conversation.
The improved voice mode will be launched in alpha version in the coming weeks, and ChatGPT Plus users will have early access to it as the company rolls out the mode more widely.
3. Uploading Screenshots, Photos, and Documents
As mentioned above, GPT-4o allows users to upload images and documents in addition to text to obtain AI insights on desired information. This is a significant improvement as this feature can be considered one of the most useful capabilities that AI chatbots can provide, and previously, it was only available to ChatGPT Plus users. Additionally, so far, the only mainstream free chatbot that offered document upload functionality was Anthropic's Claude.
Users can leverage this feature by uploading PDFs, such as research papers or lengthy legal documents, and ask the chatbot for detailed summaries or questions about the content, to which ChatGPT can provide conversational answers. Users can also use the image feature to assist with processing the content they are viewing, such as mathematical equations on a worksheet or identifying the type of plant they see.
4. More Language Support
GPT-4o now enables ChatGPT to support over 50 languages, according to OpenAI, aiming to make ChatGPT more accessible to global users. However, the expanded language support also benefits everyday users as it aids in the translation capabilities of the chatbot, which have also been improved with the new voice modes.
5. New Browser Interface and Applications
Since its launch, the user interface of ChatGPT has remained unchanged until now. The chatbot now has what OpenAI calls a "simplified look and feel" that is "friendlier and easier to converse with." When users visit the page, they will see a new homepage, message layout, and more, as shown in the screenshot below.
These changes may seem minor, with the biggest difference being a more concise and simplified appearance. Additionally, the company is releasing a new macOS application for ChatGPT Plus users starting today and will make it more widely available in the coming weeks. The company also stated that a Windows version will be released later this year.
6. Access to the GPT Store
GPTs are customized ChatGPT chatbots designed to perform specific tasks. They replaced the ChatGPT plugins in March this year as they are easier for developers to build with, and now users can find over three million GPTs in the GPT Store.
This feature is valuable as it allows users to extend the functionality of ChatGPT and integrate it with some of their favorite applications. For example, Canva has its own GPT, allowing users to design projects through the chatbot. Another example is Wolfram GPT, which assists users with mathematical calculations within the chatbot interface.
