Exploring Apple Intelligence and its Smarter Siri

2024-08-01

The AI we see on the iPhone is a series of puzzle pieces that hint at a larger picture.

Once you enable Apple Intelligence (which is now available in developer beta versions of iPhone 15 Pro and Pro Max), a startup screen will reintroduce you to this virtual assistant. When the screen edges light up, you'll know Siri is listening, which clearly indicates that something unusual is happening.



Siri's major AI update will take several months. This version brings significant improvements to language understanding, but future updates will add more features, such as understanding the content displayed on the screen and the ability to act on your behalf. Meanwhile, other parts of the Apple Intelligence feature set previewed in this update feel like a party waiting for VIPs to arrive.



That being said, Siri's improvements in this update are still useful. Double-tapping the bottom of the screen brings a new way to interact with the assistant: through text. It is also better at parsing natural language, being more patient with my hesitations and "um" sounds, while I stutter to ask a question. It also understands when I'm asking follow-up questions.





Aside from Siri, finding traces of Apple Intelligence in the operating system is like searching for Easter eggs. They are in the Mail app, where there is now a summary button at the top of each email. Anywhere you can input and highlight text, you'll find a new option called "Writing Tools," which includes AI proofreading, writing suggestions, and summaries.


"Help me write something" is now a common feature for generative AI, and Apple Intelligence does it well too. You can have it make your text friendlier, more professional, or concise. You can also create summaries of the text or synthesize it into bullet points or tables.


I find these tools particularly useful in the "Notes" app, as you can now add voice recordings to it. In iOS 18, voice recordings finally have automatic transcription, but that's not an Apple Intelligence feature, as it also works on my iPhone 13 Mini. However, Apple Intelligence allows you to transcribe the recordings into summaries or lists. If you want to freely brainstorm and list a bunch of items you need to pack for an upcoming trip while recording a memo, this will be very helpful; Apple Intelligence will turn it into a meaningful list.





These writing tools are hidden, and you could easily miss them if you're not actively looking. The more obvious new AI feature is in the Mail app. Apple Intelligence displays important emails it deems as cards at the top of the inbox, marked as priority. Below that, emails show short summaries instead of the first line or two of text you usually see.


There is a charming feeling when AI attempts to summarize promotional emails, trying to extract useful details like "Free delivery on backpacks and lunch boxes" and "Organic white peaches, sweet and juicy, now available." The descriptions in the inbox are accurate - useful in a few cases and harmless in the worst. And the emails given priority are indeed important, which is promising.


The search tool in the Photos app now uses AI to understand more complex requests. You can ask to see photos of someone wearing glasses in natural language or all the photos of food you've eaten in Iceland.


Despite the light show, Siri remains the same as before. It works very well, returns results quickly, and is generally reliable. It found the photo of the child wearing funny glasses that I had in mind, although it also showed other photos of him with other people wearing glasses. However, I think this is something people will immediately get used to and use without hesitation - intuitive and obviously useful.


It is primarily still a "let me Google that for you" machine. The most important updates are still to come, when Siri will be able to understand the content on your screen and perform actions on your behalf within apps. In theory, you'll be able to have Siri extract information from messages and convert it into calendar events or retrieve information from emails without having to search your inbox personally.


This is what I'm most looking forward to, as so far, all parts of Apple Intelligence could be the foundation for building a better Siri. Apple's AI can understand the content of emails or photos. Similarly, Siri is better at understanding human conversation. For Apple Intelligence to prove its worth, Siri needs to connect these dots.