Apple Unveils Its Augmented Reality/Virtual Reality Headset, Apple Vision Pro

Apple unveiled its augmented reality/virtual reality headset today at its Worldwide Developer Conference. It’s called Apple Vision Pro, not “RealityPro” or “RealityOne,” as some pundits predicted.

It’s also more innovative and more expensive than most folks anticipated. As described by Apple, the Apple Vision Pro is “a revolutionary spatial computer that seamlessly blends digital content with the physical world, while allowing users to stay present and connected to others.”

The headset won’t arrive until 2024. And it will cost US$3,499 (even more, if you wear eyeglasses), so start saving your moolah.

Here’s how Apple describes the Vision Pro: “It creates an infinite canvas for apps that scales beyond the boundaries of a traditional display and introduces a fully three-dimensional user interface controlled by the most natural and intuitive inputs possible — a user’s eyes, hands, and voice. Featuring visionOS, the world’s first spatial operating system, Vision Pro lets users interact with digital content in a way that feels like it is physically present in their space. The breakthrough design of Vision Pro features an ultra-high-resolution display system that packs 23 million pixels across two displays, and custom Apple silicon in a unique dual-chip design to ensure every experience feels like it’s taking place in front of the user’s eyes in real time.

“Today marks the beginning of a new era for computing,” CEO Tim Cook says in the announcement. “Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing. Built upon decades of Apple innovation, Vision Pro is years ahead and unlike anything created before — with a revolutionary new input system and thousands of groundbreaking innovations. It unlocks incredible experiences for our users and exciting new opportunities for our developers.”

“Creating our first spatial computer required invention across nearly every facet of the system,” adds Mike Rockwell, Apple’s vice president of the Technology Development Group. “Through a tight integration of hardware and software, we designed a standalone spatial computer in a compact wearable form factor that is the most advanced personal electronics device ever.”

HERE ARE SOME OF THE SPECIFICS OF THE APPLE VISION PRO, ACCORDING TO APPLE:

° An “infinite canvas.” visionOS features a three-dimensional interface that frees apps from the boundaries of a display so they can appear side by side at any scale. Apple Vision Pro enables users to be even more productive, with infinite screen real estate, access to their favorite apps, and all-new ways to multitask. And with support for Magic Keyboard and Magic Trackpad, users can set up the perfect workspace or bring the powerful capabilities of their Mac into Vision Pro wirelessly, creating an enormous, private, and portable 4K display with incredibly crisp text.

Lots of entertainment possibilities. With two ultra-high-resolution displays, Apple Vision Pro can transform any space into a personal movie theater with a screen that feels 100 feet wide and an advanced Spatial Audio system. Users can watch movies and TV shows, or enjoy stunning three-dimensional movies. Apple Immersive Video offers 180-degree high-resolution recordings with Spatial Audio, and users can access an exciting lineup of immersive videos that transport them to entirely new places.

Spatial computing makes new types of games possible with titles that can span a spectrum of immersion and bring gamers into all-new worlds. Users can also play over 100 Apple Arcade games on a screen as large as they want, with incredible immersive audio and support for popular game controllers.

° Environments. With Environments, a user’s world can grow beyond the dimensions of a physical room with dynamic, beautiful landscapes that can help them focus or reduce clutter in busy spaces.A twist of the Digital Crown lets a user control how present or immersed they are in an environment. 

° 3D Memories. Featuring Apple’s first three-dimensional camera, Apple Vision Pro lets users capture, relive, and immerse themselves in favorite memories with Spatial Audio. Every spatial photo and video transports users back to a moment in time, like a celebration with friends or a special family gathering. Users can access their entire photo library on iCloud, and view their photos and videos at a life-size scale with brilliant color and spectacular detail. Every Panorama shot on iPhone expands and wraps around the user, creating the sensation they are standing right where it was taken.

Spatial FaceTime. With Apple Vision Pro, FaceTime calls take advantage of the room around the user, with everyone on the call reflected in life-size tiles, as well as Spatial Audio, so it sounds as if participants are speaking right from where they are positioned. Users wearing Vision Pro during a FaceTime call are reflected as a Persona — a digital representation of themselves created using Apple’s most advanced machine learning techniques — which reflects face and hand movements in real time. Users can do things together like watch a movie, browse photos, or collaborate on a presentation.

° An app store. The Apple Vision Pro has an all-new App Store where users can discover apps and content from developers, and access hundreds of thousands of familiar iPhone and iPad apps that run great and automatically work with the new input system for Vision Pro. Apple’s developer community can go even further and take advantage of the powerful and unique capabilities of Vision Pro and visionOS to design brand-new app experiences, and reimagine existing ones for spatial computing.

° Revolutionary user interface. visionOS features a brand-new three-dimensional interface that makes digital content look and feel present in a user’s physical world. By responding dynamically to natural light and casting shadows, it helps the user understand scale and distance. To enable user navigation and interaction with spatial content, Apple Vision Pro introduces an entirely new input system controlled by a person’s eyes, hands, and voice. Users can browse through apps by simply looking at them, tapping their fingers to select, flicking their wrist to scroll, or using voice to dictate.

° Eyesight. Apple Vision Pro also features EyeSight, an extraordinary innovation that helps users stay connected with those around them. When a person approaches someone wearing Vision Pro, the device feels transparent — letting the user see them while also displaying the user’s eyes. When a user is immersed in an environment or using an app, EyeSight gives visual cues to others about what the user is focused on.

° Design innovation. A singular piece of three-dimensionally formed and laminated glass is polished to create an optical surface that acts as a lens for the wide array of cameras and sensors needed to blend the physical world with digital content. The glass flows into the custom aluminum alloy frame that gently curves around the user’s face, while the modular system allows for a tailored fit to accommodate a wide range of people. The Light Seal is made of a soft textile, and comes in a range of shapes and sizes, flexing to conform to a user’s face for a precise fit. Flexible straps ensure audio remains close to the user’s ears, while a Head Band — available in multiple sizes — is three-dimensionally knitted as a single piece to provide cushioning, breathability, and stretch.1 The band is secured with a simple mechanism, making it easy to change to another size or style of band.

° Hardware innovation. Featuring a breakthrough ultra-high-resolution display system built on top of an Apple silicon chip, Vision Pro uses micro-OLED technology to pack 23 million pixels into two displays, each the size of a postage stamp, with wide color and high dynamic range. This technological breakthrough, combined with custom catadioptric lenses that enable incredible sharpness and clarity, delivers jaw-dropping experiences. 

Apple Vision uses an M2 processor to deliver standalone performance, while the brand-new R1 chip processes input from 12 cameras, five sensors, and six microphones to ensure that content feels like it is appearing right in front of the user’s eyes, in real time. R1 streams new images to the displays within 12 milliseconds — 8x faster than the blink of an eye. Apple Vision Pro is designed for all-day use when plugged in, and up to two hours of use with its external, high-performance battery.

° Audio and cameras. An advanced Spatial Audio system is core to the Apple Vision Pro experience, creating the feeling that sounds are coming from the environment around the user and matching the sound to the space. Two individually amplified drivers inside each audio pod deliver Personalized Spatial Audio based on the user’s own head and ear geometry.

In addition to creating a breakthrough display and advanced audio experiences, the high-performance eye tracking system in Apple Vision Pro uses high-speed cameras and a ring of LEDs that project invisible light patterns onto the user’s eyes for responsive, intuitive input.

Users with vision correction needs will use ZEISS Optical Inserts to ensure visual fidelity and eye tracking accuracy.

° Privacy and security. Optic ID is a new secure authentication system that analyzes a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, meaning it is not stored on Apple servers. 

Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.

PRICING AND AVAILABILITY

The Apple Vision Pro starts at US$3,499, and will be available early next year on apple.com and at Apple Store locations in the U.S., with more countries coming later next year. Customers will be able to learn about, experience, and personalize their fit for Vision Pro at Apple Store locations. For more information about Vision Pro, visit apple.com/apple-vision-pro.

Apple Introduces The 15.3-Inch Macbook Air

Apple has debuted a 15.3-inch MacBook Air at the Worldwide Developer Conference.

It boasts a 15.3-inch Liquid Retina, the performance of the M2 processor, up to 18 hours of battery life, and a silent, fanless design. What’s more, with an all-new six-speaker sound system, the 15-inch MacBook Air delivers immersive Spatial Audio, along with a 1080p FaceTime HD camera, and MagSafe charging.

According to John Ternus, Apple’s senior vice president of Hardware Engineering, the new MacBook Air’s Liquid Retina display provides up to 500 nits of brightness and support for 1 billion colors. It’s also twice the resolution and 25% brighter than a comparable PC laptop, he adds.

The new MacBook Air measures only 11.5mm thin, which makes it, according to Apple, making it the world’s thinnest 15-inch laptop. It weighs just 3.3 pounds, so is nearly 40% thinner and half a pound lighter than a comparable PC laptop, Ternus says.

The MacBook Air also features MagSafe charging, two Thunderbolt ports for connecting accessories and up to a 6K external display, along with a 3.5mm headphone jack. It comes in four finishes — midnight, starlight, space gray, and silver.

With the M2 chip, the 15-inch MacBook Air is purportedly up to 12x faster than the fastest Intel-based MacBook Air. The laptop features a 8-core CPU with four performance cores and four efficiency cores, a 10-core GPU for fast graphics, and a 16-core Neural Engine. The M2 also delivers 100GB/s of memory bandwidth and supports up to 24GB of fast unified memory.

The 15.3-inch MacBook Air offers a 1080p FaceTime HD camera and three-microphone array for FaceTime and other video calls. The design of the new 15-inch MacBook Air also features a new six-speaker sound system with two tweeters and two sets of force-cancelling woofers. Apple says the new speakers deliver twice the bass depth for fuller sound, and Spatial Audio with support for Dolby Atmos provides immersive experiences whether listening to music or watching movies.

The 15-inch MacBook Air with M2 is available to order today on apple.com/store and in the Apple Store app. It will begin arriving to customers, and in Apple Store locations and Apple Authorized Resellers, beginning Tuesday, June 13.

The 15-inch MacBook Air with M2, available in midnight, starlight, silver, and space gray, starts at US$1,299 and $1,199 for education.

The 13-inch MacBook Air with M2, which comes in the colors midnight, starlight, silver, and space gray, is also now available for $999 for academic institutions. The 13-inch MacBook Air with M1, which starts at $999 and $899 for academic institutions, is still available. It comes in three colors: gold, silver, and space gray.

Apple Updates The Mac Studio And Mac Pro

During Worldwide Developer Conference, Apple announced introduced the new Mac Studio and Mac Pro, the two most powerful Macs ever made. 

The Mac Studio features M2 Max and the new M2 Ultra and is purportedly up to 6x faster than the most powerful Intel-based 27-inch iMac, and up to 3x faster than the previous-generation Mac Studio with M1 Ultra.

The Mac Pro, now featuring M2 Ultra, combines the unprecedented performance of Apple’s most powerful chip with the versatility of PCIe expansion, according to John Ternus, Apple’s senior vice president of Hardware Engineering. The Mac Pro is up to 3x faster than the previous-generation Intel-based model, he adds.

The Mac Studio with M2 Ultra and Mac Pro support up to 192GB of unified memory. The new Mac Pro completes the Mac transition to Apple silicon and, together with the rest of Apple’s pro systems, gives users the most powerful and capable lineup of pro products Apple has ever offered, says Ternus. The Mac Studio and Mac Pro are available to order today, with availability beginning Tuesday, June 13.

MAC STUDIO

Apple says the Mac Studio with M2 Max is up to 50% faster than the previous-generation Mac Studio and 4x faster than the most powerful Intel-based 27-inch iMac. It features a 12-core CPU, up to a 38-core GPU, and up to 96GB of unified memory with 400GB/s of memory bandwidth.

Apple says that, when compared to Mac Studio with M1 Max, Mac Studio with M2 Max:

• Empowers motion graphics designers using After Effects to render up to 50% faster.
• Enables developers to build new versions of apps with Xcode at blazing speed, with up to 25% faster performance.
• Is up to 3x faster than the previous-generation Mac Studio with M1 Ultra, and up to 6x faster than the most powerful Intel-based 27-inch iMac.
• Features a 24-core CPU, up to a 76-core GPU, and up to 192GB of memory with 800GB/s of unified memory bandwidth for workstation-class performance.

The new Mac Studio now has higher-bandwidth HDMI, enabling up to 8K resolution and 240Hz frame rates. With the M2 Ultra, Mac Studio supports up to six Pro Display XDRs — driving over 100 million pixels — allowing for a vast amount of screen real estate for pro workflows. Additionally, it now features built-in wireless technologies. Wi-Fi 6E delivers download speeds that are up to twice as fast as the previous generation, while Bluetooth 5.3 allows users to connect to the latest Bluetooth accessories. On the back, the Mac Studio includes four Thunderbolt 4 ports, a 10Gb Ethernet port, an enhanced HDMI port, and two USB-A ports. It also has two USB-C ports and an SD card slot on the front.

MAC PRO

Ternus says that now every Mac Pro has Apple’s most powerful 24-core CPU, an up to 76-core GPU, and starts with twice the memory and SSD storage. The new Mac Pro can also be configured with up to 192GB of memory with 800GB/s of unified memory bandwidth. According to Apple, when compared to the Intel-based Mac Pro, Mac Pro with M2 Ultra:

• Empowers demanding real-world pro workflows like video transcoding and 3D simulations to run up to 3x faster.3
• Enables video engineers to ingest 24 4K camera feeds and encode them to ProRes in real time, all on a single machine, when using six video I/O cards.5

The new Mac Pro brings PCIe expansion to Apple silicon for pros who want the performance of M2 Ultra and rely on internal expansion for their workflows. It boasts seven PCle expansion slots, with six open expansion slots that support gen 4, which is 2x faster than before, so users can customize Mac Pro with essential cards. 

The array of connectivity in Mac Pro now sports eight built-in Thunderbolt 4 ports — six on the back and two on the top — which is twice as many as before. It supports up to six Pro Display XDRs, along with Wi-Fi 6E and Bluetooth 5.3, which come to the new Mac Pro for fast wireless connectivity. 

Additionally, the desktop includes three USB-A ports, two higher-bandwidth HDMI ports that support up to 8K resolution and up to 240Hz frame rates, two 10Gb Ethernet ports, and a headphone jack that enables the use of high-impedance headphones.

PRICING AND AVAILABILITY

The new Mac Studio and Mac Pro are available to order today on apple.com/store and in the Apple Store app. They’ll start to arrive to customers, and will be available in Apple Store locations and Apple Authorized Resellers, beginning Tuesday, June 13.

The Mac Studio starts at US$1,999 and $1,799 or education. Additional configure-to-order options are available at apple.com/store.

Available in both tower and rack-mounted enclosures, Mac Pro (Tower Enclosure) starts at $6,999 and $6,599 for education. Mac Pro (Rack Enclosure) starts at $7,499 (U.S.) and $6,999 (U.S.) for education. Additional technical specifications and configure-to-order options are available at apple.com/mac-pro.

macOS Sonoma Gets Widget Tweaks, Gaming Features And More

Apple has previewed macOS Sonoma, the latest version of its operating system, at this week’s Worldwide Developer Conference.

It brings more iOS and iPadOS features to the operating system. Users can now place widgets right on the desktop, interact with them with just a click, and, thanks to Continuity, access the ecosystem of iPhone widgets on their Mac. 

In macOS Sonoma, video conferencing also gets new features to help users present remotely, like Presenter Overlay, which places a presenter on top of the content being shared, and Reactions, which enables gesture-triggered video effects in cinematic quality. 

Updates come to Safari. For example, the Profiles feature keeps browsing separate between multiple topics or projects, and web apps provide faster access to favorite sites. And Apple says that gaming gets even better, with the introduction of Game Mode, exciting new titles, and a new game porting toolkit that makes it even easier for developers to bring more games to Mac.

b

Now, tens of millions of Macs with Apple silicon can run demanding games with great performance, long battery life, and breathtaking visuals, according to Craig Federighi, Apple’s senior vice president of Software Engineering. He says that developers continue to take advantage of Metal 3, bringing exciting new titles to Mac, including DEATH STRANDING DIRECTOR’S CUT, Stray, Fort Solis, World of Warcraft: Dragonflight, HUMANKIND, Resident Evil Village: Winters’ Expansion, The Medium, ELEX II, Firmament, SnowRunner, Disney Dreamlight Valley, No Man’s Sky, Dragonheir: Silent Gods, and Layers of Fear. 

To make it easier to port games from other platforms to Mac, Metal introduces a new game porting toolkit that Apple claims eliminates months of upfront work and enables developers to see how well their existing game could run on Mac in just a few days. The tech giant says it also dramatically simplifies the process of converting the game’s shaders and graphics code to take full advantage of Apple silicon performance, significantly reducing the total development time.

macOS Sonoma also introduces Game Mode, giving players an edge when performance is measured in precious milliseconds, according to Federighi. He says that Game Mode delivers an optimized gaming experience with smoother and more consistent frame rates, by ensuring games get the highest priority on the CPU and GPU. 

Game Mode also makes gaming on Mac even more immersive — dramatically lowering audio latency with AirPods, and significantly reducing input latency with popular game controllers like those for Xbox and PlayStation by doubling the Bluetooth sampling rate. Game Mode works with any game, including all of the recent and upcoming Mac games.

WIDGET ENHANCEMENTS

According to Federighi, widgets are now even more powerful and personal on Mac. Users can place widgets right on the desktop and access the widget gallery to find the ones they want.

Widgets blend seamlessly with the wallpaper while users work in apps, so they can stay focused on the task at hand, Federighi says. Thanks to Continuity, users can also enjoy the vast ecosystem of iPhone widgets on their Mac. And widgets become interactive, allowing users to check off reminders, play or pause media, access home controls, and perform various tasks from their Mac, all directly from the desktop.

SMARTER VIDEO CONFERENCING EXPERIENCES

macOS Sonoma brings enhanced video conferencing features that enable users to present and share their work more effectively within any video conferencing app, according to Federighi. Presenter Overlay, a new video effect, elevates a user’s presence by displaying them on top of the content they are sharing. 

Additionally, Reactions allow users to share how they feel by adding balloons, confetti, hearts, and more into the video, which can also be triggered with a hand gesture. The improved Screen Sharing picker is designed to simplify the process of sharing apps during video calls. Users can click the green button in the top-left corner of an app and choose to share it in the call, allowing for content sharing from their currently open windows.

SAFARI UPDATES

Safari introduces new features to enhance the browsing experience for Mac users. Federighi says this year brings a significant update to Private Browsing, which provides even greater protection during browsing both from trackers and from people who might have access to the user’s device. 

He adds that advanced tracking and fingerprinting protections in Private Browsing go even further to help prevent websites from tracking or identifying the user. Private Browsing locks when users aren’t using them, allowing them to keep tabs open even when they step away from their device.

Additionally, profiles help users stay organized by offering a way to separate browsing between topics, while also keeping cookies, history, extensions, Tab Groups, and Favorites separate. Users can sign in to the same site with both work and personal accounts — and switch between them — ensuring a smooth browsing experience, Federighi says. Safari also enables the creation of web apps that behave like normal apps, putting users’ favorite sites at their fingertips and providing a simplified toolbar for an app-like experience.

NEW SCREEN SAVERS

macOS Sonoma provides screen savers featuring slow-motion videos of various locations worldwide, such as the skyline of Hong Kong, the sandstone buttes of Monument Valley in Arizona, and the rolling hills of Sonoma in Northern California. This photographic imagery is designed to enhance the Mac experience by shuffling through Landscape, Earth, Underwater, or Cityscape themes. 

The login experience is now repositioned at the bottom of the screen, making space for the new screen savers, which transition into the desktop.

HYBRID AND REMOTE PRO WORKFLOWS

Leveraging the advanced media engine in Apple silicon, macOS Sonoma brings a new high performance mode to the Screen Sharing app, Federighi says. Enabling responsive remote access to hybrid in-studio and remote pro workflows, it delivers low-latency audio, high frame rates, and supports up to two virtual displays. 

Federighi says this mode empowers pros to securely access their content creation workflows from anywhere — whether editing in Final Cut Pro or DaVinci Resolve, or animating complex 3D assets in Maya. With support for reference color, it also enables remote color workflows that Apple says traditionally couldn’t be performed without dedicated hardware and specialized software.

ENHANCED ACCESSIBILITY FEATURES

macOS Sonoma introduces a range of accessibility features designed to make the Mac even more customizable for all users. For customers with hearing disabilities, Made for iPhone hearing devices can connect to their Mac for calls and media consumption, while users who are nonspeaking can use Live Speech to type and vocalize their thoughts during calls and conversations. 

For users with physical and motor disabilities, phonetic suggestions appear when dictating and editing text with Voice Control on Mac. To assist users with cognitive disabilities, animated images like GIFs can be automatically paused in Messages and Safari. Additionally, users who are blind or have low vision can customize text size across Mac apps, and utilize Xcode with VoiceOver, Apple’s industry-leading screen reader.

ACCORDING TO APPLE, ADDITIONAL MACOS SONOMA UPDATES INCLUDE:

• Streamlined PDFs:Enhanced PDF functionality allows for quick form-filling with AutoFill, and smart recipient recommendations.
• Inline PDFs in Notes:Notes now displays PDFs and document scans in full width, and with linked notes, users can quickly connect related notes like recipes or homework.
• Siri:Users have the ability to simply say “Siri” to activate Siri. 2
• Passwords: Users can now create a group to share a set of passwords. Everyone in a group can add and edit passwords to keep them up to date, and since sharing is through iCloud Keychain, it’s end-to-end encrypted. Additionally, the one-time verification codes received in Mail will now autofill in Safari, making it easy to securely log in without leaving the browser.
• Messages: Users can connect with those closest to them with an all-new stickers experience, enhancements to features like search, reply, groups, and syncing with Messages in iCloud®.
• Reminders:Intelligent grocery lists in Reminders streamline weekly trips to the store. Additionally, users can organize lists into sections and arrange them horizontally using a new column view.
• Keyboard: An all-new autocorrect makes corrections more accurate and easier to fix. Inline completions help users quickly finish sentences, while Dictation brings accuracy improvements through next-level speech recognition.
• Privacy and Safety: Communication Safety expands to AirDrop, the Photos picker, incoming calls, and FaceTime messages, further broadening protections for children. Additionally, Sensitive Content Warning prevents unexpected exposure to sensitive images and videos in AirDrop and Messages, incoming calls, and FaceTime messages, while giving the user the option to block the contact or seek further resources for help.

AVAILABILITY

The developer beta of macOS Sonoma is available through the Apple Developer Program at developer.apple.com starting today, and a public beta will be available through the Apple Beta Software Program next month at beta.apple.com. The release will be available as a free software update this fall. For more information, visit apple.com/macos/sonoma-preview. 

iOS 17 Updates FaceTime, Messages And More

Apple has announced iOS 17, upgrading the communications experience across Phone, FaceTime, and Messages; makes sharing even easier with AirDrop; and provides more intelligent input that improves the speed and accuracy of typing. 

iOS 17 also introduces new experiences with Journal, an app that makes it easy for people to practice gratitude, and StandBy, a new way to view glanceable information when iPhone is set down and charging, according to Craig Federighi, Apple’s senior vice president of Software Engineering.

The Phone app is central to the iPhone experience, and it receives a big update with personalized Contact Posters, providing a new way for users to express themselves. Users can customize how they appear, bringing a completely new look to incoming calls, and choose treatments for photos or Memoji, as well as typography and font colors. Contact Posters will also be available for third-party calling apps.

Live Voicemail gives users the ability to see real-time transcription as someone leaves a voicemail, and the opportunity to pick up while the caller is leaving their message. Calls identified as spam by carriers won’t appear as Live Voicemail, and will instead be instantly declined. With the power of the Neural Engine, Live Voicemail transcription is handled on-device and remains entirely private, according to Federighi.

FACETIME CHANGES

FaceTime now supports audio and video messages so when users call someone who is not available, they can share a message for later.

FaceTime calls also get more expressive with Reactions such as hearts, balloons, fireworks, laser beams, rain, and more. The new effects can be activated through gestures, and third-party video calling apps can take advantage of them as well.

In an update to Apple TV 4K, FaceTime now extends to television sets. Powered by Continuity Camera, users can initiate a video call directly from Apple TV, or start the call on iPhone and then hand it off to Apple TV, to see friends and family on their television. With Center Stage, users will have perfect framing even as they move around the room, according to Federighi.

MESSAGES UPDATES

Messages gets significant updates with iOS 17, including an all-new stickers experience with new emoji stickers and the ability to create Live Stickers by lifting subjects from photos. Users can also add effects to Live Stickers that bring them to life, and a new drawer in the keyboard puts all of a user’s stickers in one place for access across iOS.

An expandable menu that can be accessed with a tap to display iMessage apps gives Messages a sleeker look, says Federighi. Search gets more powerful and precise with search filters, making it possible for users to start a search and add additional filters to narrow down the results and find exactly what they are seeking, he adds. 

A new catch-up arrow indicates where the user last left off in the conversation, and replying inline is as simple as swiping on a text bubble. If a user shares their location, it will update in real time right in the conversation. When sending an audio message, it’s transcribed automatically so users can read it in the moment or listen later.

Messages also introduces Check In, a feature for when a user wants to notify a family member or friend that they have made it to their destination safely. After a user initiates a Check In, their friend or family member will automatically be notified as soon as the user arrives.

If they’re not making progress toward their destination, information will be temporarily shared with the selected contact, such as the device’s location, battery level, and cell service status. Apple says all information that’s shared is end-to-end encrypted.

SHARING WITH AIRDROP AND NAMEDROP

AirDrop makes it easy to share a file with a colleague or send photos to a friend in seconds, and with iOS 17, AirDrop gets new ways to share. NameDrop allows users to share contact information by simply bringing their iPhones together, or by bringing an iPhone and Apple Watch together. With the same gesture, users can also share content or start SharePlay to listen to music, watch a movie, or play a game while in close proximity between iPhone devices.

IMPROVEMENTS TO AUTOCORRECT AND DICTATION

In iOS 17, autocorrect receives an update with a transformer language model, a state-of-the-art on-device machine learning language model for word prediction. This purportedly improves the experience and accuracy for users every time they type. It also receives a refreshed design to better support typing, and sentence-level autocorrections can fix more types of grammatical mistakes. 

Users will now receive predictive text recommendations inline as they type, so adding entire words or completing sentences is as easy as tapping the space bar, making text entry faster than ever. Dictation leverages a new speech recognition model to make it even more accurate, says Federighi.

STANDBY’S GLANCEABLE INFO

iOS 17 introduces StandBy, giving users a full-screen experience with glanceable information designed to be viewed from a distance when iPhone is on its side and charging. The feature can be personalized to display a range of beautiful clock styles, favorite photos, or widgets, including Smart Stacks, which “surface the right widgets at the right time.”

With support for Live Activities, Siri, incoming calls, and larger notifications, StandBy is designed to make the iPhone even more useful when viewed at a distance. When charging with MagSafe, the feature remembers a user’s preferred view. Your can bring up StandBy anytime by tapping the screen, and on iPhone 14 Pro with the Always-On display, it is always available.

NEW JOURNAL APP 

Journal is a new app designed to help iPhone users reflect and practice gratitude through journaling. Using on-device machine learning, personalized suggestions can be provided to help inspire a user’s journal entry. 

Suggestions are intelligently curated from recent activity, including photos, people, places, workouts, and more. Scheduled notifications can help build a journaling habit. 

With the ability to lock the app, the use of on-device processing, and end-to-end encryption, Journal is built to protect a user’s privacy and ensure no one — including Apple — can access a user’s entries, according to Federighi.

APPLE SAYS THAT ADDITIONAL FEATURES IN IOS 17 INCLUDE:

• Safari adds greater protection for Private Browsing, both from trackers as a user browses, and from people who might have access to a user’s device. Advanced tracking and fingerprinting protections go even further to help prevent websites from tracking or identifying a user’s device. Private Browsing now locks when not in use, allowing a user to keep tabs open even when stepping away from the device.
• For easier and more secure password and passkeys sharing, users can share passwords with a group of trusted contacts. Everyone in the group can add and edit passwords to keep them up to date. Since sharing is through iCloud Keychain, it’s end-to-end encrypted.
• The Health app offers new mental health features. Users can log their daily moods and momentary emotions; see what might be contributing to their state of mind; and easily access depression and anxiety assessments often used in clinics, plus resources available in their region. Additionally, increasing the distance the device is viewed from can help children lower their risk of myopia and gives adult users the opportunity to reduce digital eyestrain. Screen Distancein Screen Time uses the TrueDepth camera to encourage users to move their device farther away after holding it closer than 12 inches from their face for an extended period of time.
• Maps adds offline maps, so users can download a specific area and access turn-by-turn navigation, see their estimated time of arrival, find places in Maps, and more while offline. Maps also makes it easier than ever to discover thousands of trails in parks across the United States, and supports electric vehicle drivers with real-time charging availability information.
• AirTag can be shared with up to five other people, allowing friends and family to keep track of an item in Find My. Everyone in a group will be able to see an item’s location, play a sound, and use Precision Finding to help pinpoint the location of a shared AirTag when nearby. This also works with all other Find My network accessories.
• Apple Music introduces Collaborative Playlists that make listening to music with friends easier than ever before, and SharePlay in the car allows all passengers to easily contribute to what’s playing. Listeners can control the music from their own devices, even if they don’t have an Apple Music subscription.
• Sharing content using AirPlay is even easier with on-device intelligence now learning a user’s preferences. AirPlay will also work with supported televisions in hotels, allowing users to easily enjoy their favorite content on the TV when traveling. Built with a foundation of privacy and security, this capability will be available before the end of the year in select hotels, starting with brands from IHG Hotels & Resorts.
• AirPods receive powerful new features, including Adaptive Audio, Personalized Volume, and Conversation Awareness, that redefine the personal audio experience. Plus, improvements to Automatic Switching and call controls make AirPods even easier to use.
• The Home app adds the ability for users to view up to 30 days of activity history across door locks, garage doors, alarm systems, and contact sensors. Additionally, two popular HomeKit lock features — tap to unlock and PIN codes — are now available for Matter-compatible locks, providing even more ways to connect the home.
• Reminders features a grocery list that automatically groups added items into categories to make shopping easier. Users can change how the items are grouped and the list remembers their preferences.
• Visual Look Up is now available in paused video frames. Now users can identify food, storefronts, signs, and symbols, and lift individual subjects from photos and videos.
• Siri can be activated by simply saying “Siri.” Once activated, users can issue multiple commands in succession without needing to reactivate the assistant.
• In Photos, the People album uses on-device machine learning to recognize more photos of a user’s favorite people, as well as cats and dogs.
• Privacy updates include the expansion of Communication Safety beyond Messages to help keep kids safe when sending and receiving content via AirDrop, Contact Posters, a FaceTime message, and when using the Photos picker to choose content to send. It also expands to cover video content in addition to still images. A new feature, Sensitive Content Warning, helps adult users avoid seeing unwanted nude images and videos. As with Communication Safety, all image and video processing for Sensitive Content Warning occurs on-device, so Apple does not get access to the content.
• Accessibility updates include Assistive Access, a customizable interface that helps users with cognitive disabilities use iPhone with greater ease and independence; Live Speech, which gives nonspeaking users the option to type and have their words spoken in person, or on phone and FaceTime calls; Personal Voice, which gives users at risk of speech loss the option to create a voice that sounds like theirs; and Point and Speak, which helps users who are blind or have low vision read text on physical objects by pointing.

The developer beta of iOS 17 is available to Apple Developer Program members at developer.apple.com starting today, and a public beta will be available next month at beta.apple.com. New software features will be available this fall as a free software update for iPhone XS and later. For more information, visit apple.com/ios/ios-17-preview.