Apple is rolling out new AI-powered features for its Photos app, aiming to give users enhanced editing capabilities without compromising the integrity of their memories. While competitors are enabling drastic photo alterations, Apple's approach, led by iPhone camera chief Jon McCormack, emphasizes a more deliberate and controlled use of generative AI.
At its recent Worldwide Developer Conference, Apple unveiled several AI enhancements for the Photos app in iOS 27. Building on the existing Clean Up tool, these new features include "Extend" and "Spatial Reframe." Extend allows users to expand the canvas of a photo by generating new pixels, effectively adding more space around the subject. Spatial Reframe, on the other hand, enables users to alter the perspective of an image. Both features employ AI to "think" about what should be in the expanded or reframed areas and generate those pixels.
McCormack highlighted that these tools are designed to solve common photographic challenges and empower everyday users with capabilities previously only available in professional editing software. He stated, "You don't have to know all the details of how to do something in Photoshop or something else—it gives normal people these absolute superpowers." However, Apple is imposing restrictions to maintain authenticity. The AI-generated pixels are primarily confined to backgrounds and do not alter the subject's face. The Extend feature has a one-time limit, expanding images by only 25 percent to prevent infinite AI manipulation. Additionally, Apple plans to integrate Google DeepMind's SynthID technology to embed invisible watermarks in AI-altered images, signaling a commitment to transparency.
Apple's philosophy, as articulated by McCormack, centers on preserving the "sanctity of the moment" captured in photographs, viewing them as a way to memorialize life. This contrasts with competitors like Google, which offer more leniency in altering images to match a user's memory or desired aesthetic. Apple's new tools are specifically designed to address compositional flaws made during the photo capture process, such as an unwanted background element or suboptimal framing.
The company's product managers emphasized rigorous AI model training to minimize "hallucinations" or inaccuracies in generated content. Della Huff, product manager for Camera and Photos software, explained that the AI aims to make plausible additions, like another car on a street scene if it fits the context, but avoids generating elements that are entirely out of place. While the AI is designed to be conservative, there have been instances in testing where it added background people in the Extend feature, a decision justified by the need to maintain visual consistency when expanding a scene that already contained people.
Usage of these new features will be capped, with Apple requiring an iCloud subscription for more extensive daily use. This move aligns with Apple's broader WWDC theme of enabling natural language interactions, though photo editing via natural language is currently not supported, with Apple deeming it too complex for current AI capabilities. Siri is being integrated more deeply into the Camera app, enhancing its utility as a note-taking device or information-gathering tool, but not for direct AI photo editing. Apple also differentiates its more controlled Photos app features from its Image Playground app, which is explicitly designed for creative experimentation with AI-generated imagery.
