Modeling physical change for more realistic image editing

About

AI can replace objects, change the weather and transform entire scenes from a short text prompt. Yet edits involving physical processes, such as toasting bread or bending light through water, remain far more difficult because they depend on changes that unfold over time.

Led by researchers at King Abdullah University of Science and Technology (KAUST), in collaboration with the Chinese University of Hong Kong and Hugging Face, a new image editing framework called PhysicEdit learns these physical transitions to produce more physically realistic image edits.

Place a straw into a glass of water and it appears bent because light changes direction as it passes from air into water. Toasting a slice of bread gradually changes its color and texture as heat alters its composition. These transformations happen through continuous physical processes rather than instant changes.

Today's image editing models typically learn by mapping a source image directly to an edited image. While effective for many editing tasks, this approach captures only the beginning and end of a transformation, leaving out the physical changes that connect the two states.

The KAUST team approached the problem differently. Rather than treating image editing as a direct mapping between two images, the researchers reformulated it as the prediction of physical state transitions. Instead of learning only where a transformation begins and ends, PhysicEdit learns how it unfolds. The research was accepted to the International Conference on Machine Learning (ICML) 2026.

"Most existing image editing models learn from static before-and-after images," says Mohamed Elhoseiny, associate professor of computer science at KAUST. "Many edits, however, involve physical processes that unfold over time. By learning these transitions, the model generates results that better reflect how the physical world behaves."

The idea builds on a simple observation: a single image captures a moment, while a video records how that moment evolves. To expose the model to these missing dynamics, the researchers developed PhysicTran38K, a dataset containing more than 38,000 annotated transition trajectories across five categories of physical change: mechanics, optics, thermal processes, material states and biology. Learning from these trajectories enables the model to recognize patterns of physical change and apply them when editing a single image.

PhysicEdit combines reasoning about physical processes with implicit visual transition modeling. Before generating an edited image, the model first analyzes how the scene is expected to evolve under the given instruction. It then encodes the predicted dynamics into latent visual transition representations, which guide the image editing process toward more physically plausible results.

"Image editing becomes more realistic when AI can model how physical processes evolve over time," says first author Liangbing Zhao, a doctoral student at KAUST. "By learning from physical state transitions, we wanted to give the model stronger prior knowledge of these transformations before generating an edited image."

Across established benchmarks for physics-aware image editing, PhysicEdit achieved state-of-the-art performance among open-source image editing models. It produced more physically consistent results for edits involving light refraction, material deformation and object motion.

Beyond image editing, the research introduces a new way of teaching generative AI about the physical world. By learning how physical processes evolve rather than only how they begin and end, future AI systems could generate more realistic visual content and support applications where understanding physical dynamics is essential.