MIT tool allows creation and manipulation of "non-existent" objects in a virtual environment
In the heart of Massachusetts Institute of Technology (MIT), researchers at the Computer Science and Artificial Intelligence Laboratory (CSAIL) have made a groundbreaking development. They have created a tool called Meschers, which revolutionises the way impossible objects are represented and rendered.
The work on Meschers was supported by various fellowships, grants, and collaborations, including the MIT Presidential Fellowship, the Mathworks Fellowship, the Hertz Foundation, the U.S. National Science Foundation, the Schmidt Sciences AI2050 fellowship, MIT Quest for Intelligence, the U.S. Army Research Office, U.S. Air Force Office of Scientific Research, SystemsThatLearn@CSAIL initiative, Google, the MIT-IBM Watson AI Laboratory, from the Toyota-CSAIL Joint Research Center, Adobe Systems, the Singapore Defence Science and Technology Agency, and the U.S. Intelligence Advanced Research Projects Activity.
Meschers converts images and 3D models into 2.5-dimensional structures, creating Escher-like depictions of things like windows, buildings, and even donuts. It represents these impossible objects with screen-space vertex coordinates (x, y) representing the 2D positions on the image, relative depth values assigned per edge (differences in z between neighboring vertices) encoding how parts relate in depth locally, and a typical manifold mesh topology of vertices, edges, and faces, but where depth is relative and locally consistent.
This representation allows for rendering the impossible objects with optical illusions intact, subdivision and smoothing of mesh geometry to fine-tune visual features, use in geometry research to calculate properties on curved impossible surfaces, and enabling artists and computer graphics creators to edit, explore, and produce Escher-like designs using familiar graphic tools with novel impossible shapes.
By decoupling local depth relations from impossible global shapes, Meschers creates a flexible framework to visualise, interact with, and analyse objects that cannot physically exist but can be optically rendered to preserve their illusions and deepen understanding of complex geometry.
Meschers also uses "inverse rendering" tools to convert drawings and images of impossible objects into high-dimensional designs. It allows creatives to tweak the shading of their creations while preserving an optical illusion, enabling a wider variety of scenes like sunrises or sunsets.
The researchers will present their paper at the SIGGRAPH conference in August. Meschers demonstrated its versatility by relighting a model of a dog on a skateboard. The paper on Meschers was written by Dodik, Solomon, Isabella Yu '24, SM '25, Kartik Chandra SM '23, MIT professors Jonathan Ragan-Kelley and Joshua Tenenbaum, and MIT Assistant Professor Vincent Sitzmann.
Meschers could assist geometry researchers with calculating the distance between two points on a curved impossible surface ("geodesics") and simulating how heat dissipates over it ("heat diffusion"). It could also aid computer graphics artists in creating optical illusions out of practical objects.
M.C. Escher's artwork features depth-defying optical illusions with "impossible objects" that break the laws of physics. The perception of Escher's illustrations depends on the viewer's perspective, such as a person seemingly walking upstairs may be heading down the steps if the head is tilted sideways. Meschers can help computer graphics artists create similar illusions more easily and precisely.
In essence, Meschers offers a significant leap forward in the field of computer graphics, providing a powerful tool for both researchers and artists to explore, manipulate, and understand impossible objects in a more accessible and visually appealing manner.
