Where Screens Are Headed

We begin with an early work, the formative experience that led us to explore new possibilities in media.

Here, a sheet of paper punched with holes at regular intervals is laid over footage of stripes scrolling at a constant speed, producing a rotational motion like a ventilation fan or a propeller. The work began with a question: must an image, must a screen, be rectangular? By changing the very shape of the iPad’s screen, we set out to create a visual effect we had not seen before. 
A moving image can be divided in two: the content it carries, and the medium that shows it. Creators tend to fix their attention on content. Through this work, we began to sense that approaching a piece from the structure of the medium might lead to expression beyond what we could anticipate.

The iPad study addressed the display itself; gradually our attention turned toward projection. Wondering whether changing the form of a screen might yield something new, we explored with our hands, using materials close by. Two examples stand out.

When we project footage of points swarming within a fixed area onto a loosely crumpled tangle of wire, light behaves like scattering sparks. The screen turns two-dimensional movement into three dimensions, while its sparse, fibrous surface catches the light only in fragments. From this, flickering emerges.

In this piece, a point cloud moving by a fixed rule is projected onto an inclined screen that reinforces that behavior. The points appear to gather in response to gravity and stream down the slope. Motion that was confined to two dimensions takes on the behavior of an object occupying space.
We have accumulated many other elemental experiments alongside these two; a video showing some of them appears below. Each emerged from a simple multiplication of image and material.

In ordinary projection, the source data inside the projector passes through a lens and appears as an accurate image. The screen is asked to reproduce that light without loss. This research instead transforms the light itself at the screen to produce new visual effects, building a structure in which the data projected and the display perceived differ from one another.

What matters here is how the light is transformed on the screen. Projection onto three-dimensional objects may bring to mind projection mapping on architecture, an approach that precisely controls image data to fit the contours of a building. Our focus is not correcting light to match the screen’s shape. We project a simple image as it is, and leave the way light distorts and reflects to the physical form and material properties of the screen. Rather than covering an object with imagery, we present the phenomenon born of the interaction between light and screen as a new visual experience.

Here we introduce an exhibition where this research took form as work.
For The Texture of Light and Screen, we selected four studies from our work so far — the ones where we felt the screen and the image amplified one another most — and brought them together as a single installation.
Each projects a simple animation of geometric forms onto a screen with physical features such as ridges and hollows. We designed them so the screen not only deforms the image, but also reveals light’s inherent properties: reflection and transmission.

As projected light is deformed by the shape and material of the screen, accompanied by reflection and transmission, the image begins to take on the character of a substance present in space. The sensation shifts beyond watching a projected image toward feeling the light itself.

More recently, we have pushed our experiments further, enlarging the screen’s role to include interactions. 
The projected image is deliberately held still, and the screen’s own movement generates the change in light. We wondered whether a viewer, holding a screen small enough to carry, could create that movement themselves. 
In this work, the viewer holds a screen made of fine threads and moves it through a space where a still image of a point cloud is projected.

Light then appears to flow and flicker along the contours of the screen. That flickering, that drift of light along a line, arises because the viewer holds up the screen and moves it within a projection carrying no motion of its own. 
By inserting a screen into a space filled with still images composed of points, lines, and planes, and moving it by hand, the viewer carves light out of space in three dimensions and transforms it. Many other pairings of screen form and still image remain to be explored.

What matters in this work is not only that the light moves in curious ways, but that it responds to the viewer’s own hand, allowing them to anticipate what expression the next tilt will bring. Searching out light drifting in the space, seeking a movement that feels right, the viewer’s intent defines how the light behaves. The experience is neither watching an image nor watching light, but exploring light within a space. 
We have also found that viewers feel something akin to texture as they move. Depending on the screen’s form and the composition of the still image, the light follows the hand far more slowly, or far more quickly. In that offset between the hand’s motion and the light’s response, the viewer perceives texture. 
By making the screen something to hold, the experience departs from watching a projected image and becomes something new: moving the hand to explore light in space and predict its changes, and reading texture in the offset between hand and light.

What these experiments and works have shown us is that when the form of the medium changes, the experience itself changes. 
Digital information today is optimized for the flat screens of phones and computers. In making images, we usually think about content: what to show, what to convey. Yet even an animation as simple as a white circle moving at a constant speed can, through the form and material of the screen receiving it, take on a soft, pliant quality and a texture that seems within reach, something no flat display can offer. And when a screen can be held and moved, mere passive viewing gives way to a visual experience built on the interaction of light and screen, and on the offset between hand and light. The way we receive digital information shifts from simply seeing to something we can nearly touch.
This is not limited to projection. Within the two axes of content and medium, we can re-examine the format and environment of media, the premise underlying how information is presented, and treat it as an object of design. This perspective offers one clue for thinking about design and experience in the years ahead, and the range of expression and sensation we can create may widen far beyond what we expect.