(ambisonics are processed as binaural audio in the video – listen with headphones for the spatial effect)

My Heart is a flickering, swelling, phantasmagoric experience of memory. The full piece is for ten screens and eight speakers, composed for the RML CineChamber in San Francisco. Below is an alternate version for three screens.

Between January and May 2021, I took hundreds of photos of city blocks in New York and Los Angeles. The images were processed and turned into point clouds – three-dimensional digital models consisting of millions of points in space. Point clouds ignore most moving objects, so the scenes are largely devoid of people. This resonated with the emptiness of the city during a pandemic. They also generate artifacts derived from lighting glare or imprecise triangulation. Mistakenly captured streaks of blue sky, for example, have the same materiality as a brick wall, and a sign might only be legible from a specific angle, otherwise disintegrating into formless clouds of color. They generally have a focal point with high detail and deteriorate around the edges. All of this, in addition to the vibrant color and the ability to re-navigate and re-experience spaces I was once in, give these digital objects an intensely dreamlike quality.

I also captured point clouds of myself, family, and friends and then processed them through a GAN. Multiple angles of the same face layered and morphed into each other, distorted by the pointillistic and artifact-prone point clouds, such that one got the essence of the face without ever properly seeing it. Occasionally capturing minute detail, other times leaving out huge chunks, the visual representations of people and places look like memory.

The sound is mainly derived from a four-channel spatial recording of a walk by the ocean on a windy day, manipulated and composed in fifth-order ambisonics. Processed with granular synthesis, sonic spatial cues and temporal linearity are ripped into fragments and rearranged. The world glitches around the listener, who is suddenly in many places at once. I combined this idea with rising motifs, gestures reminiscent of smoke, dissipating as they rise, and voices making the opposite movement, densifying as they descend. Eventually, we return to the waves, the listener’s position static for the first time as the ocean ebbs around them.

Scattering of bits of spatial audio causes our perceptual systems to scramble to keep pace. Eventually we form a new conception of place from the ashes.