In A Crack in the Hourglass, a media artwork by Mexican-Canadian artist Rafael Lozano-Hemmer, people are invited to submit a photograph and story of a loved one lost during Covid. Over a period of about 40 minutes a modified robotic plotter draws the image of the loved one using hourglass sand. Once the drawing is complete and after a short pause, a mechanism gently tilts the platform and the sand falls away. Then the machine begins to draw another portrait. The process is broadcast live, showing the robotic arm drawing each portrait. The numbers of Covid deaths are no longer abstract; they are tangibly represented in the portraits of mothers, fathers, brothers, and sisters.
Our family had a unique window into the creative process behind A Crack in the Hourglass. Back in March 2020, when the pandemic abruptly shut everything down, our family became shut-ins and our home-work-school-hobby worlds collided.
My husband, Stephan Schulz, is head of R&D at Rafael Lozano-Hemmer’s studio, Antimodular Research. I am the executive producer of POP Montreal International Music Festival with a background in visual and media arts. In short, we are a family of creative innovators, and were well equipped to face the pandemic.
MATERIALS
- Hourglass sand
- Glass vessel
- AxiDraw SE/A3 plotter
- Teensy 3.2 microcontroller
- Anti-static polyurethane tubing, ID ¼”, OD 3/8″ McMaster-Carr 5790K24
- Custom 3D-printed nozzle
- Black cardboard, 32″×40″x1.25mm thick Peterboro Black Core mat board
- Linear actuator, 2″/50mm stroke, DC 12V AliExpress 1915825523
- Motor driver board, VNH5019 Pololu 1451
- USB cameras: Kurokesu C1 Micro M12 (1)
and C920 (1)
EMBROIDERY ENGINE
Our third-floor urban Montreal apartment became a laboratory, a studio, a space for dreaming, experimenting, and weathering the storm together. We found that coming up with DIY creative projects that responded to the global crisis gave us a sense of control over the situation and we wanted to contribute to efforts being made within our community. When there was a mask shortage, our daughters Thea and Odessa (11 and 10 at the time) began sewing masks for friends in the community and trying to come up with ways to personalize them (Figure A
).
At the same time Stephan was following an initiative led by Golan Levin from The Frank-Ratchye STUDIO for Creative Inquiry at Carnegie Mellon University (CMU), to create PEmbroider, an open source Processing library for computerized embroidery machines. There was a call for beta testers, and Stephan jumped at the occasion, convincing the rest of us that the best way to personalize our masks was to make custom-made embroidery designs.
We purchased a low-end Janome MC230E embroidery machine and began experimenting with it, tailoring the code, trying different approaches to make it work. The kids invited their friends to send us drawings, which were then run through the software, embroidered, sewn, and delivered to very happy customers. We also began embroidering portraits, downloading images of Ruth Bader Ginsburg (Figure 
B) and Aretha Franklin and running them through the program. The stark contrast within black and white images translated well to the contrasting colours of the thread and fabric.
THE PLOT THICKENS
As we continued to learn, play, and work together, we discovered the AxiDraw plotter. Developed by Lenore Edman and Windell Oskay, the AxiDraw was able to draw embroidery patterns using the PEmbroider library, making 2D drawings from images. When Stephan shared this finding with Rafael Lozano-Hemmer, it resonated immediately — a decade earlier, Rafael and his team had created an artwork called Seismoscopes, using vibration sensors and an automated X-Y plotter to illustrate portraits of philosophers.
Rafael had been thinking about how he, as an artist, could respond to the pandemic and contribute to the collective healing that was so desperately needed, especially at a time when coming together to mourn wasn’t possible. Rafael wanted to explore ways in which technology could be used to create a communal space for mourning, and memorialize the many people who lost their lives due to Covid-19.
The AxiDraw plotter’s open source command-line interface (CLI) tool and Python library made it possible to develop a platform for drawing images with sand. But there was a lot of research that took place in this working-from-home situation that eventually became A Crack in the Hourglass.
TRIALS AND TROUBLESHOOTING
The initial experimentation took place in a room adjacent to our kitchen, which meant that there were often odd mechanized sounds adding to the already cacophonous sounds of home; Thea’s piano practicing, Odessa’s class Zoom meetings, and my own work-related alert pings and bells. When a new sound came from the kitchen area, Thea, Odessa, and I would abandon our activities out of curiosity to come check it out. We enjoyed talking about the work as it evolved and provided feedback, which was generally well received!
Various methods, mechanisms, and materials were explored. One way involved covering a surface with sand which was then swept meticulously by a mechanized brush. At another phase, a small table vacuum was repurposed to remove small areas of sand, all combined with various techniques offered by the CMU Processing library. These materials and methods did not quite achieve the desired results. It was important that the processes reflect the care, love, and empathy that the individuals represented by the portraits warranted.
There were also physical and technological hurdles to overcome, including humidity; dry conditions caused static electricity to accumulate in the very fine sand, which clogged the tubing. Many plastic and glass vessels were tested until a method of depositing the free-flowing fine sand onto the surface was found, but they still needed to figure out how to do this in a fluid manner.
Eventually the AxiDraw in combination with Gregg Wygonik’s SquiggleDraw algorithm achieved finer control of the gradient values to display a greater grayscale range. This was made possible by controlling the amplitude of plotted sine waves based on the image brightness, which created thicker or thinner deposits of sand. The portraits were strikingly recognizable (Figure C
).
Other open source tools used in the project were openFrameworks, ofxArenaFueraDelReloj library, JavaScript, FFmpeg, and OBS.
DUST TO DUST
Rafael had the idea to use hourglass sand for its fine consistency and its ability to achieve greater definition. An hourglass uses a fixed quantity of sand to measure the ephemerality of time. Similarly, in this artwork the sand is recycled for each rendition. Rafael explains, “All of the portraits so far, hundreds of them, are made with the same small amount of sand. And for me, that was really important because it was a sense of universal solidarity around this, and a sense of connection.”
The first iteration was produced as an online exhibition for the Museo Universitario Arte Contemporáneo (MUAC) in Mexico City, launched in November 2020. The machine itself was installed at the Antimodular studio with two cameras broadcasting the action live. The apparatus behind the cameras (not visible to the public) was made with repurposed materials including a 5-gallon water jug, garden hose tubing, La-Z-Boy linear actuators, and wooden crates were used to create the tilting platform.
In the months that followed, the studio team created a custom-made version for gallery presentation and worked with a local glassblower who made the hourglass. This machine was shown at the Brooklyn Museum in 2021 and 2022, and the studio is in negotiations to show it in Montreal in the near future.
When the photographs started coming in from people around the world, the experience was intensely moving for us. As a queue of photographs waited to be rendered in sand, we waited for them too — a reminder of the importance of human connection, of leveraging strengths and capacity from multiple sources to find unity in the most challenging times.
• Submit a photo of a loved one, watch the drawings live, or view the portrait archive here.
JENNIFER DORNER is executive producer of POP Montreal International Music Festival and a board member of the Canada Council for the Arts. She enjoys working on creative projects with her two children that engage with the local community.
STEPHAN SCHULZ immigrated in 2007 from Berlin to Montreal, where he has since worked as the head of research and development at the media art studio of artist Rafael Lozano-Hemmer. His website.
ADVERTISEMENT
