Students create an interactive interface to reduce multigenerational family gaps

When Ethan Bresnick, a junior majoring in media arts and practice, was brainstorming ideas for a class project on human interaction via computer interface, he decided to pull inspiration from the close family relationships he had as a child. Using his experience as a frame of reference, he created a piece of technology that would enable old and young family members separated by geography and busy schedules to spend quality time with one another. 

“One of the greatest concerns that I have for the next generation is losing touch with their family history,” Bresnick said. “When I was a kid, I lived very close to my grandparents, and I was fortunate that they were able to share their imagination with me and able to play with me.” 

AR Playdates, an augmented reality playing platform, helps family members, such as grandparents and their grandchildren, to play together regardless of physical distance. 

In January, Bresnick began collaborating with JD LeRoy, a junior in the Iovine and Young Academy, for the final project in “Tangible and Spatial Computing,” a course that explores human interaction through virtual screen interfaces. John Carpenter, an adjunct associate professor in the School of Cinematic Arts, taught the class with the goal of teaching students how to use camera and sensor technology as a medium for their design work. 

“The goal of the program is to teach students to work with technology in interesting new ways,” Carpenter said. “The class typically involves looking at a number of different technologies and writing software and combining that with sensing technologies like cameras or 3D sensors.” 

Bresnick’s device, which resembles a coffee table, combines live video streaming, virtual graphics and depth sensors, creating a simulation between users that makes it seem as if they are interacting with each other in the same room, even if they are located miles apart.

The device captures users’ hands and toys on the panel surface that streams the image to another table in a separate location for both people to see what the other is doing. Following the course objective, Bresnick aimed to create a project that would center around utilizing camera sensor technology. 

“I was looking at how the data from the depth camera and the color camera on the play surface could be used as the user interface for creating these augmented reality effects in the background,” Bresnick said.

LeRoy said he was inspired by the vision and dedication Bresnick had for the project beyond seeing it as a normal class assignment. The product had the potential to change the toy industry by integrating more personal interactions with technology, he said. 

“There’s a lack of product on the market that enables playing creatively … especially in the telepresence industry,” LeRoy said.  “I think this kind of product … will make toy companies and other devices think more about what kinds of impact they can have and what kinds of interactions they can enable through the technology.” 

While Bresnick focused on software, LeRoy worked on hardware design given his experience in designing shoes for Adidas. 

“I was really interested in working with someone that has a really strong sense of natural forms … working with wood and materials that children are comfortable playing with so that this technology wouldn’t intrude or hinder the kinds of playing that children already do,” Bresnick said.

Currently, AR Playdates is not manufacturing the prototype for sale, but the team is recording the progress of the entire project to share with professionals in the toy industry since March. 

“With this project, specifically, [Bresnick] and I are reaching out to different manufacturers and different technology and toy companies to see if we [could] potentially work on some kind of partnership,” LeRoy said.

Bresnick credits USC for providing mentorship and advice throughout the process, specifically on designing and refining the software program 

“A lot of our faculty have been really involved in the development of technologies and augmented reality and virtual reality,” Bresnick said. “I was able to receive a lot of mentorship [on] what’s out there in telepresence products today and what considerations that have to be made in designing software for a telepresence augmented reality system.”

For Carpenter, the project is especially relevant to the current coronavirus crisis, with many states putting in place a stay-at-home order, since it will create an opportunity for families to build connections with one another when it would otherwise be impossible. 

“[They] took some of the ideas we talked about in class and figured out some interesting new way to apply it to an idea that’s really important for our community right now,” Carpenter said. “It’s especially relevant right now where we were all under social distancing, so finding plenty of interesting new ways to interact with other people and share experiences … is a great endeavour.”   

Optimistic in the progress of the product, LeRoy said he is confident in the potential of AR Playdates and its contribution to the toy and technology industry. 

“Everything is open right now and we’re not really certain, but we’re definitely open to continuing this project, or developing some kind of different mock up of it,” LeRoy said.