Students working with computer science faculty member Qing Zhang are bringing the community’s maritime heritage to life at the H. Lee White Marine Museum.

The museum, at the north end of West First Street surrounded on three sides by Lake Ontario and the Oswego River, is a longtime community treasure that preserves and interprets Oswego’s maritime history. Zhang and the students are using augmented reality (AR) and virtual reality (VR) to bring these stories to current and future generations, thanks to a grant from the Shineman Foundation.

“When I came to Oswego, I fell in love with the lake and fell in love with the lighthouse. And I learned the museum owns the lighthouse,” Zhang said. “When I learned more about them, I realized this could be a wonderful opportunity to work with them.”

Previously Damian Schofield, director of the college’s human-computer interaction master’s program, had worked with since-graduated HCI student Theo Johnson to implement a similar process using an app called Artvive at Fort Ontario on the city’s east side.

 The H. Lee White Marine Museum project built on lessons Schofield shared and on the ideas and skills of the participating students – HCI students Phoenix Boisnier and Ravi Teja (TJ) Manchi; Theresa Kamerman, a dual major in software engineering and mathematics; and computer science major John Owens.

“They reached out to us,” said H. Lee White Marine Museum Curator Michael Pittavino, who also is a 2011 bachelor’s in history and 2012 master’s in history graduate of SUNY Oswego. “This is so helpful to local organizations. We wouldn’t be able to provide this level of technology. You have so many small organizations with modest budgets.”

“The fascinating thing is seeing our students work with the local community on ways to help,” Zhang said. “It’s a win-win-win for students, faculty and community partners.”

Living history

A primary example greets visitors shortly after they enter the museum. When they reach a massive painting depicting a historical event, visitors can hold up their phones and, while using the Artvive app, receive audio and video that provides deeper information and context on the figures in the painting. Visitors can find interactive elements when they see a compass icon around the museum.

“I’ve been working with the design tool Artvive, which creates an AR overlay on items,” Boisnier said. He also voices copy written by Pittavino that provides historical context, which comes up when visitors use the app.

“I’ve been working on the website, implementing HTML, CSS and a little JavaScript,” Kamerman said. “It gives me a lot of freedom to explore the design process.”

“I work as a graphic designer and on user experience,” Manchi explained. “So I’m learning a lot about marketing and user experience. I’m learning about implementing what people want and understanding how the process works.”

“I take what TJ has designed onto the web,” Owens said. “In computer science, it’s important to learn quality control, and what it’s like to work on a small team.”

Others echoed the experience they are gaining on teamwork, on top of the technical skills. 

“We have a great team that’s willing to do what we need to get done, and it’s important to learn how to work on teams,” Boisnier said. “We’ve also been able to do some user-experience testing, which helps us readjust so we learn along the way.”

Kamerman added that they haven’t seen any hierarchy of graduate and undergraduate students, as they all work together seamlessly and with respect.

“I’m very happy for the experience our students are receiving, and very proud of the work they have done so far,” Zhang said.

Empowering experience

Zhang makes sure the project is empowering, letting the students come up with ideas and implementing solutions along the way.

“I get to work with very intelligent, responsible and disciplined students in a team-oriented environment,” Zhang said. “It’s a collaboration where they are full partners who come up with ideas. The students get to learn and grow and it gives them a sense of ownership in the project.”

The project also has involved teachers and students in Trinity Catholic School, who have helped show what is possible and interesting to a wider audience.

“I really appreciate the support the Shineman Foundation gives to faculty research,” Zhang added.

For the museum, it’s a project that highlights the past but also opens avenues for the future.

“Having a student-generated project is important because they are a future demographic for our museum,” Pittavino said. “We have to envision what audiences will look like in 10 to 15 years, so we need people who can help us figure out what this experience and what those audiences will look like. This contribution is incredibly impactful, and really elevates our offerings.”