By Sunny Um WIRED Korea

Park Yoon-jung, a businesswoman in Seoul, could not hear any sound at all. It was a congenital problem. But now she has no problem understanding other people’s speech, thanks to a cochlear implant (CI), a type of hearing aid transplanted above her ear.

Still, she has a problem with music. She cannot tell Do from Re. To Park, even a musical masterpiece is a noise.

In June, Park was on her way to find a solution when she met a composer and music player Lee Won-woo, who’s interested in the digital production of music.

Park was not the only person with a CI who had difficulty in listening to music. All the other people with CI had the same problem. Lee was surprised to find about that. “I could understand what the world of sound was like to CI users through the talks [with Park].”

Lee wanted to help them experience music. That is why he spent more than four months on designing an instrument for CI recipients. The instrument, when completed, looked like a piano. When told to test it, Park played a piece of experimental music on the instrument. She found it magical.

Lee says his project is an example of art technology, an experiment that adapts technology for use in art.

Art Technology

Back in the 1980s, art technology was a foreign concept. Not many Korean artists showed much interest in advances in technology. It was the same with engineers, who were not encouraged to share their achievements with artists.

Change came when artists and engineers joined hands and founded Art Center Nabi, a center showcasing techno-artistic experiments such as Trialogue (2001), P.Art.y (2007), and Come Join Us, Mr. Orwell (2009). Since then, the two groups have collaborated with each other more often.

Growth in art technology is accelerating in Korea, with the fourth industrial revolution being ushered in, says Lee Da-young, a researcher at Korea National University of Arts (KNUA). Many Korean universities opened up courses in art technology, including interactive design, multimedia video, and music technology.

“Almost every sector in Korea has grown so fast in the past, and the same goes for art technology. A growing number of artists turn to science and technology for help, with the two sectors being most influential in Korea,” she says.

Art Technology by Artists

The piano-like instrument for CI recipients is a different case, with its maker being a technician as well as an artist. He used Max, a computer program, to adjust the pitch levels of musical sounds on the instrument connected to a computer so that they could sound as part of a musical piece, not a noise, to CI recipients. Just like a piano, it makes a louder sound when the key is pressed harder.

The instrument was showcased at an exhibition in Seoul for a week in November. Visitors were allowed to try the instrument. But the pitch-adjusted sounds were not the musical sounds they were accustomed to. “They said [my instrument] sounded interesting,” says Lee. They probably heard the music of CI recipients for the first time.

Lee says his instrument is a good medium connecting CI recipients with those having no difficulty in hearing. “It made the interaction between the two more vibrant,” he says.

He believes art and technology pursue the same goal: a discovery of new possibilities in humans. “It seems like human abilities are less appreciated than they deserve. If more engineers and artists explore the human potential, art technology for the disabled will contribute to making our society richer.”

While Lee focuses on auditory technology, a group of artists use both visual and auditory techniques in their artwork. Park Ji-young, Park Su-young, and Park Hyun-jin work as a team using computer programming to create and mix the imagery and sounds. They call themselves SSIX.

For what looked like an avant-garde form of art, the team designed an engine that synthesizes images or animations into a randomized slide show. The imagery used in the show captured an urban scenery of buildings and streets. The slides were projected on the wall and floor for two choreographer-dancers dancing through the imagery.

There was no background music for the performance. But, there was a murmur in a woman’s voice, simulated digitally, saying “Mind the gap”, “Shocking sale,” or “Don’t miss this chance.” They were the sounds that commuters hear every day on their way to work or school. SSIX says this voice was also created and synthesized with computer software.

“We tried to show the downside of gentrification through our performance,” an SSIX member says. “Our team is concerned the next generation will live a money-centered urban life, forcing out [less privileged] people.”

Park Su-young from SSIX regards technology as one of the tools with which to express human emotions. “Technology alone wouldn’t enable us to understand the loneliness of humans 100 percent, but it can help [the artists] to address it and make it more relatable.”

Art Technology by Engineers

Another different case involves Lee Kyogu, a professor at the Graduate School of Convergence Science and Technology of Seoul National University (SNU), who says, “I have had a keen interest in music for long.”

He majored in electrical engineering as an undergraduate at SNU, studied music technology and electrical engineering as a graduate student in the United States before receiving a Ph.D. in computer-based music theory and acoustics from Standford University.

For the past two years, Lee and his team have worked on a system of algorithms, or a singing voice synthesis (SVS) system, which learns about the voice, style, and tone of vocalists with pre-recorded songs.

Once it’s trained enough, the SVS system creates vocal models of individual singers. The system then makes a computer-generated vocalist mimic the voice of the real singer when singing songs of any kind. As an example, Lee makes the model of Freddy Mercury from Queen sing “Gangnam Style” by PSY, or the model of Kim Kwang-seok sing “Bohemian Rhapsody” by Queen.

“Many deceased singers still have their fans,” he says. “Suppose they hear new songs in their voices, this should be rewarding.”

His system should be useful to amateur musici­­ans who cannot afford to spend much money in creating his work with the use of modern appliances such as synthesizers.

He says “A piece of music is made much easier than before. You no longer have to rent studios or buy expensive equipment [to compose or play music.] You can make professional-level music simply with a laptop at home.

“The virtual musical instruments [out in the market] are fairly well-built. If you can write a song, you simply can play with a virtual piano or drum set. But, though virtual vocalists are not [good enough yet.] I think our system meets such needs,” he reiterates.

As he admits, the system needs improvements, with English lyrics not sounding natural yet.

Lee believes technology will help the artists go beyond their boundaries of creation. “The creativity is what humans have. The technology can assist them, like amateur artists, in doing creative work. This will surely expand the world of art,” he says.