For the fourth consecutive summer, Institute for Culture and Society (ICS) research fellow Inés Olza has collaborated with the Google Summer of Code (GSoC), a global program that grants scholarships to young computer scientists from all over the world to collaborate with institutions, research groups and companies dedicated to developing code for free software tools.

Olza contributes as a mentor for the international consortium Red Hen Lab for the Study of Multimodal Communication, which brings together experts from more than 20 universities in countries such as the United States, Spain, Germany, Brazil and Norway.

This year, she gave advice on available text banks and linguistic tools in Spanish for applications related to automatic voice detection. This takes shape, for example, in the synchronization of subtitles with what is happening on screen in movies and series.

The researcher explains that forced alignment is key to finding audiovisual material in multimodal data bases that contain image, text, and sound. Olza relates that many of the programs that do this task, such as Gentle, are well developed in English, but not so much in Spanish.

From her experience as a GSoC mentor, she highlighted that, "it is a great opportunity to work remotely with people from other countries, learn about other disciplines, collaborate with them from the point of view of linguistics and make contributions from one’s specialty beyond one’s comfort zone."

She also stresses that Google is an example of "how the most technical knowledge can be put in the service of citizens" because the code developed at this virtual summer campis open and available to any user.

12 Red Hen Lab projects funded at GSoC

GSoC funded 12 Red Hen Lab projects in 2018. All of them are related to the development of automatic text processing tools (natural language processing), sound and image that can be incorporated into its NewsScape International Library of Television News. It is a gigantic corpus of spoken language that allows for the complete study of multimodal aspects (gesture, prosody, images and sounds that accompany speech, television productioneffects, etc.). This unprecedented tool could revolutionize the study of discourse and information coverage.

Among other topics, the projects have addressed automatic voice recognition in different languages, including recently Arabic, Chinese and Russian, the detection of emotions and the segmentation of interactions (turns, genres, conversational sequences, etc.).

In 2015, the GSoC-Red Hen scholarships focused on audio analysis, while in 2016 they focused on machine learning within the field of computational vision. In 2017, their goal was to create a multimodal processing system to extract information about human communicative behavior from text, audio and video.