Universal Advertising - When Creativity Meets Accessibility... in 30 Seconds!

María J. Valero Gisbert, University of Parma, Italy

 

María J. Valero Gisbert

María J. VALERO GISBERT, PhD in Linguistics. Associate Professor at Parma University (Italy). Department of Discipline Umanistiche, Sociali e delle Imprese Culturali (DUSIC). She teaches Audio Description in the final year of the BA. She coordinates the online MA in Audiovisual Translation: http://master.metav.unipr.it. Her research interests are Lexicografy, Hispanic Phraseology, Audiovisual Translation and Audio Description.

Some publications:

Valero-Gisbert, María J. (2024). La traducción de la imagen en la audiodescripción cuando vehicula un contenido cultural, in HIKMA. - ISSN 2445-4559. - pp. 1-22     

Valero-Gisbert, M.J. (2022), “Audio description for the screen”, in E. Perego, C Taylor, The Routledge Handbook of Audio Description, Routledge, NYork, pp. 183-199.

Valero-Gisbert, M.J. (2021). La Audiodescripción: de la imagen a la palabra. Traducción intersemiótica de un texto multimodal. Clueb, Bologna. ISBN è 978-88-491-5711-6.

 

Universal Advertising - When Creativity Meets Accessibility... in 30 Seconds!

Carlo Eugeni, University of Leeds, UK; María J. Valero Gisbert , University of Parma, Italy

Advertising emerges as a multifaceted domain encompassing linguistic, semiotic, and socio-cultural dimensions, employing strategic language, visuals, and symbols to persuade, inform, and resonate with target audiences. Mainly scrutinised for its persuasive techniques, discourse structures, and cultural implications, advertising has been analysed extensively by many scholars (Barthes 1963,1966, Hall 1984, Kilbourne 1990, Chandler 2022, Jhally 2003, Berger et al. 2020, to name just a few), and under many perspectives (semiotics, media, culture, gender, politics, economy among others).

Translation Studies scholars have paid special attention to advertising starting from the 1990s, with authors from various backgrounds exploring such many aspects (De Mooij, 2004, 2021; Guidère, 2000; Ho, 2004; Torresi, 2010; Valdés, 2011). As proposed by Torresi (2022), translation studies on advertisements can be subdivided into four main phases: studies on verbal elements; studies on verbal and non-verbal elements and multimodality; studies on the market context; and critical discourse analysis encompassing genre, cultural, and post-colonial studies.

In the field of Audiovisual Translation, advertising has also been investigated due to its unique challenges and considerations in translation and adaptation for different cultures and audiences (Díaz-Cintas 2007, Chiaro 2009, Gambier 2019, Baños Piñero 2019, Palencia-Lefler, 2023), while in Media Accessibility, the focus often extends beyond traditional advertising content, to consider how accessibility features can be implemented to make audiovisual products inclusive and accessible to diverse audiences (Matamala 2006,  Remael 2010, Orero 2012).

However, while the complexities of making advertising content accessible and inclusive for diverse audiences has been explored, the idea of a universal design for advertisements has not. It is in this context that this research addresses the specific issue of universal advertising. In the framework of universal design (Hermosa-Ramírez 2023), and based on the Accessible Filmmaking Guide (Romero-Fresco et al., 2019), this speech will report on the work carried out in Italy by a group of scholars, professionals, and experts for the Italian standardisation organisation UNI. The work lasted 2 years, and was aimed at developing a standard, quality criteria, and teaching materials for born-accessible advertisements.

After briefly touching upon Universal Design and Accessible Filmmaking (Romero Fresco 2019), this speech will be divided in two parts. In the first part, the focus will be on the challenges of integrating norms, best practices, and conventions – used in subtitling for the deaf and the hard-of-hearing, audiodescription for the blind, sign language interpreting, and dubbing in easy-to-read language – to a specific text-type: the 30-second audiovisual advertisement. The focus will be on the difficult relationship between usability and accuracy on the one side, and creativity and accessibility on the other when making advertisements accessible.

In the second part of the speech, specific guidelines for producing born-accessible advertisements will be illustrated and discussed. These are the result of a survey that has been recently carried out by the University of Leeds on the challenges that media accessibility experts encounter when having to make an audiovisual product accessible to specific audiences. In particular, a short questionnaire was submitted to over 300 professionals in the fields of subtitling for the deaf, audiodescription for the blind, sign language interpreting, and dubbing in easy-to-read language around the world.

The survey covered technical (shots, shooting, characters per line/per second, minimum gap, duration, synchronisation, etc.), semiotic (camera movements, images, actions, sound effects, background music, dialogues, on-screen text, etc.), linguistic (speech rate, product-specific terminology, accents, grammar, syntax, register, humour, orality, etc.), procedural (client’s brief, too many many/not enough guidelines, deadlines, multiple colleagues working on the same project file, pay per minute, glossary, machine translation, automatic synchronisation, template files, etc.), and other aspects.

The aim of the survey was to make a step forward in the study of accessible filmmaking, by concretely helping filmmakers understand which aspects to consider when designing their audiovisual product. Because accessible filmmaking tries to make sure that service providers do not have to decide between accessibility and accuracy when making a product accessible, the answers to this survey will help trainers and filmmakers better understand the accessible filmmaker’s training and profession.

This speech is especially relevant for both experts and beginners. Beginners will learn about accessible film-making and universal design applied to advertisements; experts like media accessibility professionals (subtitlers, audiodescribers, sign language interpreters, easy language translators, etc.), researchers, providers, creatives (designers, copywriters, directors, etc.), producers, and distributors of audiovisual content in general, and advertisements in particular, will learn about the details of the recently published standard. Its potential impact on the society as a whole will be discussed during the Q&A.

You can find the link for the presentation slides below:

https://languages-media.com/public/2024/slides/Carlo_Eugeni&Maria_Valero_Gisbert.pdf

References

Baños Piñero, Rocío (2019) Translating reality TV into Spanish: when fast-food TV challenges AVT conventions. Perspectives: Studies in Translation Theory and Practice, 27 (2), pp. 265-282.

Barthes, Roland (1963) La corte postale. Cahiers de la Publicité 7.

Barthes, Roland (1966) Rhétorique et Publicité. Communications 4/5.

Berger, Arthur Asa; Dawn Lerman; David Mechlin; Robert J. Morais; Edward Timke (2020) Author Meets Critics: Brands and Cultural Analysis, Advertising & Society Quarterly; volume 21, issue 2.

Chandler, Daniel (2022) Semiotics: The Basics, London and New York: Routledge.

Chiaro, Delia Carmela (2009) Italians doing it on the web: Translating and selling Italianness. VIAL n_6: 33-49.

De Mooij, Marieke (2004) “Translating Advertising. Painting the Tip of an Iceberg”. In The Translator 10/2, 179-198.

De Mooij, Marieke (2021) Global Marketing and Advertising: Understanding Cultural Paradoxes. Sixth Edition. London: Sage.

Díaz-Cintas, Jorge (2007) Por una preparación de calidad en accesibilidad audiovisual. TRANS: Revista de Traductología, 11, 45-59.

Gambier, Yves (2019) Traductologie et médias: les défis de l’interdisciplinarité, Syn-Thèses; n. 9-10, 11-24.

Guidère, Mathieu (2000) Publicité et traduction. Paris: L’Harmattan.

Hall, Jane (1984) Advertising Research in Canada — Where We are Today, Media Information Australia; volume 31, issue 1, 77-80.

Hermosa-Ramírez, Irene (2023) Universal and ecological design in media accessibility: finding common ground. Universal Access in the Information Society, vol. 23(2), 561 – 575.

Ho, George. (2004) “Translating Advertisements across Heterogeneous Cultures”. In The Translator 10/2, 221-244.

Jhally, Sut (2003) “Image-Based Culture: Advertising and Popular Culture.” The Gender, Race, Class and Media Reader. Ed. Gail Dines and Jean M. Humez. 2nd Ed. Thousand Oaks: Sage Publishing, 249 – 257.

Kilbourne, William E. (1990) Female Stereotyping in Advertising: An Experiment on Male-Female Perceptions of Leadership, Journalism Quarterly; volume 67, issue 1, page 25-31.

Matamala, Anna (2006) "La accesibilidad en los medios: aspectos lingüísticos y retos de formación". In R. Amat, & Á. Pérez-Ugena (eds.), Sociedad, integración y televisión en España Madrid: Laberinto, 293-306.

Orero, Pilar (2012) Audio Description Behaviour: Universals, Regularities and Guidelines. International Journal of Humanities and Social Science, 17(2), 195-202.

Palencia-Lefler, Manuel (2023) Inclusive Advertising through the Soundtrack: Insights from People with Visual Impairments. Communication & Society, 36(2), 67-81.  

Remael, Aline (2010) “Audiovisual Translation”. In Yves Gambier and Luc van Doorslaer (eds.), Handbook of Translation Studies, Amsterdam, John Benjamins, 2-17.

Romero-Fresco, Pablo (2019) Accessible Filmmaking - Integrating Translation and Accessibility Into the Filmmaking Process, London and New York: Routledge.

Romero Fresco, Pablo; Fryer, Louise; Branson, Josh; Jo-Jo Ellison (2019) Accessible Filmmaking Guide, https://accessiblefilmmaking.wordpress.com/

Torresi, Ira (2010) Translating Promotional and Advertising texts. London: Routledge.

Torresi, Ira (2022) Pubblicità. In ENTI (Encyclopedia of Translation & Interpreting). AIETI.

Valdés, Cristina (2011) “Advertising Translation”. In Gambier, Yves & Luc van Doorslaer (eds.) Handbook of Translation Studies, Volume 2. Amsterdam: John Benjamins, 1-5.

 

Carlo Eugeni

Carlo Eugeni is associate professor of audiovisual translation at the University of Leeds (UK), where he teaches Media Accessibility, Subtitling, and Interpreting, and leads the MA in Audiovisual Translation and Localisation.

Carlo graduated in Conference Interpreting and Translation at the University of Bologna in 2003, and in 2009, he received his PhD in English for Special Purposes with a thesis on live subtitling through respeaking.

He is the founder of the biennial international symposium on live subtitling and accessibility, and the chairman of the Intersteno scientific committee.

In 2016, he co-founded the international journal on linguistic and cultural mediation CoMe, and the journal of professional reporting and transcription Tiro in 2020.

In 2018, Carlo received an honorary award by the Italian association of cochlear implant users APIC for his commitment in promoting the empowerment of deaf people.

In 2023, he joined the Easy Language International Network ELIN, the Globalization and Localization Association GALA, and the UN International Telecommunications Unit experts group on Sustainability, Accessibility and Inclusion in the Metaverse.

Carlo is a professional intralingual and interlingual respeaker, subtitler, and interpreter, has been publishing extensively on respeaking, subtitling, and easy language; and is currently involved as a researcher in the Horizon-UKRI funded project iDEM on Easy-to-Read language.

Song Translation in Audiovisual Content: An Impossible Puzzle or an Unexplored Specialization?

Magdalena Chiaravalli, Audiovisual Translator, UK

Song translation was long considered an underdeveloped research topic within the scope of Translation Studies, until an upsurge in interest occurred during the first decades of the 21^st century, with the publication of several works focusing on the topic. However, the scope is rather limited and most of the literature focuses on opera translation or general song translation, while exclusive literature on song subtitling and dubbing is meagre (Díaz Cintas & Remael, 2020; Torre Fica & Saunders, 2022). Nonetheless, it is not only the lack of literature on the specific topic of song translation for audiovisual content that is of note. There is also a lack of practical models, guidelines or theory application for translators actually working in the industry and dealing with this very broad and difficult topic. Thus, a substantial number of questions arise: how should songs be translated when appearing in audiovisual products? What should a translator focus on? How can decisions be made and who should make them? Which elements of the source text (ST) can influence those decisions? How important is it to have technical musical knowledge to tackle this task? While the translation process is unique to each translator, surely some possible answers to these questions can be found. But, most importantly, some sort of road map, adaptable to match specific situations, could be put in place to assist those working in the industry and lacking the formal specialization that does not seem to exist, at least to this date, except for very few courses which focus on one AVT area (usually dubbing).

I will present an innovative theoretical approach to song translation when dealing with audiovisual content: The Song Translation Triangle for Audiovisual Texts. Combining and updating Peter Low's (2017) hexathlon principle with the technical constraints of AVT, outer context and song genre conventions, this triangle aims at providing a sort of guideline for translators when faced with the task of translating songs for subtitles and dubbing. All parts of the triangle will be explained and how/when one might take precedence over the other or can/should be sacrificed and why.

From this contribution, participants will learn a bit about what song translation entails, in terms of the musical knowledge needed to carry out the task. They will also be made aware of the six elements that need to be taken into account and how to combine them with AVT technical constraints. Ideally, they will be left with an initial idea as to what this highly technical specialization entails in terms of work and time, and what to expect if faced with a project of these characteristics, as translators, editors or managers. The aim is for the participants to at least become aware of this specialization, instead of going into song translation as if it were a general translation with meaning only in mind. If we go to great lengths as to the proper translation of technical or legal texts or scenes with technical or legal content being discussed, there is no reason for us not to do the same with music content. Participants will benefit from an introduction to all musical and poetic aspects to be taken into consideration. They will be left with the understanding that they can use different elements and combine them in different ways depending on what is most important based on the project.

You can find the link for the presentation slides below:

https://languages-media.com/public/2024/slides/Magdalena_Chiaravalli.pdf

References

Díaz Cintas, Jorge, & Remael, Aline (2020). Subtitling: Concepts and Practices. Taylor and Francis. https://doi.org/10.4324/9781315674278/SUBTITLING-JORGE-D

Low, Peter (2017). Translating song: lyrics and texts [Book]. Routledge. https://doi.org/10.4324/9781315630281

Torre Fica, Iñaki & Saunders, Nicholas (2022). Manual de traducción y adaptación de canciones para doblaje y teatro musical. Universo de letras.

 

Magdalena Chiaravalli

Magdalena Chiaravalli obtained her English<>Spanish Translation 5-year undergraduate degree at the Universidad Nacional de La Plata and a MSc Translation and Technology (Audiovisual) with honours issued by University College London (UCL). She has been a freelance audiovisual translator for over 6 years and has been part of roughly 300 projects of interlingual subtiling, SDH, dubbing and audio description, with credits in several streaming services, including Netflix, Disney+, Max (LATAM) and Prime Video. Since 2024, she is an assistant teacher in the module Introduction to accessibility, which is part of the Maestría en Nuevas Tecnologías de la Traducción, Localización y Traducción Audiovisual [Master's Degree in Audiovisual Translation, Localization and Translation New Technologies] at the University of Buenos Aires. She is also a member of the AVTPro Certification team. Her research areas include literary and audiovisual translation, with focus on songs translation and music.

Resonance Beyond Sound: Exploring Emotions in Music for the Deaf and Hard-of-Hearing with Closed Captions

María Isabel Rivas Ginel, Joss Moorkens & Irene Siragusa, Dublin City University, Ireland; Judyta Mężyk, University of Silesia, Poland

Over the past few decades, music-based short films have emerged as a growing niche, yet they are largely inaccessible to Deaf and Hard of Hearing (DHH) audiences due to insufficient closed captioning (CC) for musical elements. Despite music’s key role in storytelling, subtitling practices primarily emphasise dialogue and sound effects, often reducing music to generic tags like “[upbeat music]” or “[suspenseful music]”, which fail to capture the emotional complexity that music conveys (Aleksandrowicz, 2020). Although previous studies have shown that even brief musical phrases can evoke strong emotions without the need for verbal explanation (Revuelta et al., 2020), subtitling practices have historically struggled to address the complexity of music as a storytelling element fully. In a study across 15 international guidelines, Neves (2005) found that most relied on generic tags to describe musical elements and later further demonstrated how insufficient music captions could reduce audience engagement, stressing the importance of effectively translating the emotional nuances of music to enrich the viewing experience (Neves, 2010). Nowadays, guidelines from platforms like Netflix^[1] advocate for the inclusion of relevant song lyrics, titles, and authors in captions, marking a shift towards more comprehensive musical representation.

Description of the project and data set

This communication presents the preliminary results of an international project exploring new methods to describe music and the emotions it evokes in CC for DHH audiences in music-only short films. It aims to conceive more inclusive and nuanced subtitling practices that offer broader cultural access to DHH audiences, making music in films more accessible and emotionally resonant. Initiated in late 2023, this collaborative research involves national and international students in Poland, Ireland, Switzerland, and France. To ensure consistency, lecturers from said universities were provided with a set lesson plan that included: (1) an introduction to the importance of music in films and warm-up exercises; (2) an introduction to Netflix CC guidelines focusing on practices regarding music and songs; (3) examples of alternative practices such as those employed by Remote Access (Hamraie and Williams, 2023), Alt-Text as Poetry,^[2] or the artist Christine Sun Kim in her video Closer Captions.^[3]

Then, the students were given a shortened version of The Lady and the Reaper, a 2009 animated short film that humorously portrays a battle between a Reaper and a doctor over an elderly woman, who is eager to reunite with her late husband in the afterlife. Finally, they were asked to describe the music in their native language, paying particular attention to the emotions elicited by different musical cues and following Netflix guidelines: a maximum of 42 characters per line, two lines per caption and, depending on the language, a reading speed of 17 or 20 characters per second. Due to variations in each institution’s academic calendar and the voluntary nature of the task, each lecturer scheduled the workshop and submission deadline at a different time to better fit their course content and maximise participation. Given the disparity in the number of documents received (Table 1), we will present the results of the preliminary study of 5 files in Italian and French and expand our database in the current academic year.

Table 1: Number of files per language.

Methods for Data Analysis and Results

The data analysis employed coding techniques commonly used in qualitative research (Matthews & Ross, 2010) consisting of assigning "tags" to data segments to identify patterns in how musical elements were described by the participants. This iterative coding process, a key aspect of Constructivist Grounded Theory (CGT), allows the researcher to engage in comparative analysis, create analytical memos, and eventually formulate theory (Charmaz, 2014). Coding progresses through various stages—initial, focused, and axial—and reflects the different phases of the analysis. The initial tags remain closely aligned with the data (Table 2) whereas, during focused coding, the most significant and recurrent codes are selected concentrating on key musical parameters (Desblache, 2018; Farrant, 2024); frequently occurring elements like harmony, song structure, genre, and lyrics; and “Units of Creative Potential (UCP)” (Guerberof Arenas and Toral, 2022). The final coding phase connected these categories to their subcategories, grouping the data into meaningful clusters: Objective/active listening, Subjective/passive listening, and UCPs. After the synchronised visualisation of the data, we created the subcategory “visual anchor” following Roland Barthes’ theory of anchorage (1964) to highlight how subtitles connect music descriptions to on-screen action.

Table 2: Evolution of the tags

The study identified notable differences in how Italian and French participants approached subtitling music, highlighting distinct analytical focuses and creative practices. Among the common traits, we observed that both groups avoided directly translating lyrics and prioritised instead the overall emotional and auditory impact of the music. One key distinction between the two groups lies in their analytical approaches to musical elements. French subtitles emphasised "Timbre," focusing on the distinct sound qualities of instruments and their textural differences. In contrast, Italian subtitles featured a higher frequency of references to "Dynamics," indicating a greater focus on the variations in loudness and softness within the music. This suggests that while French subtitlers may prioritise the subtle sound characteristics of music, Italian subtitlers are more attuned to its intensity and emotional fluctuations. The Italian subtitles also placed a consistent emphasis on "Emotional Tone," further reinforcing the idea that Italian students were more focused on conveying the emotional impact of the music compared to their French counterparts.

In terms of units of creative potential, both groups employed evocative imagery, personification, and metaphors to enrich the viewer’s experience, though with different frequencies. Italian subtitles demonstrated a marked preference for "Evocative Imagery" (63 instances compared to 23 in the French dataset) and "Personification" (60 instances versus just 3 in French). These patterns suggest that Italian subtitlers often used more descriptive and imaginative language, attributing human-like qualities to music, to create a more immersive and emotionally resonant experience. French subtitles, by contrast, employed "Metaphors" more frequently (13 instances versus 3 in Italian), reflecting a preference for conveying meaning through symbolic comparisons. This suggests a divergence in how each group communicates the essence of music: more direct sensory and emotional imagery versus the reliance on figurative language to create deeper, symbolic interpretations.

The number of subtitles also varied between the two groups. Italian subtitles showed greater variability in number, with participants contributing both the highest and lowest amounts, while the French subtitles fell within a more consistent mid-range. This variability likely stems from the different instructions given to each group. Italian participants were instructed to focus exclusively on musical descriptions, which may have resulted in a higher number of music-focused subtitles. Meanwhile, French participants were allowed to include descriptions of sound effects, which, though removed from the final analysis, may have contributed to fewer music-specific subtitles overall.

In conclusion, while the findings from this study provide valuable insights into subtitling practices for the DHH community, further research with a larger dataset is necessary to solidify these observations and explore additional factors such as genre, audience, and translation norms. The efforts to standardise the data, including the removal of sound descriptions, have ensured a clear focus on music portrayal. Despite the study's limitations, it highlights the importance of developing subtitling strategies that combine technical precision with creative expression, offering a promising foundation for future research to enhance the accessibility and emotional richness of films for DHH audiences. Moreover, this preliminary study sets a solid baseline for consistently processing the data we will gather in future research, ensuring a more robust analysis moving forward.

^[1] https://partnerhelp.netflixstudios.com/hc/en-us/sections/22463232153235-Timed-Text-Style-Guides

^[2] https://alt-text-as-poetry.net/

^[3] https://www.youtube.com/watch?v=tfe479qL8hg

You can find the link for the presentation slides below:

https://languages-media.com/public/2024/slides/Maria_Isabel_Rivas_Ginel.pdf

References

Aleksandrowicz, P. (2020) ‘Can Subtitles for the Deaf and Hard-of-Hearing Convey the Emotions of Film Music? A Reception Study’, Perspectives, 28:1, pp.58-72.

Barthes, R. (1964) ‘Rhétorique de l'Image’, Communications, 4(1), pp.40-51.

Charmaz, K. (2014) Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. 2nd ed. London, New Delhi, Singapore, Thousand Oaks, Washington DC: Sage.

Desblache, L. (2018) ‘Translation of Music’ in C., Sin-Wai (ed.) An Encyclopedia of Practical Translation and Interpreting. Hong Kong: Chinese University Press, pp.297-324.

Farrant, M. (2024) ‘Glossary of Musical Terms’, Hello Music Theory. Available at: https://hellomusictheory.com/resources/glossary/ (Accessed: 6 August 2024).

Guerberof Arenas, A.G. and Toral, A. (2022) ‘Creativity in Translation: Machine Translation as a Constraint for Literary Texts’, arXiv preprint arXiv:2204.05655.

Hamraie, A., Williams, M. (2023) "Remote Access: A Crip Nightlife Party," Lateral 12.1. https://doi.org/10.25158/L12.1.7

Matthews, B. and Ross, L. (2010) Research Methods: A Practical Guide for the Social Sciences. Harlow: Pearson Longman.

Neves, J. (2005) Audiovisual Translation: Subtitling for the Deaf and Hard-of-Hearing. PhD thesis. School of Arts, Roehampton University, University of Surrey.

Neves, J. (2010) ‘Music to my Eyes…Conveying Music in Subtitling for the Deaf and the Hard of Hearing’, in Ł. Bogucki and K. Kredens (eds.) Perspectives in Audiovisual Translation. Łódź Studies in Language. Frankfurt am Main: Peter Lang, pp. 123-145.

Revuelta, P., et al. (2020) ‘Limitations of Standard Accessible Captioning of Sounds and Music for Deaf and Hard of Hearing People: An EEG Study’, Frontiers in Integrative Neuroscience, 14, p.1-9.

 

María Isabel Rivas Ginel

María Isabel Rivas Ginel is a postdoctoral researcher at the School of Applied Language and Intercultural Studies and the ADAPT Centre in Dublin City University (DCU). Her current position involves working on the topics of translation technology, large language models and generative artificial intelligence (genAI), translators’ attitudes towards new technologies, and accessibility. She holds a PhD in language science delivered by the Université de Bourgogne France-Comté (Dijon, France) under the title “The ergonomics of computer-assisted translation tools for video game localisation” and her fields of interest are audiovisual and multimedia translation, translation technology, gender studies, accessibility, and inclusivity.

Transcoding Music and Sound Into Subtitling for Deaf and Hard of Hearing Viewers: The Case of "Dunkirk" (2017)

Renata Dalianoudi, Ionian University, Greece

 

Renata Dalianoudi

Renata Dalianoudi is Associate Professor in the Dpt of Audiovisual Arts, Ionian University. She holds a Bachelor and a PhD from the Dpt of Music Studies, Athens University, with a scholarship by the State Scholarship Foundation and the Legacy “Papadakis”, Athens University. She attended Ethnomusicology and Folklore Studies in the University of Wien, and Theatre Studies in Athens University. She also studied piano, organ, accordion, guitar and ballet. Among her works are: 4 monographs, 70 articles and papers, 4 lemmas in international dictionaries, 52 documentaries, 210 tv & radio broadcasts on music in theatre and cinema. She has obtained 31 scholarships by: a) S.S.F., b)Athens University, c)Athens Conservatory, d)National Conservatory.

Foreign languages: English, French, German, Italian, Spanish.

Transcoding Music and Sound Into Subtitling for Deaf and Hard of Hearing Viewers: The Case of "Dunkirk" (2017)

Vilelmini Sosoni, Aikaterini Boutouridou & Renata Dalianoudi, Ionian University, Greece

Sound in cinema constitutes a means of non-linguistic narration, which serves the aesthetic needs of a film and complements the visual and linguistic narration (Dalianoudi 2021: 283-309). The representations that result from the interconnection of image and sound create fixed and predetermined musical and cultural associations which contribute to a film’s overall meaning. In other words, cinema provides an intertextual context in which visual, verbal and auditory narratives interact and are transformed into cultural and emotional associations. As for music especially, its primary function is to provide emotional support to the story (Kivy, 1997: 322), while the whole interplay between speech, image and sound/music becomes artistically expressive (Monaco, 1981). Yet, not all spectators have access to the auditory narrative and to the interplay between the visual, the verbal and the auditory narratives given that increasingly many people around the world –466 million people according to 2022 WHO data– have hearing problems or are d/Deaf. People who are deaf or hard of hearing identify themselves with the mainstream hearing society, while Deaf people are members of the linguistic and cultural minority of the Deaf community (Zárate, 2021: 21). However, medical and sociological reports (Gouge 1990; Darrow 1993) prove that the world of the d/Deaf is all but silent. In fact, music is for them far more important than it is given credit for (Neves, 2010: 123). The experience d/Deaf people have when “feeling” music is similar to the experience other people have when hearing music, since both musical vibrations (perceived by the d/Deaf), and sound are processed in the same part of the brain (Shibata, 2001). Furthermore, music “plays an important role in landmarking significant experiences and spaces in people’s lives”, since songs and melodies “have a shared life which goes beyond their compositional existence” (Neves, 2010: 124). It becomes clear that it is crucial for all those working in the sphere of accessibility services for d/Deaf people –and subtitling for the d/Deaf and the hard of hearing (SDH) in particular– to pay special attention to the conveyance of music. Yet, conveying/transferring non-verbal auditory elements, like sound effects and paralanguage (i.e., intonation, accent, and information about who is speaking, e.g. an identified character, or whether the voice is male or female) and above all music pose many challenges to subtitlers. As Neves (2010: 125) points out, in order to proceed with the linguistic transfer of acoustic messages, translators have to be “sensitive to the interplay between images, speech, sound and music to be able to decode their inherent messages and to find adequate and expressive solutions to convey such sensations verbally” and ultimately create subtitles that “serve the purpose of the acoustic component of the audiovisual text in all its effects”, yet in a way that is meaningful to people who might have never perceived sound before. Naturally, this balance is hard to achieve. Translating contextually occurring sound and music into written language will demand transcoding expertise that will pull the translator between the intended meaning of the acoustic messages, their function in the text and the effect any rendering may produce on the deaf viewer, while bearing in mind the spatiotemporal constraints of subtitling and the need to also convey the spoken messages. As it is expected, guidelines and codes of good practice around the world contain recommendations on how to convey sound effects and music in the form of subtitles. Yet, actual practice shows that even when the topic is explicitly addressed on paper, the actual presentation of information on screen is less systematic, while there are very few reception studies that shed light into the way d/Deaf viewers perceive the transcoding of music and sound into subtitles and into the way the transcoding affects their immersion or enjoyment (Revuelta et al., 2020). The present study seeks to address this shortcoming by adopting a novel approach to SDH whereby two subtitlers, a music expert and a deaf viewer work together to create the set of subtitles including the transcoding of music and sound and by then conducting a reception study to establish how different viewers, i.e., hard of hearing, deaf and Deaf receive the subtitles and react to the transcoding used. The film chosen for the study is “Dunkirk”, a 2017 history film directed by Christopher Nolan, which has more music and sound (by Hans Zimmer) than speech, and thus the verbal subtitling conveys both the diegetic sounds (environmental sounds, alarms, cries usually in battle fields, heard by the actors) and the non-diegetic music (the music itself, intended only for the audience) while taking into account the different genres of music: classical music which merges into electracoustic music and conventional instruments which co-exist with other non-conventional sound sources. The findings are used to produce best practices and inform the subtitling process with a view to optimising SDH and meeting the needs of the intended audience.

You can find the link for the presentation slides below:

https://languages-media.com/public/2024/slides/Vilelmini_Sososoni&Renata_Dalianoudi.pdf

References

Dalianoudi, R. (2021) “History-Music-Film. From the national epos of 1821 to the musical epos of commercial cinema” [In Greek]. Proceedings of the 10th International Conference of the Holy Synod of the Church of Greece The impact of 1821 on literature and art, 18-20 November 2020, Athens, Greece: Archontariki, p. 283-309.

Darrow, A. A. (1993) “The role of music in deaf culture: Implications for music education”, Journal of Research in Music Education, 41(2), 93-110.

Gouge, P. (1990) “Music and profoundly deaf students”. British Journal of Music Education, 7(3), 279-281.

Kivy, P. (1997) “Music in the movies: A philosophical enquiry”. In R. Allen and M. Smith (eds) Film Theory and Philosophy. Oxford: Claredon Press, 308-328.

Monaco, J. (1981) How to Read a Film. The Art, Technology, Language, History, and Theory of Film and Media. New York and Oxford: Oxford University Press.

Neves, J. (2010) “Music to my eyes… Conveying music in Subtitling for the Deaf and the hard of hearing”. In Ł. Bogucki & K. Kredens (eds). Perspectives in Audiovisual Translation. Lódz Studies in Language. Vol. 20. Frankfort am Main: Peter Lang GmbH, 123-145.

Revuelta P., Ortiz T., Lucía M.J., Ruiz B. and Sánchez-Pena J.M. (2020) “Limitations of Standard Accessible Captioning of Sounds and Music for Deaf and Hard of Hearing People: An EEG Study”. Frontiers in Integrative Neuroscience. 14:1. doi: 10.3389/fnint.2020.00001

Shibata, D. (2001), “Brains of deaf people rewire to ‘hear; music”. https://www.sciencedaily.com/releases/2001/11/011128035455.htm [last accessed 15/3/2023]

Zárate, S. (2021) Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences. UCL Press. https://doi.org/10.2307/j.ctv14t478b

 

Vilelmini Sosoni

Dr Vilelmini Sosoni is Associate Professor at the Department of Foreign Languages, Translation and Interpreting at the Ionian University in Greece. In the past, she taught Specialised Translation in the United Kingdom at the University of Surrey, the University of Westminster, Roehampton University and the University of Wolverhampton, and in Greece at the National and Kapodistrian University of Athens, Metropolitan College and the Institut Français d’ Athènes. She also has extensive professional experience having worked as a professional translator, editor and subtitler. She studied English Language and Literature at the National and Kapodistrian University of Athens and holds an MA in Translation and a PhD in Translation and Text Linguistics from the University of Surrey. Her research interests lie in the areas of the Translation of Institutional and Political Texts, Corpus Linguistics, Audiovisual Translation and Accessibility, as well as Machine Translation and MTPE. She is one of the chairs of the international conferences New Trends in Translation and Technology (NeTTT) and Translating and the Computer (TC). She was the vice-president of the Hellenic Society for Translation Studies and a founding member of the Laboratory "Language and Politics" of the Ionian University and the Greek Chapter of Women in Localization. She is also a member of the Advisory Board and the Management Board of the European Master’s in Technology for Translation and Interpreting (EM TTI) funded by Erasmus+. She has participated in several EU-funded projects, notably Resonant, Resonant+, Trumpet, TraMOOC, Eurolect Observatory and Training Action for Legal Practitioners: Linguistic Skills and Translation in EU Competition Law, while she has edited several volumes and books on translation and published numerous articles in international journals and collective volumes.