top of page

Integrative approach to mutlimodal narratives

My research triangulates multimodal semiotic analysis (substantially drawing on functional semiotic theories) and the findings from socio-cultural, aesthetic and empirical studies.         

​

On the basis of the integrative approach, I have published papers on the following research themes (most papers in pdf are downloadable from my publication list): 

Research topics

Analytical methods of multimodal narratives

  • Cohesion

  • Actions

  • Narrative space and time

 

Combining social issues and empirical studies of media effects:

  • Pro-social messages and persuasive functions

    • Characters' actions, point-of-view and motivations as persuasive devices for social influences

    • Pro-social and critical effects of blending fact and fiction in graphic, filmic and interactive narratives   

  • Narratives in science, risk/benefit communication 

  • Screen violence

 

Media and aesthetics

  • Transmedia, cross-cultural adaptation

    • E.g. cross-cultural adaptation of a Kurosawa's film High and Low and a detective novel 

    • E.g. comparisons between the meta-fiction City of Glass and its comic adaptation  

  • Character analysis and development

  • Non-linear/puzzle films

    • Embedding and blending of narrative space in complex films

    • Film beginnings and Film endings in complex films 

    • Cohesive devices in scene transition in puzzle films such as Memento, 2046, The Fountain, etc. 

  • Genre comparison

    • Comparisons of action patterns between War and Western films 

    • Silent thriller graphic novel and meta-fiction comics 

  • Authorship

    • Cohesion and auteur features in Ingmar Bergman's films

  • Multimedia frames and channels in film and graphic novels​

  • ​Narrative space in film as theatrical performing space

​

Multimodal literacy:

  • Design of transmedia narratives for teaching and learning

  • Children's comprehension of time in audiovisual narratives

​

Testing multimodal semiotic methods with empirical investigations:

  • Narrative strategies and AI (automatic detection of narrative patterns in film and news videos)   

  • Cohesion devices and eye-tracking experiments

  • Cohesion devices and neural functional analysis (fMRI)

  • Temporal features in film and children's comprehension tests

​

​

bottom of page