AI and Multimodal Learning

Project Name

AI and Multimodal Learning

Project Description

This project’s aim is therefore to explore and evaluate the opportunities brought by the use of GenAI tools within education to support disciplinary knowledge creation in the form of multimodal learning and the development of digital capabilities by students and staff.

Our interest in multimodality follows Kress’ (2010) social semiotic approach, which explores the ways in which different semiotic modes (text, speech, sound, image, moving image, touch, gesture etc.) are present and combined within one communication to produce a multimodal artefact. For instance, an infographic combines text and image in ways that produce extra meaning. Multimodal learning is concerned with designing learning utilising multimodal texts across media, forms and formats appropriately to the given context (van Leeuwen, 2017). Other examples of multimodal texts include posters, video, dance or virtual simulation. Multimodal learning also involves the use of ‘semiotic technologies’ such as collaborative tools, visualisation apps, online quizzes to support teaching practice. The ascent of GenAI offers further expansion to eductors’ repertoire of semiotic technologies.

GenAI is a form of technology that uses deep learning techniques to access a huge swathe of data and produce artefacts based on the prompt(s) provided to it by a human user, is increasingly capable of producing multimodal content, including (but not limited to) text, speech, audio, image, video and even three-dimensional models (Fui-Hoon Nah et al., 2023). Multimodal GenAI can be, and is being, used to create, manipulate, and adapt content and combine different semiotic forms together, to produce multimodal artefacts.

The proposed guide will offer practical learning designs for integrating present and future freely-available GenAI tools for multimodal learning, paying attention to equality, diversity and inclusivity, in three strands:

  • Teaching, i.e. how GenAI can be used to represent subject knowledge multimodally,
  • Learning, i.e. how students could encounter, explore, evaluate and express ideas via multimodal GenAI, 
  • Assessment, i.e. how students’ using GenAI to create or critique/reflect on multimodal artefacts.

The guide will support educational developers (and educators) in understanding the potential and challenges of integrating GenAI into teaching/learning practices.

Project Members

Project Leader: Varga-Atkins, Tünde document-files.png (Centre for Innovation in Education, University of Liverpool, UK), Saunders, Samuel document-files.png (Generative Artificial Intelligence Network, Centre for Innovation in Education, University of Liverpool, UK)

Project Members: 

Dr Na Li (XJTLU, China), Dr Wen Run (XJTLU, China), Sue Beckingham (Sheffield Hallam), Dr Sam Elkington (Teesside), Professor Peter Hartley (Edge Hill), Nayiri Keshishi (Surrey), Dr Nataša Lackovic (Lancaster), Dr Isabelle Winder (Bangor), Rob Lindsay (University of Liverpool)

Project Outcome

Research grant:

  • 2024 XJTLU SURF, award number: 000000000339, project title: A Guide to Utilising Multimodal Learning and Generative Artificial Intelligence for Enhancing Student-Centred Active Learning and Teaching in Higher Education

Award:

  • SEDA Research and Evaluation Small Grants 2024, Project title: An Educational Developer’s Guide to Multimodal Learning and Generative AI (artificial intelligence). Principal investigators: Dr Tunde Varga-Atkins & Dr Sam Saunders, University of Liverpool, Sum awarded: £1,000.00  

Event

  • Saunders, S., Varga-Atkins, T., Beckingham, B., Hartley, P., Keshishi, N., Lackovic, N., Li, N., Lindsay, R., Wen, R., Winder, I. (organisers and presenters). (Jan 21, 2025). Beyond Text: Symposium on Multimodal Teaching, Learning and Assessment. [online]

Invited talks

  • Saunders, S. & Varga-Atkins, T. (Feb 4, 2025). SEDA talk on Impact and Implications on Generative AI.
  • Saunders, S. & Varga-Atkins, T. (4 Dec, 2024) Using Generative AI in Multimodal Contexts to Develop Assessment Literacy. Advance HE Assessment festival. [Presentation]
  • Varga-Atkins, T., & Saunders, S. (Oct 28, 2024). Multimodal Learning and Generative AI. GAIN network webinar.
  • Varga-Atkins, T. (Oct 15, 2024). Multimodal methods and Generative AI. [presentation.] Scopicasting: Contemplating the future of education methods with visual/multimodal media. Lancaster University. 
  • Beckingham, S. and Hartley, P. (2025) Will Generative AI reshape learning, teaching and assessment in higher education? Senior Fellows Network. Lancaster University. [Invited Speaker]
  • Beckingham, S. and Hartley, P. (2024) Getting ready for the wave(s) of Generative AI. EmpowerEd Webinar, Keele University [Invited Speaker]
  • Beckingham, S. and Hartley, P. (2024) Will Generative AI reshape learning, teaching and assessment in Higher Education? Visayas State University, Visca, Baybay City, Leyte, Philippines. [Keynote]
  • Beckingham, S., and Hartley, P. (2024) What should we do now with Generative AI? 2024 National Conference on Generative Artificial Intelligence in Education. Ulster University. [Keynote]
  • Beckingham, S., Hartley, P., Powell, S. and Lawrence, J. (2024) Using Generative AI Effectively in HE teaching, learning and assessment: ethical and sustainable pedagogic practice. Oxford Brookes University International Teaching and Learning Conference 2024. Oxford Brookes University. [Invited Speaker]
  • Beckingham, S. and Hartley, P. (2024) Looking ahead to the next waves of GenerativeAI: applications and implications for HE staff and students. SOLSTICE and CLT Conference, Edge Hill University. [Invited Speaker]
  • Beckingham, S. and Hartley, P. (2024) What do we do now with GenerativeAI? ALT ELESIG Scotland. [Invited Speaker]
  • Beckingham, S. and Hartley, P. (2024) Looking ahead to the next waves of Generative AI: applications and implications for HE staff and students. Talking Teaching across the Globe, Oxford Brookes. [Invited Speaker]
  • Beckingham, S. and Hartley, P. (2024) Reshaping Higher Education Learning, Teaching and Assessment through Artificial Intelligence: what do we need to know, do, and be concerned about? Pedagogic Research Conference, University of Liverpool. [Keynote Speaker]

Infographic

Podcast

Publications

Presentation/Conferences

  • Varga-Atkins, T. & Saunders, S (Nov 28, 2024). How is Generative AI being used in multimodal learning, teaching and assessment in higher education contexts? A scoping literature review. SEDA Autumn Conference. [Presentation online]
  • Saunders, S. (2024, November 28). Introducing the Generative AI and Multimodality Project. Staff and Educational Development Association (SEDA) autumn conference 2024: New challenges for educational development, Online.
  • Beckingham, S. and Hartley, P. (2024) In search of ‘Responsible’ GenerativeAI as an essential prerequisite for future-proofing education for all. International Federation of National Teaching Fellows (IFNTF) Symposathon.
  • Beckingham, S. and Hartley, P. (2024) In search of 'responsible' Generative AI. Provocation. Autumn SEDA Conference. 
  • Beckingham, S. , Pickard-Jones, B., Short, F. and Hartley, P. (2024) Generative AI and Educational Development: opportunity, challenge, or potential disaster? SEDA Spring Conference. Liverpool John Moores University. 

Multimedia content

Webinars

Recruitment

Comments

Add comment

Fields marked by '*' are required.
Comments are moderated. If you choose to make this comment public, it will not be visible to others until it is approved by the owner.

Reply to:

Public
Private: This reply will only be visible to you and the author of the preceeding comment.
Updated on 08 March 2025, 10:17 AM; 1145 page visits from 10 January 2024 to 5 April 2025