EALTA Summer School 2019 : Setting and maintaining standards across languages

CIEP, British Council Assessment Research Group, ACTFL


Summer school : Monday 15 July 2019 to Thursday 18 July 2019
Open Colloquium: Friday 19 July 2019

CIEP, Sèvres, France

Logo EALTA

The Summer School will focus on standard setting in relation to the CEFR, with a particular focus on establishing and maintaining standards across assessments in different languages. The focus will aim to address the issue of selecting a strategy for linking in testing programs that involve assessments in more than one language. The sessions also aim to expand the framework for linking exams to the CEFR outlined in the Manual (Council of Europe, 2009) to emphasise the importance of multiple sources of evidence and strong construct definition as an integral part of the alignment process. The Summer School will use Reading tests as the focus, and will propose a method of aligning tests across multiple languages to a common standard such as the CEFR through the use of Multi-Facet Rasch Measurement (MRFM) with the FACETS program.

An exciting addition to the program of this year’s Summer School is a colloquium, open to the public, on issues related to Standard Setting with the CEFR. The Colloquium will be on the final day, Friday 19th July. The colloquium will bring together a number of prominent invited speakers to discuss topical issues related to aligning tests with the CEFR, including the main focus of the Summer School, setting and maintaining standards across languages. In addition, the Colloquium aims to answer the call made at the 2017 EALTA conference in Sevres for more interaction and information exchange between the field of language testing and non-experts in the government and policy-making arenas that have direct involvement in commissioning and / or running language assessments.

Jamie Dunlea (ARG, British Council), Richard Spiby (ARG, British Council), Vincent Folny (CIEP), Margaret Malone (ACTFL). The trainers have extensive experience of standard setting in a range of contexts and bring experience of aligning assessments not only to the CEFR but a range of international frameworks and standards.

Summer School (includes Colloquium): 285 euros (covers all training and materials, coffee breaks Monday to Friday, lunches Tuesday to Friday, opening drinks reception on Monday night).
Colloquium only: 20 euros (includes coffee breaks and lunch)

Week schedule

 




  • 3:00 pm : Check in and register
     
  • 3:30 pm
    • Introduction to workshop, overview of schedule, housekeeping.
    • Overview of alignment and linking to the CEFR: background, materials developed by Council of Europe (Manual, Supplement, example items especially the recent Reading and Listening additions, example performances), case studies of linking projects.
    • Focus on standard setting across languages: CEFR and ACTFL case studies.
       
  • 6:00 pm : Evening: Reception

 

Morning
  • Overview of standard setting methodologies
  • Focus on standard setting for tests of reading
  • Training and familiarization with CEFR Reading scales

Afternoon
  • Carry out standard setting on EFL Reading test items
    • Basket Method :
      • Explain method, make individual R1 judgments
      • Normative feedback on panellists judgments and discussion
      • Finalize judgments

    • Modified Angoff approach
      • Explain method, make individual R1 judgments
      • Normative feedback on panellists judgments and discussion
      • Empirical feedback
      • R2 judgments

  • Reflection and discussion on processes
Morning
  • Explanation of approach (conceptualization of common minimally competent candidate across languages to enable FACETS linking of results).
  • Split into groups for languages
  • Choose your weapons (methods).
  • Design your schedule and activities for the day.

Afternoon
  • Carry out standard setting for Reading items
  • Reflection and discussion on processes (whole group)
     
Morning
  • FACETS linking results of test items across different languages on the same CEFR scale.
  • Discussion and reflection from the group on possible applications for their own contexts.

Afternoon
  • Invited expert session: Workshop session on text analysis tools for multiple languages to help define criterial features of proficiency levels (contributing to the multiple sources of evidence needed to support linking claims beyond standard setting).
  • Whole group discussion on standard setting processes, challenges, and applications to their own contexts.
     
Colloquium open to the public.

9:00 am : Registration opens.
9:30 am : Opening.
9:45 to 10:30 am : Plenary 1 Neus Figueras: the Companion volume: implications for standard setting with the CEFR.
10:30 to 11:00 am : Coffee break.
11:00 to 12:30 am : Symposium 1: Standard setting across languages, techniques and challenges: Meg Malone, Gregory Cizek, Carol Spoettl, Neus Figueras.
12:30 to 2:00 pm : Lunch break.
2:00 to 2:45 pm : Plenary 2: Thomas Francois: Text analysis tools for describing languages.
2:45 to 3:30 pm : Plenary 3: Constant Leung: Harnessing linguistic resources of linguistically diverse societies.
3:30 to 4:00 pm : Coffee break.
4:00 to 5:30 pm : Symposium 2: Setting standards in challenging contexts: Juliana Bahia (Canadian Foreign Service Institute), FCO, Mark Crossey (British Council, experience as testing coordinator and director for Global Peacekeeping English Project), Constant Leung (Kings College, London), Meg Malone.
5:30 pm : Closing.

 

Registration

Complete the registration form.

You will receive a confirmation of your registration within 5 days (registered activities).
For any change in your registration please contact ealta@ciep.fr before payment Secured online payment should be done after you receive instructions by email. Please take note that there is no reimbursement policy.

 

CIEP Accommodation

https://www.research.net/r/RL5TCTF

Accommodation is available at CIEP at a cost of 53 euros per night. Other accommodation options are also available in the area.