ABOUT/CONTACT SEMINAR SLAA （Second Language Acquisition & Assessment Research Group）  FOR STUDENTS  TEASY  LINK

---

Chapter 4

Investigating Stakeholder Perspectives on Interact (4.4 to 4.8, pp.87-98)

This part evaluates the fundamental principles of interact against the theoretical framework for usefulness and presents the methodology for the 2-year study that has sought stakeholder views (both teachers and students) during the initial phases of the implementation of interact (2012-2013).

4.4. The Theoretical Usefulness of Interact

According to Bachman and Palmer’s six qualities of test usefulness (1996), it seems that interact is a particularly valid and useful measure of spoken communicative proficiency:

Counter-arguments to a claim to usefulness:

Because theoretically, interact has both good points and bad points, there is a need to see what is happening with interact in practice.

4.5. A Study into Teachers’ and Students’ Views

The study investigates stakeholders’ perspectives on the assessment reform during the period of its initial roll-out (2012-2013).

Research questions:

-          What are teachers and students making of the innovation?

-          What is working, what is not working, what could work better?

-          What are the implications, both for on-going classroom practice and for on-going evaluation of the assessment?

Why investigate perspectives at an early stage of the implementation? To focus on the comparison of interact with converse.

Data collected during the 2 stages of the empirical study:

1.      2012 anonymous teacher survey

2.      2012 and 2013 interviews with teachers

3.      2012 and 2013 anonymous student surveys

4.6. Study Stage I

4.6.1. Nationwide Teacher Survey

Target of the survey = teachers of the 5 principal FLs taught in New Zealand (Chinese, French, German, Japanese and Spanish)

Construct measured by the survey = ‘perceived usefulness’ of both interact and converse

Perceived usefulness measured according to 4 sub-constructs of the usefulness construct:

1.      Perceived validity and reliability

2.      Perceived authenticity and interactiveness

3.      Perceived impact

4.      Perceived practicality

The survey

Section I            Some close-ended questions about the perceived usefulness of interact.

Prompting more precise and nuance responses (≠Likert scale):

Section II         4 open-ended questions:

1.      Comment on the perceived advantages of interact in comparison with converse

2.      Comment on the perceived disadvantages of interact in comparison with converse

3.      If you are using interact, describe briefly your experiences with introducing the new assessment at levels 1 and/or 2. If you are not using interact, explain briefly why.

4.      How might interact be improved?

4.6.2. Piloting the Teacher Survey

10 teachers => comment on their understanding of the statements + how long did it take to respond? Conclusion: the survey can be regarded as a reliable measure of teachers’ perceptions of the usefulness or fitness for purpose of converse/interact as assessments.

What did the pilot survey tell us?

-          Internal consistency of the survey is higher for perceived validity and reliability, but lower for perceived impact, and considerably lower consistency for perceived practicality

-          The fact that students might enjoy the opportunities to speak does not necessarily make the experience less stressful

-          The survey could be completed rather quickly

4.6.3. Administering the Main Survey

4.6.4. Teacher Interviews

Interviews (November – December 2012) with 14 teachers using interact about:

-          Interviewees’ understanding of the main purposes of interact, and opinions about interact

-          Comparisons and contrasts between interact and converse

-          Advantages and disadvantages of interact in comparison with converse

-          Experiences and challenges with the implementation of interact

-          Types of assessment tasks used and perceived student reception of these

-          Advice for others about how to implement interact successfully

Three key domains of concern arose from the survey and interview data:

-          The importance of the task

-          The concept of ‘spontaneous and unrehearsed’ (a focus on fluency)

-          A de-emphasis on grammar (the place of accuracy)

4.7. Stage II

4.7.1. Teacher Interviews

Interviews (November – December 2013) with 10 teachers who had not taken part in the Stage I interview + 3 teachers who had already participated in stage I interview.

Focus on the 3 key issues of concern from Stage I interviews at interact level 3.

4.7.2. Student Surveys

2 student surveys:

-          Year 13 students who had taken level 3 converse in 2012 (last year of operation)

-          Year 13 students who had taken level 3 interact in 2013 (first year of operation)

Similar as teacher surveys but simplified and focus on students’ perception about interact rather than comparison interact/converse.

Analyses of students’ responses focused on the 3 key issues identified from the teacher surveys from Stage I.

4.8. Conclusion

This book proposes two contrastive means of assessing spoken communicative proficiency: which of the 2 assessment formats realized in converse and interact better reflects assessments of spoken communicative proficiency that are valid, useful and fit for purpose?

This book is an attempt to address this issue by taking account of stakeholder perspectives in a study that is largely qualitative (independent and complementary data sources such as surveys and interviews), but also quantitative.

ð  The study is robust, and its findings will be discussed in the remaining chapters.

Discussion questions

1.      What kind of interact assessment can lay the focus on fluency? Provide examples of appropriate tasks for interact and explain why you think they are appropriate.

Where should be the place of accuracy (or grammar) in interac