|
|
|
| 2019年度 異文化言語教育評価論 |
Chapter 4
Investigating
Stakeholder Perspectives on Interact
4.1
Introduction
- The
introduction of interact meant a shift from assessment tasks with
recommended topic areas, vocabulary and grammatical structures, to an
open-ended spontaneous and unrehearsed interactions, which did not require any
particular topic, vocabulary or grammar.
- In the
early stages of planning interact, there were divided opinions on its
efficacy. As explained in Chapter 2, communicative proficiency cannot be
measured by creating a theoretical definition and then planning a test that
would asses speaking performance based on that definition. There are several
dimensions to planning a test:
- assessment for learning / of
learning
- fulfilling a task / knowledge of a construct
/ communicative proficiency
- individual / pair/ group
-
interact is
- intended for learning, but used as
a summative measure
- focusing on both task completion
and construct knowledge
- conducted in pairs, but used to
measure an individual
4.2
Bachman and Palmer's Test Usefulness Framework
- This
framework provides 6 dimensions, which describe how the assessments will be
conceptualized and enacted:
- construct validity
- reliability
- interactiveness
- impact
- practicality
- authenticity
4.2.1
Construct Validity and Reliability
- Task
considerations (what students are required to do in the task, expected
outcomes) are important.
-
Construct validity can be achieved by relating speaking proficiency to a
theoretical model, or by defining the facets of a specific construct which is
to be operationalized (apologizing, negotiating, complaining).
- Test
reliability is concerned with how consistently (test-retest reliability,
parallel-form reliability) performances are measured.
4.2.2
Interactiveness, Impact, Practicality and Authenticity
- There
are three individual characteristics of Interactiveness which influence how the
test-taker will engage with the task:
- language ability: does the task
allow them to fully demonstrate their ability?
- topical knowledge: does the
test-taker know enough to answer the questions
- affective response: positive /
negative effect on test-takers mental state
- Impact
is the influence of the task on the test-takers.
- macrolevel impact: what will the
assessment be used for (stakes)
- microlevel impact: environment
(hot/cold..), stress related to the height of the stakes
- Practicality
is concerned with whether there are sufficient resources (people, time,
materials) to carry out a certain type of test.
- Authenticity
(as noted in Chapter 1), is how the assessment relates to target language use
outside of classroom (TLU domain). A speaking task may focus on situational
authenticity, interactional authenticity, or both.
- situational authenticity: in
classroom, we can only imitate real-life interactions
- interactional authenticity: the
same language they would use outside of classroom
- It is
important to combine these six characteristics to achieve maximum overall
usefulness. However, not all characteristics must be equally distinguishable.
- For
example, a formative communicative task with no stakes for the test-taker
requires high interactional authenticity, but reliability doesn't have to be
considered. A high-stakes grammar test might value reliability over
authenticity or interactiveness.
- However,
high-stakes FL students' spoken communicative proficiency assessment arguably
requires all six qualities to be noticable. Such tests should:
- be construct valid (representative
of the thing we wish to measure)
- be reliable
- promote positive impact and
positive interaction
- be as practical as possible, while
collecting meaningful data
- be at least interactionally, but
possibly also situationally, authentic
4.3
2011 Onwards: Interact in Practice
- Teachers
have been provided with a wide range of resources to help them replace converse
with interact. There were two important aspects in this change.
(1) The
shift to open-ended task which doesn't require a particular grammar or
vocabulary
- outcomes judgements are influenced by CEFR
- the assessment cannot expect students to go
beyond their level, test-takers should show their ability "in a natural
way"
- fluency is measured as the ability to enter
conversation without rehearsal
- grammatical accuracy is only measured only
if they hinder communication
|
NCEA level 1 |
NCEA level 2 |
NCEA level 3 |
|
Show evidence of language to communicate personal information, ideas
and opinions in different situations |
Show evidence of language to share information and justify ideas and
opinions which involves communicating information, giving explanations or
providing evidence to support own views and/or the views of others |
Show evidence of language to explore and justify varied ideas and
perspectives which involves finding out about, evaluating and giving
explanations or evidence to support or challenge the ideas and perspectives
of others |
Fig 4.1 Outcome requirements of interactions
(NZQA, 2014c)
(2) The
shift from a one-time summative test model to an on-going formative assessment
- assessment of students' discourse and
sociolinguistic competence can be measured by how they adapt their language
choices to fit different contexts
- feedback and feedforward are encourages, but
the final product must be a result of student's own ability
- Written / audio-recorded annotated exemplars
for several languages have been published (Ministry of Education, 2014b).
- The fact
that interact is used for high-stakes assessment and thus must be
notified in advance etc. potentially negates the validity of the task as an
impromptu measurement (of the language that students use in the process of
completing a task).
Questions:
1. Does
announcing the test in advance really negate the impromptu character of the
assignment? If yes, what step could be taken for the assignment to promote
unrehearsed communication.
2. What do
you think are the perspectives of stakeholders such as universities or workplaces,
which will use this measurement to decide whether to accept them.