(Full marks: 100)
Question 1
Using Generative AI, design a classroom-based English language test comprising 10 multiple choice questions and one essay question with a mark scheme or rubric. This English language test is used as assessment for learning in your class.
Your ECA submission must have the following sections:
a) Introduction (10%)
Based on a framework or taxonomy, describe the skills and levels you have selected for this English language test to be assessed. Discuss why and how these skills and levels are critical in your context.
b) Test construction using Generative AI (45%)
In this section, you are required to use Generative AI (e.g. ChatGPT3.5, Claude 2) to generate 10 multiple choice questions and one essay question with a mark scheme or rubric. You are required to:
i) provide a test blueprint or table of specifications for this test (5%)
ii) provide all prompts that you used to generate multiple choice questions, essay
question and mark scheme or rubric in Appendix (5%)
iii) explain your prompting process, from the beginning to the end of your question and
mark scheme or rubric generation process, and justify why and how these prompts
are critical in designing your classroom language test (15%)
iv) generate 10 multiple choice questions (10%)
v) generate 1 essay question with a mark scheme or rubric (10%)
c) Critique of test based on Messick’s construct validity framework (15%)
Critique the 10 multiple choice questions and one essay question with a mark scheme or rubric that you generated from Generative AI using Messick’s construct validity framework in relation to construct irrelevant variance and construct underrepresentation.
d) Examination of the process of designing classroom language tests (20%)
Reflect on the shift towards connecting assessment with learning in the era of digital technology and Generative AI. Discuss how they can be applied to your classroom learning and teaching.
e) APA citation and referencing (10%)
The overall perception of the quality of the assignment will be impacted (and reflected in the marks awarded) if there is non-conformity to the APA.
The weighting of the different sections merely gives an indication of their relative importance. Your ECA will be marked based on the following criteria.
Observance of the directions listed in the question
Reasoning and evidence of connection to in-context examples
Appropriate and innovative test items and tasks
Adherence to the APA guidelines
The word limit for your ECA is 3500 words (Times New Roman 12-point font size, double line spacing, 2.54cm margins all round) excluding references and appendices. Any text beyond the word limit will not be graded.
