|
Description
The overall scheme consists of 4 parts: critiquing a standardised test; developing a PowerPoint presentation on a topic of interest (eg IQ tests); reflective writing about the experience of problem-based learning tasks (PBL) and self-assessment, and a ‘culminating task' involving carrying out a psychoeducational assessment and writing a professional-standard report.
Of particular interest is the assessment activity related to the PBL tasks. Linda gives her students PBL tasks which introduce them to Kevin, a fictitious school student, who is having problems at school and home. Over three stages, case information is gradually released in order to try to simulate a real-life experience as closely as possible, much as it would be in professional practice. Students independently reflect and then discuss the PBL tasks on the discussion board, developing hypotheses and outlining plans of action for how to proceed with Kevin's case. The three stages of the PBL tasks are aligned with the sequence of the study book modules and students are encouraged to apply the course content and theory to their responses and discussions. At the completion of the PBL tasks, students write a reflective piece where they synthesise the experience of the PBL tasks in terms of what they have learned about assessment (knowledge and skills), the development of their own frameworks and approaches for assessment, and what they have learned about their own learning processes. At that time they also self-assess their own preparedness to undertake the assessment processes required for the culminating assessment task, and identify skill areas on which they need to work before attempting the final assignment. The mark a student awards herself/himself comprises 10% of the overall mark.
Learning goals and objectives
The idea of the total assessment scheme is to use the first three assessment tasks formatively to assist students to develop the skills necessary for the culminating final assessment report.
Given the external offering of the course, the problem-based learning approach was incorporated into the course assessment to try to simulate a real-life experience as closely as possible, much as it would be in supervised professional practice. The course objectives assessed by this task include demonstrating: an understanding of the nature of assessment and how assessment data are used in educational decision making, and an understanding of a range of issues relating to assessment practice in inclusive settings, including assessment accommodations, ethical-legal practice, and assessment standards.
In terms of broad graduate qualities and capabilities, Linda is seeking to develop and assess students discipline knowledge and skills, research and enquiry skills (problem-solving skills, analytical and critical thinking skills), self-management skills (independent learning skills, reflection and self-evaluation, intellectual autonomy), and personal and professional responsibilities (personal and professional ethics).
Target audience
Students taking the Masters of Education course EDU 8317: Educational Measurement and Assessment in Inclusive Education. Most students are majoring in either Guidance and Counselling or Special Education.
Informing students
Assessment task requirements are provided in the introductory book, and are supplemented by FAQs responses posted to the Study Desk. Very detailed information and examples are provided to avoid ambiguity and misunderstanding.
Feedback
Effective feedback for all assessment items is facilitated in a number of ways: (1) use of the detailed marking criteria sheet/rubrics for each task which provides students with general feedback about their performance on each criteria, (2) use of marking codes to allow markers to flag within the essay common written communication problems as well as good points eg ‘sp' for ‘spelling, ‘[' for ‘new paragraph required,' ‘4' for ‘reference required,' ‘8' for excellent point, etc, and (3) developing a bank of detailed comments which markers can cut and paste into the comments section of the electronic version of the rubric.
Given the external offer of the course and with students often in multiple (global) locations it can be difficult to get feedback to students in a timely manner. Linda is currently considering the benefits of electronic submission and return for improving timeliness of feedback.
For the self-assessment, the feedback is immediate in the sense that the students mark their performance using the detailed rubric, and based on this determine the actions they need to take to develop required knowledge and skills.
Roles
Linda worked with a past staff member in 2005 to revitalise the assessment approaches in the course. Apart from occasional marking assistance, Linda manages and teaches the course. Linda monitors the PBL task discussion board component of the self-assessment but for the most part she tries not to "interfere" too much and encourages the students to lead the discussions in this postgraduate course.
Moderation processes
In developing a scheme like this, Linda stresses that it's vitally important to have a clear rubric to improve reliability, especially for the self-assessment task. For the self-assessment item, Linda finds that students are surprisingly hard on themselves and very few students award themselves top marks. It is not unheard-of for students to award themselves zero. But generally the marking distributions of self-assessed tasks closely approximate the distributions of the staff marked items. Unless there is a strong reason to do so, and as evidenced in the reflective writing, Linda does not upgrade or downgrade students' self-assessed mark as she believes it is important to respect the judgement of these students about their own learning.
When there is more than one marker for staff-marked items, Linda undertakes moderation by first looking at the distributions and descriptive statistics for each marker, followed by blind-marking a sample of each marker's assignments to determine reliability and consistency of marking.
Results
EDU8317 student comments from course developed surveys and unsolicited emails show that students are generally pleased with the overall approach to assessment in the course:
"I thought …having four [assessment] items instead of the usual two was very effective in reinforcing required learnings."
Comments about the PBL tasks suggest that this supports their engagement with the course and subsequent learning:
"Loved the PBL …very useful to gauge if you are on the right track".
"PBL great activity, Study desk [discussion] was also great. Felt like I was on campus!"
Problems and advice for others
It has also been a challenge to find the right weighting for the individual components of the assessment scheme, especially deciding on what weighting to give the self-assessment component. The biggest challenge is getting past student disbelief at being required to award themselves their own marks. Linda does this by pointing out that most of them are already professional educators who are capable of the task, and that the rubric is sufficiently detailed to guide them. Once they understand what is required, students do not find it onerous and enjoy the process. Some of the comments that Linda has received about the course, and in particular the self-assessment, have been very gratifying, and she encourages others to consider the inclusion of a self-assessed component, particularly at the postgraduate course level.
Additional comments
In the future, Linda plans to develop the PBL task resources further through the incorporation of audio visual materials.
|