Published Mar 22, 2009
VickyRN, MSN, DNP, RN
49 Articles; 5,349 Posts
Required discussions of course readings provide motivation for students to learn course content and can be used to validate students' comprehension of course content and processes. A tool for grading the students' weekly discussions of course work was tested to evaluate the interrater reliability of grading by two faculty members. The purpose of this article is to describe psychometric testing of the interrater reliability of this grading method. Using the grading tool, independent ratings of five students' online discussion postings were recorded by both faculty members over a 5-week period, which provided the data for this study. Data were analyzed using Spearman [rho] and Kendall [tau]-b statistics. The findings revealed that the overall correlations of rater scores were satisfactory and indicated that an acceptable level of interrater reliability was obtained through use of the grading tool. Reliable tools for evaluation of students' online discussions contribute to the knowledge needed for the implementation of online courses.
http://www.nursingcenter.com/pdf.asp?AID=830886
Very helpful resource! :)
marachne
349 Posts
I am all for using Rubrics where ever and when ever it makes sense to do so. The up front work is pretty intense, but after that, grading is so much easier and faster -- and by giving the students rubrics up front, you cut off a lot of the questions that, basically are "what are you looking for?"
It also provides for consistency when multiple individuals are involved with grading.
And, you don't find yourself making the same comments on individual's papers because that's all in the Rubric. Individualized comments come a lot easier if so desired.
The best is when the program has overall competency rubrics, the course has overarching rubrics, and each substantive assignment has a rubric.
They are also good tools for course development -- if you figure out what you want them to learn (what you are judging them on) and then work backwards from that, your content reflects you expected outcomes.
A great book for learning how to design rubics is Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and P:romote Student Learningby Stevens and Levi Stylus Publishing, 2005.