Centre for Academic and Instructional Development - Interpreting ICE

Interpreting ICE

The ICE system at Saint Mary’s will be used for both formative and summative evaluation purposes.

Formative Evaluation

Summative Evaluation

Reflecting on Student Feedback – A Developmental Framework

Formative Evaluation

This is the primary role of the new ICE system – to provide data that can be used by faculty to make improvements in teaching.  Improvements might eventually affect tenure and promotion decisions, but the primary intent in formative evaluation is faculty development.

For this reason, the data generated by ICE on the Instructor’s Report will be seen by individual faculty only and will allow instructors the opportunity to review student data and open-ended comments, compare this with your own self-rating for the course, and interpret the data in an attempt to understand the feedback given by students, and how that feedback could inform individual teaching improvement. (Return to Top)

Summative Evaluation

The report of the SEEQ (ICE) Implementation Committee, November 2005, stated “The Committee has consulted the extensive literature on summative evaluation in regard to student evaluation of teaching instruments.  We have found strong consensus among researchers in the area as to what should and should not be included in a summative report.  The literature indicates that a report should include the results of overall, or summative, questions.  A report should not include the results of questions which are primarily formative as they attempt to measure instructor-specific characteristics and his/her teaching/administrative style/practices. Research indicates that faculty members and administrators alike have preconceptions about what personal characteristics and pedagogical practices are correlated to teaching effectiveness.  Individuals who do not conform to these preconceptions are likely to be judged ineffective despite other evidence of teaching effectiveness”.

Based on recommendations in the research literature, the ICE system at Saint Mary’s will provide academic administrators with the results of the two overall or global questions only for summative decision making purposes on both the Summative Report and the Annual Summary.  (Return to Top)

Reflecting on Student Feedback – A Developmental Framework

To help instructors reflect on the feedback provided by students, the following framework is recommended.

1.  Download and complete the Self-Rating Form before opening the ICE Reports and Open-Ended Comments for a particular course.

Think about what happened in this course this term?  Was the class a positive experience for you?  For students? Why?  Did things go as planned? Why not?  What will students say about the course, about their learning experience?

2. Analyze the Student Data section of the Instructor’s Report in conjunction with your own knowledge of the class.

What special or unique features are there in regard to the subject, the students or other contextual factors?

3. Analyze the Instructor’s Report.

• What are the main strengths of your teaching? What are the main weaknesses?
• Do these findings differ from your Self-Rating Survey responses? How?

4. Analyze Students' Comments to the Open-Ended Questions

The original ICE forms completed by students will be returned to instructors along with the ICE Reports because they contain student responses to the open-ended questions. Instructors will receive student comments as they were written.  You might wish to record them electronically in a Word document or Excel file for future reference.

 Avoid the temptation to just glance at student comments as you could miss valuable information. Read through all comments carefully and categorize them in order to identify common themes. Students often use this opportunity to include comments on teaching methods, learning activities or assessment items that worked well for them in other courses - use this information to your advantage.

• What characteristics of instructor/course did students find most valuable to their learning?
• What characteristics of instructor/course did students suggest needed improvement?
• How do the comments compare to the ratings on the Instructor’s Report? On your Self-rating survey?

5. Select Key Areas for Improvement.

• Which are the one or two areas that received the lowest ratings? Check the Instructor’s Report for specific responses and the open-ended comments to supplement information.
• What strategies for improvement could you use that would be appropriate to your particular situation?

6. Suggested Follow-Up Actions

After reflecting on the responses to the above questions you might want to:

• Refer to the Targeted Teaching Strategies booklets or other resource materials held in the Centre for Academic and Instructional Development to help you increase your knowledge.
• Engage in some further reflective practice by conducting some form of classroom-based research into different teaching and learning approaches and techniques. Ask your students about their needs, their perceptions of teaching and learning, or what constitutes successful practice in the classroom.
• Invite a colleague to act as a mentor, to review your Instructor’s Report and students' comments and to discuss possible strategies for achieving improvement.
• Ask for a meeting with the Director, CAID, to discuss your students' feedback and strategies for improvement.

If you invite a colleague or the CAID to assist you in this process, it does not necessarily follow that they need to have expertise in your subject area or even know what is "best". Their role can be that of facilitator, discussing the feedback with you and helping you to think of some strategies in areas you have identified for development.

7. Closing the Loop

• Through the evaluation process, students have provided important feedback to help you identify your teaching strengths and areas for improvement. To acknowledge their part in the I/CE process, you could tell students in a following course or students enrolled in the same course when it is running again, about the changes you have made based on the feedback/comments from previous students. By doing this, students will feel that their opinions are valued and they have made constructive input into how they are being taught.

• Keep a record of what you have learned, how you have responded to student feedback – and use this information to develop your Annual Report and, over a longer period of time, your Teaching Dossier.

(Return to Top)