2013+MA+Program+Evaluation

The 2013 evaluation of the MA program is based upon data collected during the calendar years of 2011-2013. The following are the two columns we are required to submit as the findings of our evaluation. See rubric following the table for the criteria for evaluating each learning outcome.


 * LO || Conclusions Based on Evidence || Rating || Actions Taken or Planned ||
 * Scholarship || There was one graduate student during the two calendar years 2011-2012 who dropped out of the program, and she did not meet the criteria for this learning outcome. The twelve other students who left the program did so by graduating, and all twelve of them successfully completed a project/thesis proposal and defense. Thus, they met the criteria for this learning outcome. || Excellent || Of the 13 graduate students who have left or finished our program over the past two years, only one did not meet this learning outcome. When the graduate committee met to discuss this student, they acknowledged that she was perhaps the weakest student admitted to the program, and that there had been concern at that time as to whether she would be successful in the program. Consequently, we feel that her failure in our program may be more of a result of our admissions process than the program itself. The graduate committee has discussed the issue of recruiting better qualified candidates for the graduate program, and the department has been pursuing an aggressive recruitment program. We will continue to monitor the issue of recruitment.

As for actions taken, we have established data collecting procedures so that the requisite information for assessing our learning outcomes is available. ||
 * Research || There was one graduate student during the two calendar years 2011-2012 who dropped out of the program, and she did not meet the criteria for this learning outcome. Eleven of the 12 other students met the criteria for this learning outcome. The graduate who did not meet this requirement received a 66% on sections II-III on the master's exam. || Excellent || We believe that the ability of most of our students to successfully defend their project or thesis and to pass sections II-III on the master's exam demonstrates that our program is preparing our students well for research. We will continue to monitor our students' performance in this area. ||
 * Mathematics || Eight of the 13 students who left our program during 2011-2012 received a score below 70% on section III on the master's exam. However, all 13 students passed two or more courses with substantial mathematical content. || Fail || The graduate faculty will meet together to discuss how to improve our students' understanding of the mathematical content of their courses. Also, because this outcome is determined almost entirely by graduates' responses to 2 questions on the master's exam, graduate faculty will discuss other possible measures of graduates' mathematical knowledge. ||
 * Teaching || Because of the newness of this learning outcome, we have do not have the requisite data for all of the graduate students who left the program during the calendar years 2011-2012. All 4 of the graduates who have both peer reviews and student ratings meet the criteria for this learning outcome. For 8 of the remaining 9 graduates, we have student ratings, and 7 of these meet the criterion for this learning outcome. || Good || Student ratings have been regularly collected for our graduate students. However, we need to improve in conducting and reporting peer evaluations of graduate students' teaching. ||
 * Professionalism || The MAPIBS, the instrument used to measure this learning outcome, has only been recently implemented. Two of the graduates have no MAPIBS scores. Four graduates have one MAPIBS score, all of which have 1 or fewer items marked as being unsatisfactory. This suggest that these 4 students have met the learning outcome. All 7 of the remaining students who completed or left the program during the calendar years of 2011-2012 had two MAPIBS scores, and only one student received more than 3 unsatisfactory ratings on the combined scores. This indicates that a large majority met this learning outcome. || Good || We have successfully implemented the MAPIBS. An MAPIBS is filled out for every student in every graduate class, and each graduate student's advisor fills out an MAPIBS for that student every six months. An additional MAPIBS is completed by the advisor after the graduate student's successful project or thesis defense.

Although the data is incomplete for measuring this outcome, the data we do have suggests that students are doing well in meeting this learning outcome. The one student who did not meet this learning outcome left the program without completing the degree. While we hope that every student will be successful in our program, we acknowledge that some students will fail. We believe that success on this learning outcome by most of the students who completed or left the program during the calendar years of 2011-2012 indicates that the program is strong in this area. We will continue to monitor our students performance in this area. ||
 * Spiritual Stewardship || The MAPIBS, the instrument used to measure this learning outcome, has only been recently implemented. Two of the graduates have no MAPIBS scores. Four graduates have one MAPIBS score, all of which have 1 or fewer items marked as being unsatisfactory. This suggest that these 4 students have met the learning outcome. All 7 of the remaining students who completed or left the program during the calendar years of 2011-2012 had two MAPIBS scores, and only one student received more than 2 unsatisfactory ratings on the combined scores. This indicates that a large majority met this learning outcome. || Good || We have successfully implemented the MAPIBS. An MAPIBS is filled out for every student in every graduate class, and each graduate student's advisor fills out an MAPIBS for that student every six months. An additional MAPIBS is completed by the advisor after the graduate student's successful project or thesis defense.

Although the data is incomplete for measuring this outcome, the data we do have suggests that students are doing well in meeting this learning outcome. The one student who did not meet this learning outcome left the program without completing the degree. While we hope that every student will be successful in our program, we acknowledge that some students will fail. We believe that success on this learning outcome by most of the students who completed or left the program during the calendar years of 2011-2012 indicates that the program is strong in this area. We will continue to monitor our students performance in this area. ||


 * 2013 Rubric for Assessing Program LOs**

The MA program will be assessed annually at the beginning of winter semester. Because the purpose of the assessment is to see if the learning outcomes have been met by the end of the program, it does not make sense to consider data from students who are still in the program. However, if data is limited to only students who either graduated or dropped out of the program the year before, the data points are insufficient to establish clear trends. Consequently, at each annual evaluation of the program, data will be considered for students who either graduated or dropped out over the past two years. While this means that the data from a particular student will be used in two consecutive evaluations of program learning outcomes, we nonetheless feel that data aggregated across two years will provide a fuller understanding of the success of our program than data from a single year.

In preparation to assess the program learning outcomes, each student who has graduated or dropped out of the program will be evaluated individually on each learning outcome to determine if that learning outcome was met by that student. The criteria for a learning outcome to be met by a student are given below. Once each student has been evaluated on each learning outcome, the number of students who have met each learning outcome will be tallied. If 70% of the students meet a particular learning outcome, then we will conclude that the program is satisfactory with respect to that learning outcome.


 * Criteria for Determining Whether a Student has Met a Particular Learning Outcome**

LO 1: Scholarship For each student, data from Sections I and III of the Master's Exam, from their Thesis/Project proposal defense and from their Thesis/Project final defense will be gathered. (If students were given a second chance on any of these measures, data will come from the last attempt.) Master's Exam Sections I and III: Students are considered satisfactory on this measure if the average of their scores on the four problems from Sections I and III is at least 70%. Thesis/Project proposal defense: Students are considered satisfactory on this measure if they passed (initially or eventually) their proposal defense. Thesis/Project final defense: Students are considered satisfactory on this measure if they passed (initially or eventually) their final defense. Students meet this learning outcome if their performance on any two of the three measures is satisfactory.

LO 2: Research For each student, data from Sections II and III of the Master's Exam, and from their Thesis/Project final defense will be gathered. (If students were given a second chance on either of these measures, data will come from the last attempt.) Master's Exam Sections II and III: Students are considered satisfactory on this measure if the average of their scores on the four problems from Sections II and III is at least 70%. Thesis/Project final defense: Students are considered satisfactory on this measure if they passed (initially or eventually) their final defense. Students completing a project meet this learning outcome if their performance on both measures is satisfactory. Students completing a thesis meet this learning outcome if their performance on the thesis defense is satisfactory.

LO 3: Mathematics For each student, data from Section IV of the Master's Exam will be gathered. (If students were given a second chance to take the Master's Exam, data will come from the last attempt.) Students meet this learning outcome if the average of their scores on the two problems from Section IV is at least 70%.

LO 4: Teaching For each student, teacher evaluation and student rating data from the last two courses taught will be gathered and the better of each of those measures will be used for analysis. For students to meet this learning outcome, they must receive a satisfactory rating on both measures.

STUDENT RATINGS: On the student ratings measure, there is an overall course rating, an overall instructor rating, and 23 sub-questions. Eighteen of the 23 sub-questions are rated on a 1-8 scale. If 9 or more of the sub-questions are rated below a 6 or if the overall instructor rating is below a 6, the student will receive an unsatisfactory rating on this measure. TEACHER EVALUATIONS: On the teacher evaluations measure, if the student receives a rating below a 3 in any of the 4 areas (classroom organization; instruction, discussion, and tasks; content; communication), they will receive an unsatisfactory rating on this measure.

LO 5: Professionalism For each student, two MA PIBS—the last MA PIBS from the project/thesis advisor and the MA PIBS from the last 3-credit-hour mathematics education course the student took—will be considered. For the student to be considered satisfactory on an item, the student must receive a rating of "meets" or "exceeds" expectations for that item on both MA PIBS. If a students is unsatisfactory on 3 or more items, then the student will be considered unsatisfactory in professionalism.

LO 6: Spiritual Stewardship For each student, items 1-3, 7, and 9 from two MA PIBS—the last MA PIBS from the project/thesis advisor and the MA PIBS from the last 3-credit-hour mathematics education course the student took—will be considered. For the student to be considered satisfactory on an item, the student must receive a rating of "meets" or "exceeds" expectations for that item on both MA PIBS. If a students is unsatisfactory on 2 or more items, then the student will be considered unsatisfactory in spiritual stewardship.