Quicklinks
myMILLIKIN |
Search
Millikin Home
Big Blue RSS
Millikin University - Decatur, IL
Assessment of Student Learning  
Millikin University > Assessment of Student Learning
Higher Learning Commission Focus Visit on Assessment of Student Learning
Executive Summary - March 9, 2010

In Brief

 

Millikin University uses student learning assessment practices to develop curriculum, allocate funds, and meet student needs.  The university has progressed from creating assessment plans to thoroughly collecting, evaluating, and utilizing valuable data, which demonstrates our university-wide commitment to assessment.

 

Section 1. Progress Since the 2007 Comprehensive Visit

 

Over the past six years, institution-wide assessment of student learning has gradually become more systematic.  In 2004, two committees were charged with developing ongoing systems of assessing student learning in the Millikin Program of Student Learning (MPSL): the Assessment Steering Committee, to focus on the majors; and the University Studies Advisory Committee, to focus on general education programs.  In 2005, the Accreditation Self-Study Committeedeveloped specific assessment plans for data collection and annual evaluation.  University-wide student learning outcome goals were formally codified and ongoing systems for annual assessment were implemented. 

 

By the Spring 2007 Higher Learning Committee (HLC) comprehensive accreditation visit, Millikin programs had developed assessment plans. At that time, the University could only generate a baseline of data. The HLC visit team noted “significant inconsistency in the implementation of assessment plans, a paucity of data in several departments, and little evidence of comprehensive feedback in using assessment data,” and recommended, by the March 2010 Focus Visit, that the university demonstrate it can:

 

  1.  sustain the assessment effort over a multi-year period of time;
  2. systematically and comprehensively collect and analyze data about student learning in all educational programs;
  3. use the analysis to draw conclusions about improving curriculum and pedagogy and to inform decisions about budgets and strategic priorities; and
  4. evaluate, modify, and continue to improve the student learning assessment process.

Section 2. Assessment Trend Review: Methods & Analysis

 

The university has responded to the challenge of establishing a culture of assessment.  We have addressed HLC’s recommendations, implemented assessment plans in every major and program, and developed a system to evaluate campus-wide assessment efforts. A Focus Visit Leadership Team (FVLT), including academic deans, program leaders, and faculty members, coordinated efforts to: (1) evaluate annual assessment reports; (2) evaluate assessment process trends over the past three years; (3) offer feedback on student learning assessment processes; (4) participate in the development of a report that evaluates assessment efforts both in individual majors and programs, and in the university as a whole; and (5) use assessment data to inform decisions for program improvement.     

The FVLT initiated a deliberate trend analysis study to review the quality of the university’s assessment of student learning.  Based on the four recommendations from HLC, the FVLT developed a rubric for evaluating assessment efforts based on the following data points: (1) annual assessment reports from each major and program; (2) three-year assessment trends reports from each major and program; (3) reports from Council on Curriculum on institutional trends in curricular improvements.  An analysis of this data demonstrates how the institution has effectively established systematic, ongoing assessment processes.

 

Section 3. Evaluation of Millikin University’s Assessment Processes

Goal 1: Sustain the assessment effort over a multi-year period of time.

 

2007 Overall University Rating: RED (55% RED)

2009 Overall University Rating: GREEN (70% GREEN)

 

TABLE 7: Comparison of 2007 and 2009 Programs on Goal 1

Rating

2007 Programs

2009 Programs

2009 Examples

GREEN

1/38 (3%)

30/40 (75%)

Music

YELLOW

16/38 (42%)

6/40 (15%)

Fitness & Sport

RED

21/38 (55%)

4/40 (10%)

International & Global Studies Major

 

Annual assessment reports and three-year assessment trends reports demonstrate:

 

 In 2007, 1 of 38 programs sustained assessment efforts for three or more years.

 In 2009, 30 of 40 programs sustained assessment efforts for three or more years.

 

This overall change from RED in 2007 to GREEN in 2009 indicates a remarkable achievement in Goal 1.  In 2007, some programs (42%) had sustained the effort for two years, but most were building their assessment plans and first reports.  By 2009, successful programs (90%) have been developed and are developing a pattern of assessment sustained for three years or more.  The four programs struggling are either new programs or have experienced significant leadership transitions.

Goal 2: Systematically and comprehensively collect and analyze data about student learning in all educational programs. 

 

2007 Overall University Rating: RED (52% RED)

2009 Overall University Rating: YELLOW (48% GREEN)

 

TABLE 8: Comparison of 2007 and 2009 Programs on Goal 2

Rating

2007 Programs

2009 Programs

2009 Examples

GREEN

12/38 (32%)

19/40 (48%)

IN150 Critical Writing Reading & Research

YELLOW

7/38 (18%)

16/40 (40%)

Natural Science with a Lab

RED

19/38 (50%)

5/40 (12%)

Fine Arts Requirement

 

Annual assessment reports and three-year assessment trends reports demonstrate:

 

In 2007, 19 programs systematically and/or comprehensively collected and analyzed data.

In 2009, 35 programs systematically and/or comprehensively collected and analyzed data.

 

This overall change from RED in 2007 to YELLOW in 2009 indicates stable progress toward Goal 2.  In 2007, only half of the programs had initiated viable assessment processes.  By 2009, many successful programs (88%) are both systematic and comprehensive in their data collection and analysis processes. Some programs are still working to become either more comprehensive or more systematic.  The few programs still struggling lack insufficient data to draw conclusions or make informed decisions.

Goal 3: Use the analysis to draw conclusions about improving curriculum and pedagogy and to inform decisions about budgets and strategic priorities. 

 

2007 Overall University Rating: RED (53% RED)

2009 Overall University Rating: YELLOW (58% GREEN)

 

TABLE 9: Comparison of 2007 and 2009 Programs on Goal 3

Rating

2007 Programs

2009 Programs

2009 Examples

GREEN

8/38 (21%)

25/40 (63%)

Tabor Programs

YELLOW

10/38 (26%)

9/40 (22%)

Communication

RED

20/38 (53%)

6/40 (15%)

Natural Sciences with a Lab

 

Annual assessment reports, three-year Assessment Trend Reports, and Council on Curriculum Summary Analysis of Curriculum Changes Fall 2006-Fall 2009 demonstrate

 

In 2007, 18 programs variably used assessment to improve curriculum, initiate pedagogies, or institute budget priorities.

In 2009, 34 programs variably used assessment to improve curriculum, initiate pedagogies, or institute budget priorities.

 

This overall change from RED in 2007 to YELLOW in 2009 indicates noticeable improvement in Goal 3. In 2007, only 53% of programs with assessment processes were using them to makerecommendations for changes in curriculum, pedagogy, or budget priorities.  By 2009, the majority (85%) of programs are recommending and often implementing such improvements.  Programs struggling to make recommendations lack enough data and/or analysis to discuss or make assessment-based improvements.

 

Goal 4: Evaluate, modify, and continue to improve the student learning assessment process. 

 

2007 Overall University Rating: RED (58% RED)

2009 Overall University Rating: YELLOW (58% GREEN)

 

TABLE 11: Comparison of 2007 and 2009 Programs on Goal 4

Rating

2007 Programs

2009 Programs

2009 Examples

GREEN

3/38 (8%)

26/40 (65%)

Education

YELLOW

13/38 (34%)

7/40 (17%)

IN250 United States Studies

 RED

22/38 (58%)

7/40 (17%)

Political Science

 

Annual assessment reports and three-year assessment trends reports clearly demonstrate:

 

In 2007, 16 programs were evaluating, modifying, and improving student learning assessment processes.

In 2009, 33 programs are evaluating, modifying, and improving student learning assessment processes.

 

This overall change from RED in 2007 to YELLOW in 2009 indicates excellent progress in Goal 4.  In 2007, 58% of programs were not yet fully evaluating their assessment processes.  By 2009, a majority (82%) of programs have successfully implemented deliberate, formal forums for conversations about improving and changing the assessment process, as needed. 17% of programs struggling are working on developing such forums for disseminating information and facilitating discussion about assessment processes.

Section 4.  Traditions of Data Collection & Data Sharing

 

Over the last six years, Millikin University has become successful in establishing strong traditions of student learning assessment.  We are consistent in our implementation of annual assessment plans for all academic programs. We have developed methods to collect rich and varied data on student learning, with feedback mechanisms in place to use assessment data to inform decisions and to continue to improve assessment processes. 

 

In the October 2009 National Institute for Learning Outcomes Assessment report, “More Than You Think, Less Than We Need: Learning Outcomes Assessment in American Higher Education,” George Kuh and Stanley Ikenberry remark, “At the program level the most popular approaches to assessing learning outcomes were student portfolios, measures of specialized knowledge and other performance assessments, and rubrics” (p. 3, Abridged Version).

 

Our academic programs systematically collect data regarding student learning in a variety of ways.  A review of Millikin University assessment trend reports and annual assessment reports shows that by 2009, 90% of our programs use rubrics to assess student learning through sample artifacts of student work. Over 50% evaluate student performances, such as juried recitals, art exhibitions, academic competitions such as poster exhibits, and oral presentations, and 50% utilize national license tests and exams.

 

Millikin takes pride in its emphasis on a performance-learning pedagogy. Therefore, it makes sense that almost all of our programs would assess student learning outcomes by careful examination of student performances including written artifacts, public performances, and scores on professional certification exams. This emphasis on culminating performance is a university-wide expectation in all programs.

 

Section 5. Recommendations for Continued Development

 

Through the trend reports, we now have a better understanding of the quality of our assessment processes in all programs, and we are better positioned to make recommendations for continuing university-wide assessment efforts and building a stronger culture of assessment. 

 

To ensure ongoing quality assessment of student learning, the university will:

  • continue to support and review annual reports, and implement a five-year program review process including multiyear assessment trend reports;
  • continue Council on Curriculum review of assessment-based rationales for curriculum improvements;
  • sponsor and promote forums for discussing assessment results and processes; and
  • review quality of student learning and assessment reports through the University Assessment Committee recommendations.

 

Program leaders and administrators will continue to demonstrate commitment to the traditions of sharing and discussing assessment results across the entire university.  Council on Curriculum will continue to take a leadership role in overseeing assessment-based changes in the curriculum. The University Assessment Committee will extend the culture of assessment developed for academic programs to all university offices and departments. Millikin University will continue to embrace and build a culture of assessment that is pervasive throughout all offices and branches of the institution.

 
Millikin University - Decatur, IL
 
Millikin University - Decatur, IL
Guest