The Challenge of Organizational Knowledge

The New Opportunity

Faculty-Driven Assessment

Authentic Assessment

Making Achievement Visible

Our Mission and Vision

Who We Are

Seminars and Events

Our Clients

eLumen Achievement: A Summary of the Software

How eLumen Achievement Models the Essential Process

Special Projects & Services

The Collaborative

Contact Us | Home

The situation in which we find ourselves

Among the many challenges facing higher education, the newest and perhaps the most difficult challenge is making our educational results visible. The growing expectation is that we answer many different versions of the question “how well prepared are our students for the world they are entering?” with empirical data – data with a clarity and a specificity to which we are unaccustomed. That data would be used both to link expectations and resources to actual results and to make changes that result in improved student learning. This summarizes what is now being expected by regional accreditation agencies, by professional accreditation associations, and by grant-making agencies. It speaks to the intent of actions taken, in many states, by governing boards and legislatures. It generalizes the question asked, explicitly or implicitly, by parents about the college or university which their son or daughter is attending.

The problem is that we are being asked to exercise an organizational capability that post-secondary institutions have never had. We have embedded information systems that were never intended to accomplish these new ends. They were developed to support an academic practice in which it was sufficient to measure a student’s educational progress by accumulated credits and to measure the quality of a student’s achievement by faculty assignment of a course grade. Credits and grades, though, are now seen as lacking the needed clarity and specificity.

This places us in a difficult position. It is learning -- the personal, professional, and intellectual development of individuals – that is the primary process for the creation of value in colleges and universities, yet this is the one process about which colleges and universities, as organizations, have little or no data. Most learning is invisible, because our institutions do not have the means to capture or retain the crucial information: what a student has learned – that is, has demonstrated that he or she knows and can do -- while being one of our students. Yes, there are recorded grades, but what did the student learn in order to earn any individual grade? The organization cannot say because it does not know. How does what the student learned in course 1 differ from what he or she learned in course 2? The organization cannot say because it does not know. How ready, really, is a student, after completing one course, for the next follow-on course? The organization cannot say, because it does not know. What, then, are the student learning outcomes for that student in a given term, or, indeed, in a given degree program? The organization cannot say, because it does not know.

This lack of organizational knowledge, this structural inability for post-secondary institutions to systematically recognize student achievements as they occur, has multiple negative effects. It inhibits student learning, which, in turn, means that students are not as well prepared for the world they are entering as we (or they) may wish they were. It obscures variation among students in the classroom, which, in turn, makes teaching more difficult. It forces us, as concern for “results” increases, to over-control the curriculum, which, in turn, leads to curricular monotony and standardization. Finally, it presents a strong impression to outsiders that we are perhaps not preparing students well, which, in turn, contributes to difficulties in legislative budget and accountability decisions.

For academics, this perspective is unfamiliar and uncomfortable, We certainly want the academy to be a better place, but “better” would have a meaning drawn from a perspective less concerned with demonstrating results than with having an overall environment that has certain characteristics. These would include creating a distinctive institution, recognizing individual strengths among faculty, maintaining academic standards, and refreshing a dynamic curriculum. Given a finite amount of time beyond one’s academic load, most academics would choose to put their efforts working toward ends like these, not specifying results.

Thus, as presently construed, the new expectations for data are at odds with long-standing academic practices and prevailing academic culture. Our systems cannot accommodate these expectations, and most academics dislike them. Colleges have come to expect that the task of producing data about student achievement is onerous, disruptive of learning, and distractive. Unsurprisingly, there is no positive “vision of the future” to motivate those working on assessing and documenting outcomes. Only two futures are imagined: a continuation of the present difficulties or the hope that this concern for specified results will somehow disappear.

Envisioning a far better future

Let us envision a far more positive future in which faculty and administrators can work to make their colleges and universities better academically and still have continuously produced and appropriately aggregated data on student achievement. This would mean to envision both a thoroughly academic accountability and a thoroughly accountable academy. The term “thoroughly” precludes any carefully crafted political co-existence. Rather, the future to be imagined is one in which a college or university can more completely develop and demonstrate its distinctiveness, more fully reward individual strengths in both faculty and students, more rigorously maintain academic standards across the curriculum, and offer a creative and more dynamic curriculum -- and do all of these not in spite of but precisely because of systematically and continuously produced data on student achievement.

The academic practices and structures of this future academic setting are much like they are today with the following exceptions:

1. Administration has made available a new Web-based information system to students, faculty, and all college staff who advise or supervise students. The system has an elegant and empty structure for specifying and calibrating student achievements (or learning outcomes) and the academic standards by which these learning outcomes are evaluated. For each specific outcome, the system keeps track of its status: e.g., anticipated, required by program [both indicated by faculty], intended [indicated by student], in-progress [system-generated], or completed [which can only be indicated by faculty or authorized staff]. Indicating a completed status creates one of many records with an identical structure – linking a specific student to a specific achievement at a specific time in a specific curricular (or other college) setting.
   
2. Faculty work together to pre-define which achievements will be evaluated and which shared standards will be used for that evaluation. The generic structure makes it easy for them to identify a specific achievement as being of a certain type, area, and level, all of which are determined by faculty. Faculty in a specific academic program identify what they would like (or require) students to know and do in that program. College-wide faculty committees identify what they would like students to know and do, regardless of their enrolled academic program. While only some faculty take coursework time to instruct students in these college-wide expected outcomes, the evaluation standards are generic enough and clear enough so that every faculty member using that outcome in his or her course evaluates students by the same standards. Overall, faculty carefully choose learning opportunities for a course and then they “teach to the course.”
   
3. Each student understands that the college is attending to his or her personal, professional, and intellectual development. Their tasks, then, are to learn and to demonstrate that learning. Students have access to a description of the full range of achievements that the faculty stand ready to evaluate, as well as the accompanying standards. From the first day at the college, they understand that the value of their degree will be the record of the actual learning outcomes that the college acknowledges for them. In the end, that record is a creative combination of the strengths of the college, the faculty, and the individual student.
   
4. At the point that students have demonstrated an achievement, faculty evaluate their work and create the record that links each student to an achievement, including how well that student did (e.g., 3 on a 4-point rubric or a score of 85% on a test). Faculty and students both attend to the learning that actually has occurred, whether that achievement was expected or not. It is recognized that much of this actual learning is not reflected in the ostensible subject of the course (or internship, extracurricular activity, service learning, study abroad or other supervised learning setting). Students attend to this, because it is in their interest to have their learning achievements acknowledged. In the case of an unexpected achievement, faculty ask for evidence from the student and use their critical judgment to place that achievement in its proper place.
   
5. The database provided by administration now contains records for expected student achievement, standards, and actual student achievement. Standardized and customized searches permit a wide range of data to be instantly available.

A data search for a specific student provides the fundamental capacity to document and track each student’s educational progress, across-the-curriculum, by demonstrated achievements. The student’s on-line view of this -- available at any time – compares required, intended, in-progress, and actual achievements. Another version, showing only actual achievements, is available for students to send to outside viewers.

A data search for a specific achievement provides the capacity to review how often a given achievement has been completed in the college and where this has occurred in the curriculum.

A data search for a specific time frame provides a running chronological record of recorded achievements.

A data search for any specific curricular setting (any combination of program, course, section, term) provides the capacity to review the expected, in-progress and actual achievements.

An Accountable Academy

It may be noted that the changes in academic practice between this envisioned future and the present are relatively small: faculty engage in ongoing discourse to assure consistency in academic standards, and they enter data per student per achievement. From this small difference, the benefits of what we are imagining are disproportionately large. The college or university, as organization, focuses on each student and his or her demonstrated achievements, allows him or her to find their strengths among the full range of types of achievement, and follows the student’s development across the curriculum. From a single efficient data-gathering process – each faculty member evaluating the work of his or her own students – it becomes possible to have a wide range of aggregate data on actual student learning. An academic program now has constructive feedback comparing its expected and actual outcomes – the “enacted curriculum.” Additionally, the college can ascertain whether a student is ready for the next learning opportunity or has completed the program outcomes. An instructor can know, as a course begins, what level of achievement the students are bringing to the course. Students have another college record to show to prospective employers and, realizing that this record will be viewed as an accurate record of their preparedness for what they choose to do after college, they begin to take ownership of what that record shows. In sum, accountability is well allocated across the college (including students), and there is a wealth of data about actual educational results.

An Academic Accountability

Nevertheless, the way in which this data is constructed actually makes it more possible to address improvements that academics would readily choose.

A college now has a new way to demonstrate its distinctiveness: its “catalog” showing the faculty’s collective choice of anticipated student achievements (and the places in the curriculum for learning opportunities in which these achievements can de demonstrated). No two colleges will be alike, and each will have clear emphases that set it apart. These emphases are not just rhetoric; translated into anticipated learning outcomes, they are integrated into the day-to-day teaching and learning of the institution. The college can also be proud of its record of actual student achievements. While it can continue to point to graduates who excel after college, it now has direct ways to point to excellence in achievement among current students.

This distinctive and rich set of anticipated student achievements is, in large part, the result of the strengths of individual faculty who have chosen to be at that college. A faculty member’s special emphases in his or her primary field become crucial to enhancing the achievement for some students. Additionally, while many of these faculty strengths fall within a primary field, many others (especially those that concern performance, perspective, or capability) do not. Avocations become relevant when they are done with attention to excellence, and faculty and staff can become mentors to (and evaluators of) students in areas outside their primary field.

Academic standards become the basis of this approach. Students soon learn that, once having learned to meet a certain standard of achievement for, say, writing or public speaking or research methods, they will be held to that standard everywhere in the curriculum. This reinforcement of standards is a powerful lesson. So is the faculty expectation that a student use what has already been demonstrated as the foundation for subsequent work. The challenge is to reach for the next level of achievement in that area.

Finally, when it is achievement data, not course enrollment data, that shows readiness for the next learning opportunity, the curriculum can become more dynamic and flexible. Courses, as actually taught by faculty, should be creative combinations of learning opportunities, and this allows that to be so. Faculty can change courses primarily defined by subject matter by introducing new course activities that emphasize different desired student capabilities. Faculty can change courses primarily defined by capabilities by choosing topics that are responsive to current events.

All four of these academic interests become more possible precisely because the educational progress of individual students is “measured” by their demonstrated achievements.

Summary of a presentation at the annual conference of the North Central Association, Higher Learning Commission. Chicago, March 2002

David A. Shupe
Minnesota State Colleges and Universities