eLumen: The New Opportunity

In the near future, a few colleges will have become significantly better -- by present expectations, astonishingly better. Their faculty will collectively set academic and professional standards for students that are maintained across-the-curriculum. Their academic programs will know precisely the extent to which their students are demonstrating expected learning outcomes. Student advisors will precisely compare a student's actual achievements to-date to the expected achievements of that student's chosen program. Colleges will have a continuous picture of each student's readiness for the next learning opportunity. Institutional researchers will have a new wealth of data for investigating student learning. Students will receive a new college record showing what he or she has demonstrated they know and can do. As students realize that this record reflects their achievements to prospective employers, they begin to care about what it shows. In short, these colleges will distinguish themselves by the demonstrated quality of their academic outcomes and by the ability, as an organization, to attend to the personal, professional, and intellectual development of their students.

It may be difficult to believe that this many new organizational capabilities can appear, given the fact that higher education has changed so little in the last century. However this can and will happen, for the following reasons.

1. There can be an unexpectedly strong positive ratio of effort to results.

A college can evidence all of these accomplishments by doing well one single sustained activity -- standards-based evaluation of student achievement. This is a straightforward academic process: faculty work together to define expected student achievements and related standards of evaluation; subsequently individual instructors evaluate and document individual student work using these definitions and standards. (Likewise, those in student affairs work together to define non-academic achievements and related standards of evaluation; subsequently, in the course of their oversight of co-curricular activities, they evaluate and document individual student achievement using these definitions and standards.) More simply: within college-determined authorizations, everyone who chooses to participate evaluates and documents the student achievements that he or she has personally observed. Applying this approach in any chosen part of the college yields focused information on actual student achievement for that part. As this approach comes to be applied more widely throughout the college, the other results identified above grow out of each other in approximately the order in which they are described.

2. This academic approach to outcomes is being re-invigorated by direct technological support.

Up to now, colleges have been prevented from developing these powerful capabilities because they have lacked the deeper, more fundamental capability from which all of these other capabilities derive, namely, the organizational capability of documenting student achievement per student and per achievement. This is a complex set of data, and organizational procedures put into place before digital information systems could not deal with it well. Therefore, to the present, standards-based evaluation has been applied piecemeal. Each department has been left to do this for itself, or even worse, each professor/instructor has been left to do this for him- or herself. It has been manual, laborious, and has yielded none of the results identified above. Given these constraints, the broad use of standards-based evaluation throughout a college has been almost inconceivable and certainly impractical.

All of this has now changed with the introduction of new software that directly underwrites the process of standards-based evaluation, easily captures data per student and per achievement, generates new information on student achievement and appropriately distributes it to those who should know. For those teaching credit courses, the revised process of scoring students looks like improved grading. For those who are responsible for noncredit instruction, service learning, work-study, and the full array of extra-curricular activities, there is now an equally robust way to attend to student achievements in those settings as well. The software captures data linking a specific student to a specific achievement to a specific setting at a specific date and time. Out of this granular data, a college can define on-request reports that aggregate data by student, by achievement area, by setting, by date, or by any combination of these -- a wealth of information on actual student achievement that has never been so rich or so available. It is the careful re-introduction of this information back into the educational process itself that yields the new capabilities.

3. The introduction of this approach is feasible, because a college can start small and grow incrementally, on whatever timetable is comfortable.

Faculty or staff who are most interested can begin. The emphasis can be on activities that are already occurring: work that students are already completing in courses (and other college settings), critical judgments that faculty are already making (presently, in order to calculate a course grade), and decisions about educational goals for students that faculty committees are already making. Begin with one or more "core competencies" within programs, if you wish, or begin with a single college-wide expected student achievement. Begin with assignments or begin with a program's overall exit goals and work backward, if necessary. Begin with accreditation expectations or mission-driven goals that are distinctive to the college. Begin with knowledge or with skills or with perspectives.

Wherever a college begins, as faculty come to see how easily this is done, they will choose to do it more fully. As other faculty come to see that their own teaching is made easier by new information about the readiness of students who are coming into their courses, they will choose to join, at least if they contrast this with other more labor-intensive and less rewarding assessment activities. There is a network effect here; the more information (and the better information) on student achievement that is fed back into the educational process, academic term by term, the greater is the value for everyone, both faculty and students.

4. This approach will be easier, faster, and deliver far more results than other existing approaches to outcomes assessment.

Indeed, this is an academic alternative to "assessment" if, by that term, is meant all of the extra work that colleges have added to have data on student learning. It does ask faculty and staff of a college to decide what to attend to -- what should we expect of our students? by what standards should they be evaluated? -- and allows college committtees to apply those expectations and standards wherever in the curriculum it is most appropriate. The system-supported process then makes actual student achievement in those chosen areas visible, upon request. By doing this, this approach assists a faculty in making decisions, applying them, reflecting on the judgments made in that application, and revisiting decisions. Once reflected on, it will become apparent that this will make academic settings better. For organizations whose purpose is the intentional learning of its students, this is a fundamental capability. Once we begin using it, we will wonder how we ever functioned without it.

Of course, there are also reasons why, in any given college, this will not happen. Most colleges will be too busy or pre-occupied to notice the arrival of this opportunity. Many will perceive this narrowly as yet another software system for organizing their current assessment data and thus entirely misunderstand the opportunity. If a college does recognize the opportunity, it may still fail to do this well because it believes that if anyone does this, everyone must do it, carrying over a spirit of compliance that was needed in previous assessment initiatives in which faculty had no good reasons of their own to participate. It may also fail because faculty assume that all expectations and standards must be perfectly defined before beginning to apply any of them. On the other hand, it may fail because faculty simply choose not to work together to define consistent and coherent expectations and standards. (Certainly a century or more of academic practice without the ability of colleges, as organizations, of attending to student achievement has created a culture in which collaboration among faculty, and between academic and student affairs, is often missing.) Given all of these reasons, perhaps 9 out of 10 colleges will not choose this opportunity...

...but some will, and that, after all, is what I stated in my thesis: in the near future, a few colleges will have become significantly better. They will choose this because they will have reasons of their own to distinguish themselves from other colleges. These few will presently have the right combination of characteristics: a stable administration, no organizational crises to distract their attention or to consume their collective energy, the ability as an organization to take the long view, and above all, a small group of faculty with a willingness to learn by doing, starting small and growing the areas of the college curriculum and co-curriculum that use standards-based evaluation of student achievement. These colleges will become astonishingly better.




The Challenge of Organizational Knowledge The New Opportunity Faculty-Driven Assessment Authentic Assessment Making Achievement Visible Our Mission and Vision Who We Are Seminars and Events Our Clients eLumen Achievement: A Summary of the Software How eLumen Achievement Models the Essential Process Consulting Services The Collaborative		Contact Us \| Home
		The New Opportunity In the near future, a few colleges will have become significantly better -- by present expectations, astonishingly better. Their faculty will collectively set academic and professional standards for students that are maintained across-the-curriculum. Their academic programs will know precisely the extent to which their students are demonstrating expected learning outcomes. Student advisors will precisely compare a student's actual achievements to-date to the expected achievements of that student's chosen program. Colleges will have a continuous picture of each student's readiness for the next learning opportunity. Institutional researchers will have a new wealth of data for investigating student learning. Students will receive a new college record showing what he or she has demonstrated they know and can do. As students realize that this record reflects their achievements to prospective employers, they begin to care about what it shows. In short, these colleges will distinguish themselves by the demonstrated quality of their academic outcomes and by the ability, as an organization, to attend to the personal, professional, and intellectual development of their students. It may be difficult to believe that this many new organizational capabilities can appear, given the fact that higher education has changed so little in the last century. However this can and will happen, for the following reasons. 1. There can be an unexpectedly strong positive ratio of effort to results. A college can evidence all of these accomplishments by doing well one single sustained activity -- standards-based evaluation of student achievement. This is a straightforward academic process: faculty work together to define expected student achievements and related standards of evaluation; subsequently individual instructors evaluate and document individual student work using these definitions and standards. (Likewise, those in student affairs work together to define non-academic achievements and related standards of evaluation; subsequently, in the course of their oversight of co-curricular activities, they evaluate and document individual student achievement using these definitions and standards.) More simply: within college-determined authorizations, everyone who chooses to participate evaluates and documents the student achievements that he or she has personally observed. Applying this approach in any chosen part of the college yields focused information on actual student achievement for that part. As this approach comes to be applied more widely throughout the college, the other results identified above grow out of each other in approximately the order in which they are described. 2. This academic approach to outcomes is being re-invigorated by direct technological support. Up to now, colleges have been prevented from developing these powerful capabilities because they have lacked the deeper, more fundamental capability from which all of these other capabilities derive, namely, the organizational capability of documenting student achievement per student and per achievement. This is a complex set of data, and organizational procedures put into place before digital information systems could not deal with it well. Therefore, to the present, standards-based evaluation has been applied piecemeal. Each department has been left to do this for itself, or even worse, each professor/instructor has been left to do this for him- or herself. It has been manual, laborious, and has yielded none of the results identified above. Given these constraints, the broad use of standards-based evaluation throughout a college has been almost inconceivable and certainly impractical. All of this has now changed with the introduction of new software that directly underwrites the process of standards-based evaluation, easily captures data per student and per achievement, generates new information on student achievement and appropriately distributes it to those who should know. For those teaching credit courses, the revised process of scoring students looks like improved grading. For those who are responsible for noncredit instruction, service learning, work-study, and the full array of extra-curricular activities, there is now an equally robust way to attend to student achievements in those settings as well. The software captures data linking a specific student to a specific achievement to a specific setting at a specific date and time. Out of this granular data, a college can define on-request reports that aggregate data by student, by achievement area, by setting, by date, or by any combination of these -- a wealth of information on actual student achievement that has never been so rich or so available. It is the careful re-introduction of this information back into the educational process itself that yields the new capabilities. 3. The introduction of this approach is feasible, because a college can start small and grow incrementally, on whatever timetable is comfortable. Faculty or staff who are most interested can begin. The emphasis can be on activities that are already occurring: work that students are already completing in courses (and other college settings), critical judgments that faculty are already making (presently, in order to calculate a course grade), and decisions about educational goals for students that faculty committees are already making. Begin with one or more "core competencies" within programs, if you wish, or begin with a single college-wide expected student achievement. Begin with assignments or begin with a program's overall exit goals and work backward, if necessary. Begin with accreditation expectations or mission-driven goals that are distinctive to the college. Begin with knowledge or with skills or with perspectives. Wherever a college begins, as faculty come to see how easily this is done, they will choose to do it more fully. As other faculty come to see that their own teaching is made easier by new information about the readiness of students who are coming into their courses, they will choose to join, at least if they contrast this with other more labor-intensive and less rewarding assessment activities. There is a network effect here; the more information (and the better information) on student achievement that is fed back into the educational process, academic term by term, the greater is the value for everyone, both faculty and students. 4. This approach will be easier, faster, and deliver far more results than other existing approaches to outcomes assessment. Indeed, this is an academic alternative to "assessment" if, by that term, is meant all of the extra work that colleges have added to have data on student learning. It does ask faculty and staff of a college to decide what to attend to -- what should we expect of our students? by what standards should they be evaluated? -- and allows college committtees to apply those expectations and standards wherever in the curriculum it is most appropriate. The system-supported process then makes actual student achievement in those chosen areas visible, upon request. By doing this, this approach assists a faculty in making decisions, applying them, reflecting on the judgments made in that application, and revisiting decisions. Once reflected on, it will become apparent that this will make academic settings better. For organizations whose purpose is the intentional learning of its students, this is a fundamental capability. Once we begin using it, we will wonder how we ever functioned without it. Of course, there are also reasons why, in any given college, this will not happen. Most colleges will be too busy or pre-occupied to notice the arrival of this opportunity. Many will perceive this narrowly as yet another software system for organizing their current assessment data and thus entirely misunderstand the opportunity. If a college does recognize the opportunity, it may still fail to do this well because it believes that if anyone does this, everyone must do it, carrying over a spirit of compliance that was needed in previous assessment initiatives in which faculty had no good reasons of their own to participate. It may also fail because faculty assume that all expectations and standards must be perfectly defined before beginning to apply any of them. On the other hand, it may fail because faculty simply choose not to work together to define consistent and coherent expectations and standards. (Certainly a century or more of academic practice without the ability of colleges, as organizations, of attending to student achievement has created a culture in which collaboration among faculty, and between academic and student affairs, is often missing.) Given all of these reasons, perhaps 9 out of 10 colleges will not choose this opportunity... ...but some will, and that, after all, is what I stated in my thesis: in the near future, a few colleges will have become significantly better. They will choose this because they will have reasons of their own to distinguish themselves from other colleges. These few will presently have the right combination of characteristics: a stable administration, no organizational crises to distract their attention or to consume their collective energy, the ability as an organization to take the long view, and above all, a small group of faculty with a willingness to learn by doing, starting small and growing the areas of the college curriculum and co-curriculum that use standards-based evaluation of student achievement. These colleges will become astonishingly better. David A, Shupe Director, Evaluation of Student Achievement eLumen Collaborative top \| Authentic Assessment \| The eLumen Story \| Assessment Tools \| Contact Us \| Home © Copyright 2004, eLumen. All Right Reserved. 1776 Scheffer Avenue • Saint Paul, MN 55116 1-800-630-2149 • info@elumen.info