National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

EIA Designation: Why Campus-Level Assessment of Student Learning Outcomes?

The ability of a campus to clearly and convincingly communicate the learning outcomes of all their graduates, regardless of program of study, is paramount to the success of our students, institutions, and larger national economic and competitive priorities. Policymakers and external stakeholders are increasingly questioning the value of higher education experiences as a whole,1 focusing on labor market outcomes to hold certain types of programs or majors up as preferred.2 Institutions and the higher education industry as a whole have struggled to push back on these claims, citing the complexity of evaluating student learning across varied and disparate programs in easily comparable ways.

Despite these challenges, many campuses are successfully designing and implementing campus-wide assessment systems that provide evidence of the learning of all students. These systems are horizontally and vertically integrated to encompass learning both in and outside of the classroom, and are validated by participation and evaluation of external stakeholders, including alumni, employers, and schools their students subsequently attend for additional study. By incorporating all areas of campus, not just the academic experiences that occur in the classroom, institutions are able to confidently assert the competency of their students in areas of leadership and teamwork—essential outcomes for employers in today’s diverse workplaces.3,4 Building intentionally integrated, layered systems that rest on the foundational work of faculty in the classroom, campuses are able to provide deep and rich evidence of students’ knowledge, skills, and abilities.

Why do we need a national designation of excellence?

One of the main goals of the EIA program is to create a national recognition for campuses that are successfully integrating assessment practices across campus to provide evidence of student learning outcomes that are representative of all students who attend their institution. There is currently no such recognition, which hinders our efforts in at least three areas.

First, if we can’t identify more than a handful of institutions that have become known for their assessment work, we limit the models available for campuses to consider. There is not one right way to implement a broad and deep assessment plan on all campuses, but rather many right ways. Not only does this limit our examples, it limits our ability to celebrate the work being done on campuses that contributes to student success.

Assessment done well is integrated throughout the work of many campus faculty and staff; by offering a national recognition of those institutions doing exemplary work, the EIA designations create an opportunity to reward and celebrate those efforts.

Second, because we can’t universally identify campuses who are engaged in good assessment practice, we can’t connect individual institutions with other institutions who may be similarly situated to help foster learning and sharing of what works and what doesn’t. By explicitly recognizing that there are many models for effective assessment of student learning, the EIA designations serve as both celebration of the work these campuses have accomplished as well as guideposts for other campuses looking to improve their own evaluation of student learning. The EIA designations are open to all regionally accredited institutions and the goal is to identify a plethora of examples from across sectors and levels to share broadly. By actively seeking to identify those institutions who are doing this work, we will create a larger network of examples for other campuses to explore.

Third, the lack of a common, national recognition program for campus assessment limits our ability to engage with external stakeholders and hold up concrete examples of the good assessment work our campuses are engaged in. We are continually questioned about the value of higher education for our students, but lack a nationally recognized and respected means to rebut the claims that we are disorganized and muddled. While still respecting the diversity of what good assessment looks like in practice, the EIA designations provide a signal and a standard for external audiences to look to.

We already do program-level assessment; what does campus-level assessment add?

Campus-level assessment is often thought of as an accountability or accreditation concern that is somewhat removed from the work of faculty teaching in specific programs or classes.5 Faculty may be disinvested from the administration of a campus-wide assessment instrument, at best seeing it as a benign requirement to appease external cries for accountability. The EIA program, however, understands that campus-level assessment builds from a foundation of faculty assessment of student learning, as an integrated component designed to serve as a “tip of the iceberg” indicator for the depth and breadth of student learning happening on our campuses (Figure 1).
 

 

Figure 1: Pyramid of Integrated Campus-wide Assessment

Campus-level assessment does not necessarily consist of using an identical or standardized assessment instrument administered to all students, but represents a broader and deeper assessment plan that is integrated across departments and programs. An ideal campus-wide assessment plan would consist of aligned outcomes at the student-, course-, program-/major-, and degree-levels. What that looks like for any given campus may be varied and diverse, and recognizing the multiple right paths to demonstrating student learning outcomes is the goal of the EIA designations.

In this view, assessment activities occur at multiple levels where each level is related to and either builds from or supports the levels below and above it, creating a scaffold of evidence across all students. The nature and setting of assessment activities at each level may or may not be different depending on the program and institution. For example, assessment of campus-level learning outcomes may occur within the context and setting of an individual capstone course or it may occur in a separate, proctored exam.

The results of the assessment at each level, however, need to be comparable across all students so results can be interpreted and used to identify the need for, and guide the implementation of, program and curricular improvements. Just as all students in an Accounting program need to pass a licensure exam to ensure that they have all learned the knowledge, skills, and abilities necessary to perform as a Certified Public Accountant, all students at an institution should be assessed with approaches that provide results comparable to each other to ensure all students have learned the core knowledge, skills, and abilities the institution has identified as necessary to earn a degree.

Ideally, scaffolding of learning outcomes occurs across courses and programs such that the assessment work from an individual course can link to and inform the assessment work of a program. This scaffolding allows for fewer assessment activities to occur at higher levels of the pyramid than at lower levels—the evidence built from beneath serves to support the evidence provided at the top. This explains, in part, why a single representative sample of campus-level assessment outcomes is one way to reliably represent the learning of all students on campus.

How does the National Institute for Learning Outcomes Assessment’s Transparency Framework fit in?

Even those campuses that are engaged in intentionally aligned assessment of student learning struggle to tell their stories to the variety of stakeholders they are accountable to. While most campuses nowmake a practice of publicly stating what their learning outcomes are, they generally share assessment findings internally, and frequently only to faculty or assessment committees. Rarely are assessment plans or rationales shared with students or alumni, much less with external stakeholders such as employers, institutions who accept students for continued study, state legislators, or the public.6 Put another way, campuses talk about assessment to themselves when they talk about it at all.

The National Institute for Learning Outcomes Assessment (NILOA) created the TransparencyFramework “to help institutions evaluate the extent to which they are making evidence of student accomplishment readily accessible and potentially useful and meaningful to various audiences.”7 Based on a national review of campus assessment websites, the Transparency Framework consists of six components centered around a set of underlying principles common to all components (Figure 2). Guidance on the NILOA website includes key questions institutions can ask to help ensure they are making evidence of their assessment and student learning readily accessible and useful to both internal and external audiences.

Figure 2: The National Institute for Learning Outcomes Assessment (NILOA) Transparency Framework

Conclusion

As increased attention has been paid to campus-level assessment outcomes as an indicator for campus accountability, pressure on campuses to simply report results to meet external demands has intensified. In some cases, this pressure has led to decoupling the campus-level assessment activities from those that support and give credence to their results. Even campuses who are engaged in strong student learning outcomes assessment struggle to tell their stories to stakeholders both on and off campus. The purpose of the EIA designations is to recognize the work of those campuses that are engaging in the full breadth and depth of vertically and horizontally integrated student learning outcomes assessment, ensuring that all systems are linked and cross-validated. These campuses deserve recognition of their accomplishments and by identifying them, the EIA designations also highlight them as exemplars for other campuses to explore.


1See for example: USA Today College (December 10, 2015). “Is college worth it? Goldman Sachs says not so much”; The New York Times TheUpshot (May 27, 2014). “Is College Worth It? Clearly, New Data Say”; and Money (October 5, 2015). “Why College is Still Worth It Even Though It Costs Too Much”.

2Carnevale, A.P., Cheah, B., & Hanson, A.R. (2015). The Economic Value of College Majors. Washington, DC: Georgetown University, Center on Education and the Workforce.

3Hart Research Associates. (2015). Falling Short? College Learning and Career Success. Washington, DC: Association of American Colleges & Universities (AAC&U).

4Deming, D.J. (2015). The Growing Importance of Social Skills in the Labor Market. Cambridge, MA: Harvard University and NBER.

5Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. (2014). Knowing What Students Know and Can Do: The Current State of Student Learning Outcomes Assessment in US Colleges and Universities. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

6Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. (2014). Knowing What Students Know and Can Do: The Current State of Student Learning Outcomes Assessment in US Colleges and Universities. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

7National Institute for Learning Outcomes Assessment. (2011). Transparency Framework. Urbana, IL: University Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Retrieved from: http://www.learningoutcomesassessment.org/TransparencyFramework.htm

The Excellence in Assessment Designation is co-sponsored by the Voluntary System of Accountability (VSA), the National Institute for Learning Outcomes Assessment (NILOA), and the Association of American Colleges and Universities (AAC&U). VSA College Portrait is a joint initiative of the American Association of State Colleges and Universities (AASCU) and the Association of Public & Land-Grant Universities (APLU).

 

Carol Geary Schneider
President, Association of American Colleges & Universities (AAC&U)

 

George Kuh
Director, National Institute for Learning Outcomes Assessment (NILOA)