A Framework for Measuring the Sustainability of Academic Programs in the Technical Fields: Initial Validity Study Findings

Abstract

A Framework for Measuring the Sustainability of Academic Programs in the Technical Fields: Initial Validity Study FindingsThe term sustainability aims to describe the capacity of meeting the needs of the present withoutcompromising the future. Sustainability is a key characteristic of continuous improvement, acriterion required by numerous outcomes-based quality assurance and accrediting bodies, such asABET. It is well known that “closing the loop” of assessment and evaluation processes remains asignificant challenge for academic programs worldwide. In efforts to address this issue, the chairof a computing program and an accreditation and assessment specialist partnered to develop theSustainability of Technical Education (SoTE) framework.The SoTE framework consists of criteria, measures, indicators, and a set of analytic rubrics thataid the calculation of different indicators including a one main indicator called the SustainabilityIndicator. The nine sustainability criteria are: leadership and governance, student learning byacademic program, student learning by research program, faculty research and consultancy,industry and community engagement, academic support services, student support services,faculty and staff support services, and general support services and facilities. This frameworkcan be used by programs, colleges or institutions to guide them in system-wide development andmeasurement of policies, practices and procedures to ensure not only sustainability, but also topositively impact student, faculty and staff learning for continuous improvement purposes.In this paper, we focus on student learning by academic program and present the results andanalysis of a pilot study using a case-study methodology. Included is a discussion on thepresented comprehensive evaluation tool’s usefulness for the continuous improvement atprogrammatic and institutional levels, as well as for collecting and providing evidence for qualityassurance and accreditation organizations, such as ABET.This study’s theoretical proposition is that the presented measurement tool accurately a)describes the content and constructs that comprise sustainability of technical education in ahigher education setting, and b) measures the sustainability. The case study aims to answer thefollowing questions: • Do measurement tool scores reliably provide information about the sustainability of education that the program provides? • To what extent does the implementation of the study assessment bring benefits to participating program? • To what extent does the implementation of the study contribute to demonstrating its value for the improvement of sustainability of education?The presented framework provides opportunities for wide and deep measurements. Themeasurements could be interpreted per criteria, measures, indicators, and/or combined forms.The 5-point rubric scale of indicators – Nascent, Beginning, Developing, Competent, andAccomplished are mapped onto constant values. The measured indicators are then each dividedby measurements from a reference institution for normalization and for producing performanceratios. One of the combined measurement forms is the Sustainability Indicator (SI), which is theGeometric Mean of all ratios.