Skip to Content

California State University, Sacramento
Phase II Report
March 15, 1997

Table of Contents

Chapter 1 - Introduction

Responses to Review Team Requests for Additional Documentation

Chapter 2 - Additional Data on Student Outcomes: Methodological Framework
Desired Student Outcomes
Survey Instruments
ACT-COS Graduating Student Teaching and Learning Survey
ACT Alumni Survey
Student Needs and Priorities Survey (SNAPS)
ACT-COS Non-Returning (Withdrawing) Student Survey
General Education Survey on Teaching and Learning
CASPER Surveys
Assessment of General Education Race and Ethnicity Courses
Non-Survey Sources of Data
Remaining Issues

Chapter 3 - Teaching and Learning at CSUS
Desired Student Outcomes
Overall Perception
Perception of General Education
Perception of Major
Perception of Faculty Teaching Practices
Perception of Intellectual Stimulation
Perception of Library and Information Resources
Concluding Thoughts

Chapter 4 - Student Outcomes at CSUS
Desired Student Outcomes
Critical Thinking Skills
Information Skills
Communication Skills
Quantitative Skills
Pre-professional Skills
Retention and Graduation
Concluding Thoughts

Chapter 5 - Learning Community at CSUS
Desired Student Outcomes
Cultural Diversity
Intellectual Tolerance
Racial Integration
Sense of Belonging
Access to Faculty
Administrative/Student Support
Career Assistance
Facilities and Services
Concluding Thoughts

Chapter 6 - The Ripple Effect: From WASC Self-study to Institutional Commitment to a Culture of Evidence
The Institutional Commitment: University Assessment Model
Link Among Assessment, Planning, and Budget
Using Evidence to Build Institutional Effectiveness:
Actions Taken in Response to Assessment Findings

Teaching and Learning Theme

Class Schedule
Learning Communities
Technical, Communication and Research Skills
Use of Technology in Teaching
Service Learning
Other responses

Campus Life Theme

Student/University Relationship
Student Centeredness
Information on Campus Events
Student Input to Decision-making
Campus Environment
Student Support Services

Enrollment Planning Theme

University enrollment management
School-based retention
Entrance requirements
Writing Proficiency Examination
Schedule Improvements

Pluralism Theme

Campus climate
Staff development

Academic Programs Theme

Distance education
Outcomes assessment
Regional emphasis of programs

Public Life and Capital Campus Themes

Intercollegiate Athletics
Regional partnerships
State capital partnerships

Faculty Scholarship Theme


Chapter 7 - Creating an Infrastructure to Support Assessment
Academic Program Assessment Initiatives

Biological Sciences
Electrical and Electronic Engineering
Special Education, Rehabilitation, and School Psychology

Nonacademic Program Assessment Initiatives

Procurement Services

Faculty Evaluation
Current Infrastructure to Support Student Outcomes Assessment in Academic Programs

Role of the Associate Vice President for Academic Affairs
Role of Institutional Studies
General Education

The Infrastructure: Additional Plans for the Future



The history of accreditation on this campus, as is probably the case with most others, is a history of compliance, not culture change. What usually happens is that a comprehensive inventory of the thousands of parts of the whole is submitted in a report approximating the size of a large metropolitan telephone directory. Everyone treats the exercise with the gravity usually accorded a major IRS audit, but this is pretense--which all parties to the ritual charade understand at least at some level. Absent huge and unmistakable felonies, the accrediting agency accepts the document with the implicit understanding that reaffirmation will be forthcoming along with a few recommended tweaks in academic policy that do no serious damage to deeply vested interests. After much self-congratulation all around and a decent interval, everything settles back into pleasant slumber for another seven or so years.

For whatever reasons, WASC and CSU, Sacramento entered into a highly unusual bargain for the current accreditation effort, one that could not come at a more critical moment in the history of higher education. The terms of the agreement, simply put, are that the campus could forego "writing to the nine standards" that have been at the heart of the compliance model for decades, and which usually produce a counting house document presenting a detailed picture of a good many trees, while leaving the forest virtually unexamined.

By sharp contrast, this campus was permitted to design its own approach centering on several "pressure points" within the academic enterprise and to write a relatively brief document that focused on highlights and left detail to an Appendix. This represents nothing less than a paradigm shift in accreditation.

The assumption was that in accreditation reports, not unlike architecture, less is more. By identifying and trying to develop a deep understanding of those aspects of the university that are at the center of a student's intellectual development, we would be doing something more worthwhile than the traditional effort, which can be likened to the Powder River in that it is three miles wide and only about an inch deep. Beyond anything else, our experimental approach was to move toward building a culture of evidence that presumably could serve as the foundation of cultural change in the broadest sense.

This experimental path, we have discovered, has not always been smooth. Those doubtful of the approach, while not huge in number, are nonetheless vocal. There are those who feel that the Self-Study slighted their particular areas of interest (research and scholarship come to mind). Others are concerned about the report's extensive use of survey data, which, measures attitude and perception rather than behavior, not to mention those who feel the draft report did not sufficiently address our academic programs, the center of the campus enterprise. There is some concern about the prospects for lasting change on the campus from a focus on building a culture of evidence for WASC Self-Study, a concern also raised by some members of the WASC team.

For those who feel that important areas of university life were left unexamined, one can only argue that what we have done at the macro level will pave the way for a far more useful inquiry into the micro than has usually been the case in the past. For instance, scholarship and research, at least at a teaching institution such as CSU, Sacramento, probably is most usefully explored within the context of larger questions, such as how well students are served on the campus. We have now begun to answer some of those questions and to do so in ways that may move us beyond the usual terms of debate over research/scholarship vs teaching, an intellectual cul de sac that has served no one well.

As for the survey methodology which is at the heart of the current study, one has to begin somewhere if we are to get beyond the culture of anecdote that prevails on this campus, particularly in the realm of teaching effectiveness. The WASC survey of graduating seniors, to take only one example, is the first such survey in the university's history. Already it has provided useful data to departments undergoing review and to a working group of faculty exploring the effectiveness of teaching at CSUS, and it will provide a benchmark for future surveys, which is the first step in building any culture of evidence. To be sure, such surveys measure perception not reality, but to the degree that what is believed is thought to be true, perception becomes reality--and therefore we'd best be aware of what our students and alumni think is true. Finally, in this regard, if we understand the dominant anthropological approach correctly, assessment of culture must precede any attempts to plan for change. We have begun that assessment.

The absence of judgments about quality of programs in the draft report is not an inconsequential complaint. That said, we had neither the resources not charge to judge program quality in any meaningful sense at the macro level. We did feel we had sufficient resources and information to raise significant questions about our programs and performance that cannot help but lead to questions of quality in future inquiry, and these are included at the end of each chapter. At the micro level, we assumed that including student outcomes assessment in our program review process, for which the current WASC process was directly responsible, would bring into sharper focus questions of programmatic quality.

Although they are not always readily apparent, perhaps most exciting about the WASC experience are the prospects for cultural change which it has made possible. Where previous accreditations have led to changes in specific programs (e.g., General Education), there is every reason to believe that the one now underway could lead to much broader cultural change. In this regard, the experimental WASC approach has served as an incubator for some insignificant and long overdue initiatives.

Whether or not these "ripples" are sufficient to impress the WASC team, what must be understood is that "ripples" they are, and far greater so than any of which I'm aware from past accreditation efforts. One "ripple" effect that may prove the most important of all in the long term, yet is virtually impossible to demonstrate, is the reality that the experimental approach has profoundly changed the nature of the conversation at CSUS. For instance, instead of asking whether we have enough computers and parking spaces, we're beginning to ask whether there is any evidence that we've made a substantive difference in the lives of our students. I think we've taken some important steps to make certain the conversation continues, but, finally, whether it continues or not will have far more to do with the willingness of people to engage in change than whether we can create new agencies for affecting it. We've got more of a chance now than we had before the most recent WASC exercise, and I'll settle for that. After all, life's not an insurance company.

Comments of Professor William Dorman
WASC Steering Committee Member
January 1997


Chapter 1


Change is in the air. Three years ago California State University, Sacramento (CSUS) and Ralph Wolff, Executive Director of the Western Association of Schools and Colleges (WASC), entered into an agreement with CSUS to pilot a new approach to reaccreditation. Instead of preparing a Self-Study organized around the traditional nine standards, CSUS decided to undertake a Self-Study that documented institutional effectiveness through assessment, particularly student outcomes. Instead of the traditional week-long 20-person team visit to the campus, there would be two campus visits with a much smaller team. Further, in a May 1996 planning meeting, WASC and CSUS agreed that every aspect of the visits was negotiable.

CSUS accepted the challenge. By the time of the WASC visit, far-reaching changes had accumulated and the Self-Study document took shape. The report, Self-Study (Phase I), described efforts to document the character and effectiveness of the institution with data in three areas selected for study by the campus WASC Steering Committee: Teaching and Learning, Student Outcomes, and The Learning Community. The document also contained a chapter briefly describing how CSUS has built upon the WASC Self-Study process to institutionalize a culture of evidence. Congruent with agreements reached at the May planning meeting, the report was uncommonly short--about 100 pages (supplemented by an extensive appendix).

Dr. Virginia Smith, leading a seven-person team, visited CSUS December 4-6,1996. The WASC response was not long in coming. On December 14, she wrote to President Gerth:

  1. The team was so impressed by the many ripple effects that have accompanied the Self- Study's focus on assessment, that it recommends the Self-Study be augmented to include further description of these other activities. Some of these were mentioned in Chapter 6 of the Self-Study but they were much more fully described, particularly in terms of emerging results, in the various meetings the team held on campus. In those discussions the team began to see a developing conversation about assessment that could have a real impact on the overall way the University does its work.
  2. During the course of our conversations on campus, and also to a more limited extent in Ch. 6, it became clear that the University has a great quantity of data. We think there are other data that could be drawn from this store relating to the themes of the Self-Study that would enrich the treatment of the three themes. And, of course, information from the Focus Groups, which had not been completed at the time of Phase One of the Report, could also be incorporated. Some of the Strategic Plan themes discussed in Council for University Planning (CUP) meetings also overlap with the Self-Study themes, but use additional data.
  3. The team was concerned at many points about how this emerging and still fragile attention to assessment would continue after the involvement with WASC is no longer a motivating factor. Is there an infrastructure that will provide the focus on assessment after the Steering Committee ceases to exist? We realize that changes of the sort we are talking about don't occur in one unit or office. To become institutionalized it has to be a systemic change. What the team lacked is an understanding of how the parts of the system are linked and what is considered the responsibility in each part and the connections among the various parts that will facilitate the continuation of this strengthened emphasis on assessment. Some of this was clarified in the discussions on campus but it would help to have it as a part of the report. To make it a part of the Self-Study would signal that the University sees the establishment of a recognized infrastructure as an important element in continuing and building a culture of evidence. Without some deliberate and conscious steps to provide an adequate infrastructure most institutions trying to change find themselves relaxing to their "default" mode of operation. We know that some of this infrastructure is being forged now and would like that described. We know that other steps, including ways to provide a broader base of knowledge about assessment in the faculty and staff, are under consideration and may already have started. We would like to hear about them also.

Self-Study--Phase II: Responses to Review Team
Requests for Additional Documentation

In the opinion of the WASC visiting team, the CSUS Self-Study document (Phase I) represented only a small fraction of the assessment activities, and evidences of change, that they noted in conversations or witnessed first hand during the first campus visit. Their request to us was to more fully document for them, and for the Senior Accrediting Commission, the multitude of changes that have occurred on the campus as a consequence of the University's decision to undertake this experimental Self-Study almost four years ago.

The visiting team asked for all available data that illuminates our understanding of the three WASC Self-Study themes of Teaching and Learning, Student Outcomes, and The Learning Community. We present these in Chapters 3, 4, and 5. Although corresponding to the original Chapters 3, 4, and 5, their contents and organization differ from those of Phase I in several ways:

  • Each of the three new chapters is organized around specific desirable outcomes. These outcomes are survey-derived perceptual measures of learning and environment, complemented by objective measures such as test scoring.
  • The data in each chapter are limited to students. Staff and faculty perceptions were presented in Chapter 5 of Phase I.
  • The framework presented in each chapter creates the basis for an ongoing institutional tracking of student outcomes.
  • Additional data have been added to those collected under WASC auspices.
  • Chapter 2 outlines Chapters 3, 4, and 5 and explains the methodological framework.

We have chosen to document the "ripple effect" (Smith Letter #1) and to describe the "sustaining infrastructure" (Smith Letter #3) in Chapters 6 and 7. Because the two topics are interrelated in so many ways, they serve mainly as touchpoints in the wide- ranging discussions in Chapters 6 and 7, and should not be considered defining categories at this time.

Chapter 6 builds on Chapter 6 of Phase I by more fully describing the consultative and governance infrastructure that we have created to link assessment, planning, and budget allocations. Chapter 6 also partially addresses the team's request for a more complete description of the "ripple effect". A significant number of changes in practices, programs, and activities that have occurred as institutional responses to "evidence" are described.

In Chapter 7, we contribute further to the team's understanding of the "ripple effects" on campus by providing detailed descriptions of the Assessment Plans for selected Academic programs. During their campus visit, team members met with representatives of these departments to discuss the plans briefly summarized in this chapter. In this chapter, we include a summary of a non-academic program review, and we describe at length another consequence of our WASC endeavor, efforts underway by faculty to rethink how faculty are evaluated.

The presentations of Chapters 6 and 7 taken together are integral to how CSUS plans to sustain its commitment to using evidence in making decisions in the future. While Chapter 6 describes the formal governance processes that will sustain the change, Chapter 7 describes the creation of expertise and expectations among our faculty, staff, and students critical to any longer-term change. We conclude Chapter 7 by describing other efforts designed to institutionalize change over time.

Chapter 2

Additional Data on Student Outcomes:
Methodological Framework

Phase II of the WASC Self-Study examines student outcomes using a two- dimensional matrix with the rows of the matrix representing desired student outcomes and the columns representing the instruments from which indicators of those outcomes are obtained. The cells of the matrix represent the specific number of indicators (e.g. survey questions) that provide data describing the extent to which CSUS is achieving the desired outcomes.

Desired Student Outcomes

Chapter 3: Teaching & Learning (Perceptual measures of learning)


3.1 Overall Perception: Students leaving CSUS are satisfied with the overall quality of the instruction they received at CSUS.
3.2 Perception of General Education: Students leaving CSUS are satisfied with the contribution General Education made to their personal development.
3.3 Perception of Major: Students leaving CSUS are satisfied with the technical skills and professional preparation acquired while enrolled in a major area of study at CSUS.
3.4 Perception of Faculty Teaching Practices: Faculty employ effective teaching practices.
3.5 Perception of Intellectual Stimulation: CSUS intellectually stimulates and inspires students.
3.6 Perception of Library and Information Resources: CSUS provides students with the library and information resources necessary to complete a superior university degree.

Chapter 4: Student Outcomes (Objective measures of student learning)


4.1 Thinking Skills: Students leaving CSUS think clearly and logically.
4.2 Information Skills: Students leaving CSUS find information and examine that information critically.
4.3 Communication Skills: Students leaving CSUS communicate effectively both orally and in writing.
4.4 Quantitative Skills: Students leaving CSUS reason quantitatively.
4.5 Pre-professional Skills: Students leaving CSUS are prepared to assume career responsibilities, meet credential requirements, and gain admission to graduate or professional schools.
4.6 Retention and Graduation: CSUS retains students who have not yet achieved a credential or degree and graduates students seeking a degree

Chapter 5: Learning Community
(Perceptual measures of learning community--from the perspective of students)


5.1 Cultural Diversity: Students leaving CSUS appreciate cultural diversity.
5.2 Intellectual Tolerance: Students leaving CSUS have been intellectually stimulated, but not harassed or coerced by, contradictory political, moral, or spiritual beliefs.
5.3 Racial Integration: Students interact with students of other racial/ethnic groups in a variety of settings.
5.4 Non-Discrimination: Students are judged by character, effort, and performance, rather than by gender, ethnicity, or other personal characteristics.
5.5 Sense of Belonging: Students have a sense of belonging to the CSUS campus community.
5.6 Access to Faculty: Students have ample access to faculty.
5.7 Administrative Support: Faculty, staff, and administration handle the administrative details of acquiring an education conveniently, pleasantly and in a helpful manner.
5.8 Career Assistance: CSUS assists students in beginning careers.
5.9 Physical Resources: CSUS provides adequate physical facilities and resources for learning.
5.10 Advising: CSUS provides helpful advising services.



Survey Instruments

ACT-COS Graduating Student Teaching and Learning Survey - The ACT-College Outcomes Survey (ACT-COS) was administered to the CSUS graduating class of 1995. Thirty campus-specific questions, developed by a subcommittee of the Steering Committee, were added to the survey. A great majority of the added questions were based on the 1987 Wingspread report, Seven Principles for Good Practice in Undergraduate Education, which in turn represented a synthesis of "best practices" derived from a number of studies of pedagogy in higher education.

A total of 3,994 graduating students (2,902 undergraduate and 1,092 post baccalaureate) were sent the ACT-COS and the campus-specific Teaching and Learning Survey. The response rate was 15.6 percent (N=622). Among the respondents, 67.5 percent were female and 32.5 percent were male. Another 2,000 Teaching and Learning surveys were distributed to selected General Education courses. Of the 657 General Education students (32.9%) who completed the surveys, 53.4 percent were females, and 35.2 percent were males (the balance declined to identify gender). Both groups of students responded to the same 30 Teaching and Learning questions, but graduating seniors were asked to answer them in terms of their experience in the major while the on-campus group responded only in terms of their experience with General Education at CSUS.

ACT Alumni Survey - By far the largest group of respondents to any of the surveys was an alumni sample who responded to the ACT-Alumni Survey sent to 1,667 alumni in departments undergoing program reviews during the 1995-96 cycle. Alumni from ten programs undergoing program review were surveyed: Asian Studies, Bilingual- Multicultural Education, Biological Sciences, Counselor Education, Electrical and Electronic Engineering, French, German, Interior Design, Spanish, and Special Education. A total of 1,667 surveys were mailed with an 18 percent response rate (N=291).

Student Needs and Priorities Survey (SNAPS) - The Student Needs and Priorities Survey (SNAPS) is a study conducted every four years by the Chancellor's Office of the California State University (CSU). Its purpose is to determine the characteristics, objectives, and needs of CSU students, and the degree to which campuses are meeting student objectives and needs. The most recent study was conducted in Spring 1994. The survey gathered responses of 952 CSUS students. This sample was representative of the student population in terms of ethnicity, gender, and class level.

ACT-COS Non-Returning (Withdrawing) Student Survey - This survey paralleled in form and, to some extent content, of the ACT-COS Graduating Senior Survey. It took approximately 15 minutes to complete. The sample size completing this survey totaled 188 non-returning students.

General Education Survey on Teaching and Learning - The campus- specific questions on Teaching and Learning were administered to students in selected General Education classes, along with four additional questions. The sample size completing this survey totaled 655 students enrolled in General Education courses.

Computer Assisted Student Phone Registration (CASPER) Surveys - Students who phone the University during CASPER schedule classes or make changes to their schedules are required to answer two questions before completing their call. During Spring 1995, for example, all students responded to a question on their interest in and ability to participate in NCAA sanctioned sports. Students were also asked to respond to one other question randomly selected from a pool of ten questions developed by Academic and Student Affairs. Selection was based on the last digit of the student's social security number. During Spring 1995, 11,054 students responded to the NCAA question and approximately 1,000 students responded to each of the ten remaining questions. During Fall 1995, 21,436 students responded to a pool of 20 questions. Approximately 2,000 students responded to each question. During Spring 1996, roughly the same number of students responded to a different pool of twenty questions.

Assessment of General Education Race and Ethnicity Courses - In Fall 1994, 1,701 students enrolled in "Race and Ethnicity in American Society" General Education courses were surveyed at the end of the semester to determine whether they perceived that the course they were enrolled in met the objectives, which were to develop ". . .a significant and useful understanding of the perspectives and contributions to human activities and experiences of people from a diversity of cultures and backgrounds." Of the students surveyed, 60 percent were female; 40 percent male; 41 percent were seniors; 27 percent were juniors; 20 percent were freshmen; and 10 percent were sophomores. Self- identification of race/ethnicity status revealed 54 percent were Caucasian, 18 percent were Asian, 11 percent were Mexican American and Latino, and 6 percent were Black. The General Education Committee used the criteria statements for the "Race and Ethnicity" category in developing the instrument.

Non-Survey Sources of Data

Over the past several years a variety of non-survey instruments have been used to gather data from students. Collegiate Assessment of Academic Proficiency (CAAP) tests were used to examine the critical thinking and writing skills of CSUS students. A survey of mathematical skills was added to the CAAP test when it was administered to graduating students.

Four of the standardized tests currently being taken by CSUS students to meet credential requirements and to determine admission to graduate and profession schools were reviewed as part of the assessment of student outcomes. The Graduate Record Examination (GRE) is a general test of verbal, quantitative, and analytical reasoning designed to assess undergraduate achievement or the qualifications of students for graduate study. Since 1983, prospective teachers have been required to take the California Basic Educational Skills Test (CBEST) to ensure an acceptable level of competence for credentialed K-12 teachers. The Medical College Admission Test (MCAT) measures proficiency in verbal reasoning and writing, and competency in biological and physical sciences. The Law School Admission Test (LSAT) is used by Law Schools for admissions purposes.

Other sources of data for Phase II of the Self-Study included results obtained from CSUS's Writing Proficiency Exam (WPE). All CSUS students are required to pass the Writing Proficiency Exam as a condition of graduation. University data regarding student retention and graduation rates were also examined as part of phase II. Focus groups were conducted in late November 1996 with faculty, staff, and students. It was difficult to incorporate the results into this methodological framework. A summary, however, is included in Appendix A. Student response was particularly low, probably because both focus groups were conducted in the evening.


Remaining Issues

One problem with our approach to assessment at this time is the absence of standards against which to measure the data. For instance, is the finding that nearly a quarter of the students rated their major curriculum as "less than adequate" or "very poor" in providing needed technical skills cause for alarm or not? In considering all of its data, the Steering Committee really had no way of saying what was an "acceptable" threshold and what was not. A second issue is whether or not "outcomes" should be restricted to those sources of data that truly measure outcomes, or whether certain inputs should also be considered. A third issue is that a primary focus on student outcomes assessment left other members of our Learning Community, i.e. faculty, staff, and administration, out of the picture. Phase I was more inclusive and looked at all four campus groups, whereas phase II took the more narrow approach of looking only at student outcomes. A fourth issue pertains to the outcomes chosen for future study. It would be presumptuous of the authors to assume that this report contains a static and enduring set of outcomes for further study. Rather this report proposes a starting point, a draft set of indicators and outcomes. Future assessment efforts will undoubtedly edit the list, refine the content, and improve the form.



Chapter 3


A teacher affects eternity; (s)he can never tell where her/his influence stops.

--Henry Adams

CSUS has always identified itself as a teaching institution, one in which the pursuit of excellence in teaching is of critical importance. The dimension of teaching and learning, therefore, is an important theme upon which to focus the University's Self-Study efforts. The Steering Committee wanted to explore what was happening in our classrooms--how instructors instruct, how learners learn, and what everyone thinks about the process. The Committee decided it had to move beyond the prevalent "culture of anecdote" to a "culture of evidence"--in other words, to create an environment that replaced the telling of ostensibly representative stories with more objective and supportable information.

Toward that end, the Steering Committee launched the most far-reaching and systematic survey of perceptions about pedagogy and related issues in the history of this campus. The goals were threefold: (1) to produce a comprehensive profile of student, faculty, and alumni perceptions of teaching and learning at CSUS; (2) to establish a baseline against which the results of future departmental self-studies and program reviews could be measured; and (3) to suggest a range of questions about teaching and learning that would engage the campus community during and after the WASC Self-Study. Ultimately, the results will be used as a basis for decision-making. There has been significant progress toward each of these goals. While Chapter 3 in Phase I primarily focused on teaching effectiveness, this chapter will focus on student perception of outcomes.


Desired Student Outcomes

Teaching and Learning will be operationally defined using these six outcomes: student perceptions of General Education, Major, Faculty Teaching Practices, Intellectual Stimulation, and Library and Information Resources; and Overall Perception. (A point of terminology: although Library, e.g., can be seen as an input, student perception of Library adequacy is regarded as an outcome.)

We describe each outcome and list its survey indicators. Tables and graphs summarize the percentages of positive responses for each of the indicators. Positive responses typically come as response options such as "excellent" or "good;" "very satisfied," or "satisfied." Neutral responses ("fair," "somewhat satisfied," "undecided") and negative responses ("poor," "unsatisfied," "disagree") are not reported but can be found in the appendices to Phase I of the WASC Self-Study.

3.1 Overall Perception: Students leaving CSUS are satisfied with the overall quality of the instruction they received at CSUS.

3.2 Perception of General Education: Students leaving CSUS are satisfied with the contribution General Education made to their personal development.

3.3 Perception of Major: Students leaving CSUS are satisfied with the technical skills and professional preparation acquired while enrolled in a major area of study at CSUS.

3.4 Perception of Faculty Teaching Practices: Faculty employ effective teaching practices.

3.5 Perception of Intellectual Stimulation: CSUS intellectually stimulates and inspires students.

3.6 Perception of Library and Information Resources: CSUS provides students with the library and information resources necessary to complete a superior university degree.

3.1 Overall Perception - Students leaving CSUS are satisfied with theoverall quality of the instruction they received at CSUS. Indicators for this globaloutcome came from two sources: (1) survey questions that ask respondents to rate CSUS as, for instance, "excellent," "good," "fair," or "poor" and (2) indicators that ask respondents for their inclination to engage in some behavior such as "recommending CSUS to another person."

Indicators of Overall Perception:

1. If choosing a college I would choose this one
2. I would recommend this college to others
3. Quality of instruction
4. This college in general
5. Overall quality of instruction
6. If you could begin again, would you attend this school?
7. Overall, how would you rate this school (for the time during which you were attending it)?
8. Quality of instruction
9. I am pleased with my overall experience on this campus
10. If choosing a university again, I would choose this one
11. I would recommend this university to others

The overall impression of CSUS, as measured by three different survey instruments, was generally positive. On the ACT Alumni Survey (ACT Alum), better than four out of five respondents rated CSUS positively. The same survey found that 82% rated the overall quality of instruction positively. Across all of the indicators in this outcome category, three of every four evaluations were positive (76.5%).

For some reason, the quality ratings tended to be much higher than the behavioral indicators. Among those responding to the ACT Alumni survey, 85% rated CSUS as "excellent" or "good." By contrast, the Student Needs and Priorities Survey (SNAPS) of enrolled students found that only 46% would repeat the behavior of attending CSUS. Similarly, only 57% of those responding to the ACT Graduating Senior Survey (ACT Grad) would choose CSUS if they had it to do all over again. Perhaps many of those hypothetical responses not to attend CSUS were more the result of immediate practical and temporal student concerns than they were the result of negative attitudes toward CSUS. The exception to this inconsistency between attitudes and behavior came from the ACT Alumni Survey. Here, 77% of the respondents said they would attend CSUS all over again.

It was disappointing to discover that more CSUS students would not "recommend CSUS to others." Only 70% of the respondents to the ACT Graduating Senior Survey, and 61% of the respondents to the SNAPS Survey, said they would recommend CSUS to others. This is an indicator that needs to be taken seriously. One would hope that future measurements of outcomes will find a higher percentage of interviewees recommending CSUS to students seeking higher education.

In looking at much of the same data, the WASC Steering Committee came to the same conclusion in Phase I of the Self-Study. The committee concluded that, in general, student perceptions of CSUS are markedly favorable. A significant majority of graduating students surveyed in the ACT Grad survey in Spring 1995 reported positive perceptions of the University.

A majority of enrolled students who responded to the SNAPS survey had positive perceptions of the instructional experience at CSUS (Appendix G). Sixty-one percent reported being pleased with their "experiences at this University," and 61 percent said they "agreed" or "strongly agreed" with the question that asked them if they "would recommend this University to others."

Similarly encouraging were alumni responses to the question, "How would you rate the university?" Thirty-one percent responded "excellent," 56 percent responded "good," 12 percent responded "average," while only two percent responded "poor." (ACT- Alumni Survey, 1995)

The WASC Committee noted contradictions in some of the overall perceptual data. For instance, 31 percent of the alumni respondents rated the University as "excellent" and 56 percent rated it as "good." On the other hand, 46 percent said they would recommend it only "with reservation." Similarly, 33 percent said they would definitely attend CSUS if they had it to do over again, 46 percent said "probably yes," 13 percent responded "uncertain," and 7 percent said "probably no." In some respects, this is less than unbridled enthusiasm. It might be useful to explore the degree to which teaching effectiveness may have something to do with these mixed feelings.


3.2 Perception of General Education - General education serves a wide variety of purposes. The goals of General Education include fostering intellectual growth and moral development as well as increasing understanding and appreciation of the arts. Ultimately, general education should contribute to students' personal growth and prepare them for effective citizenship. Generally stated, the desired outcome is that Students leaving CSUS are satisfied with the contribution General Education made to their personal development.

Indicators of General Education:

1. Appreciating the fine arts, music, literature, and the humanities
2. Develop as a "whole person"
3. Acquiring a well-rounded general education
4. Intellectual growth (acquiring knowledge, skills, ideas, concepts, analytical thinking)
5. Personal growth (developing self-understanding, self-discipline, and mature attitudes, values, and goals)
6. Understanding and appreciating art, music, literature, etc.
7. The general education or core requirements at this school were a valuable component of my education
8. I have used what I learned in my GE course(s) at CSUS in my major coursework
9. I have used what I learned in my GE course(s) at CSUS in work, social, volunteer or other activities apart from school
10. My learning experience in GE courses at CSUS have helped me deal more effectively with personal, moral or social problems I have had
11. My learning experiences in GE courses at CSUS have helped me appreciate and understand people from diverse groups

A significant finding from this entire assessment of perceived CSUS outcomes is that students are leaving CSUSnot satisfied with the contribution General Education has made to their personal development. Generally speaking, only half of the respondents believed that General Education was a valuable component of their education. The mean percentage of positive responses (51.7%) was the lowest of any of the six perceptual measures of learning discussed in this chapter. Clearly this conclusion, based on several measures, seems valid. Indeed, three of the lowest percentage scores came from a survey that was specifically designed to measure perceptions of General Education.

The lowest scores appeared to cluster around the ability of General Education to do as the name implies, generalize. For some students General Education did not "help (them) deal more effectively with personal, moral or social problems" (47% positive), largely failed to be "used... in my major coursework" (36% positive), and was not "used... in work, social, volunteer or other activities apart from school" (29% positive). On the other hand, 81% of the respondents to the ACT Graduating Senior's Survey did rate their intellectual growth (acquiring knowledge, skills, ideas, concepts, and analytical thinking) as "very great" or "great." Both General Education and major coursework include these goals.

The WASC Steering Committee similarly noted that fewer than half of the graduating students said that their education at CSUS prepared them "very much" or "much" to participate effectively in the electoral process, to recognize their rights, responsibilities and privileges as citizens or to gain insight into human nature through the study of literature, history, or the arts.

While it may be the case that alumni are simply now reporting the lack of something in their education that they themselves did little to embrace, it may also be the case that their recollection of the CSUS curriculum, and that of graduating students as well, reflects an emphasis in the classroom on the transmission of information to the exclusion of an examination of questions related to values and their roles as citizens. If so, this is cause for concern given the promise in the University's Mission Statement that "[CSUS] will be known for graduating students with the knowledge and skills to assume productive roles in society."

The WASC Steering Committee was concerned about several aspects of the University's General Education program, both its pedagogy and intended goals. A comparison of General Education with "major" teaching (Appendix K) found that good teaching practices in virtually every category are reported less frequently--sometimes dramatically so--in General Education courses compared to major courses. Seventy-one percent of the graduating students indicated that professors in major courses had high expectations and were willing to help them "almost always" or "frequently" compared to 55 percent of the students enrolled in General Education courses. Similarly, 70 percent of the graduating seniors, and only 46 percent of the General Education students, noted that instructors use a variety of teaching methods "almost always" or "frequently." Seventy-five percent of the graduating seniors compared to 46 percent of the General Education students claim that faculty knew their names "almost always" or "frequently." Less than 50 percent of the General Education students indicated that they needed to use the library to complete assignments, compared to 71 percent of the graduating students. More cooperative learning experiences, both in and out of the classroom, appear to occur in major courses than in General Education courses.

Some members of the WASC Steering Committee observed that a good many students at a comprehensive university, such as CSUS, consider General Education a hurdle to be surmounted before they can get on to what matters to them--their major. To be sure, the survey asked students to recall specific classroom practices, and not to express their general impression toward the program itself. That said, there can be no guarantee that their attitudes toward General Education did not enter into the results. (The complexity of the program and its seeming arbitrariness have been suggested as possible explanations for the somewhat negative student views of the program.)

Although 61 percent of the graduating students indicated that most of their General Education coursework was completed at CSUS, exactly how much is not known. More than 75 percent of the undergraduate student population at CSUS transfer primarily from community colleges. While the students may not be critiquing our General Education program specifically, the responses indicate perceptual differences in students' experiences of General Education and major courses.

Other explanations of the perceived differences between General Education and major courses can be offered. Some faculty may resent General Education courses almost as much as the students; they might prefer teaching an upper-division specialty course. Their displeasure may be communicated to their students. Large class enrollments in some General Education courses may discourage faculty from using the same classroom practices in General Education that they use routinely in major instruction.

Whatever the explanation, the question of pedagogy and general education is clearly one that has been raised by this inquiry and should be pursued. Student resistance to the program or not, it is precisely the General Education program that separates CSUS from, say, those proprietary universities that primarily specialize in professional career development and training.


3.3 Perception of Major - While many students might be willing to give CSUS a pass on its General Education program and still rate the university as excellent or good, it is doubtful they would be so lenient with regard to their major area of study. With the major comes important and immediate personal concerns with their own future careers. Hence, this outcome is critical: Students leaving CSUS are satisfied with the technical skills and professional preparation acquired while enrolled in a major area of study at CSUS.

Indicators of Major:

1. Acquiring knowledge and skills needed for a career
2. Becoming competent in my major
3. Quality of my program of study
4. Quality of the program in my major/field
5. Classes that are focused on career concerns
6. Relevance of coursework to major
7. Courses in my major that are required for graduation
8. My experiences here have equipped me to deal with possible career changes
9. Course content in your major field
10. Instruction in your major field
11. Preparation you are receiving for your future occupation
12. How well does the curriculum in your major provide you with broad knowledge of theories and principles in the discipline?
13. How well does the curriculum in your major provide you with needed technical skills?
14. How well does the curriculum in your major provide you with understanding of the methods and practices of the profession?

For the most part, student satisfaction with the major was markedly better than satisfaction with General Education. Four out of every five alumni reported that they were either "very satisfied" or "satisfied" with their major. More to the point, better than two- thirds of all the responses to indicators of the "major" outcome were positive.

There was, however, a large variability among the scores achieved by different indicators. Although the performance of most majors in "providing a broad knowledge of theories and principles" was superior (84% positive), its performance in providing "preparation for a future occupation" was much less satisfactory (46% positive).

Interestingly, three of the four highest scoring indicators came from the Phone Registration (CASPER) data. While this may be indicative of the University's success with this particular outcome, it may be an artifact of the scale used to conduct the CASPER survey. "Positive" on the CASPER survey was operationally defined as "exceptionally" plus "more than adequately." The response category of "more than adequately" may not, however, be an unambiguously positive response. Although the logistics of an automated telephone interview limit choices, authors of future CASPER surveys might want to experiment with more conventional response categories.

If a need is suggested by these data, it would come in the area of specific career preparation. Most University departments appear to provide students with the methods and practices of respective professions, along with necessary technical skills. A more frequent shortcoming, however, involves the transition to a career or profession. Only 46% expressed confidence in their preparation for their future occupations. It may be that students believe they are technically equipped to begin a career, they simply feel unprepared about getting started in their chosen career. Given the apparent desire for assistance, academic departments might consider offering professional seminars within the major area of study or engaging student professional organizations in this effort.


3.4 Perception of Faculty Teaching Practices - Not surprisingly, given its central importance, faculty teaching practices has the longest list of indicators, covering all aspects of the student-teacher interaction. We would like students to report that Faculty employ effective teaching practices.

Indicators of Faculty Teaching Practices:

1. A variety of teaching methods was used by professors (lecture, small group, projects, etc.)
2. Grading standards were higher than those of high school and/or community college
3. Homework, papers, examinations, etc. were graded and returned promptly (1-2 weeks)
4. Professors were responsive to student questions and concerns
5. Instruction was systematic. New information was connected to what was previously presented to the class
6. Professors seemed genuinely to enjoy teaching
7. Feedback on homework, exams and assignments was helpful
8. Professors seemed enthusiastic about the subject matter
9. Professors with high expectations were willing to help me meet them
10. Faculty ability to communicate the subject matter
11. Faculty preparation for class
12. Faculty enthusiasm for teaching
13. Fairness of testing and grading
14. Grading standards were higher than those of high school and/or community college
15. Homework, papers, examinations, etc. were graded and returned promptly (1-2 weeks)
16. Professors were responsive to student questions and concerns
17. Instruction was systematic. New information was connected to what was previously presented to the class
18. Professors seemed genuinely to enjoy teaching
19. Feedback on homework, exams and assignments was helpful
20. A variety of teaching methods was used by professors (lecture, small group, projects, etc.)
21. Homework, papers, examinations, etc. were graded and returned promptly (1-2 weeks)
22. Instruction was systematic. New information was connected to what was previously presented to the class
23. Professors seemed to genuinely enjoy teaching
24. Feedback on homework, exams and assignments was helpful
25. Professors seemed enthusiastic about subject matter
26. Professors with high expectations were willing to help me meet them
27. Grading standards were higher than those of high school/or community college

There was little consensus across surveys, even among similarly worded survey questions. For example, the indicators relating to prompt returns ranged from a high of 81% positive on the ACT Graduating Seniors Survey to a low of 68% on the ACT Non- returning Students Survey.

On the other hand, there appeared to be greater consistency within surveys, even across dissimilar indicators. For example, the six indicators drawn from the ACT Non- Returning Students (ACT Non) survey were among the thirteen lowest ranked indicators. By contrast, with the exception of the item relating to "teaching methods," eight of the fourteen highest ranked indicators came from the ACT Graduating Seniors Survey. Common sense would suggest that those who succeed by graduating would be happier with their instructors than those who leave school or transfer before achieving their degree.

Responses to the survey on teaching practices, administered to General Education classes, were the most varied. A variety of the lowest ranked indicator, teaching methods, received only a 46% positive response from students. Among the higher ranked items, "enthusiasm" of General Education instructors was viewed positively by 79% of the respondents.

Overall, 69% of the responses to the twenty different items measuring perception of faculty teaching practices were positive. Except for the ACT Graduating Seniors Survey, many of the lowest scores seemed to come from items pertaining to feedback and responsiveness. This may be an area where instructors need to focus their efforts at self- improvement. Similarly, the "variety of teaching methods" scored low (69% positive ACT Graduating Seniors and 46% positive General Education Student Survey). The one topic that scored the highest across all instruments was "enthusiasm and enjoyment of teaching." Eighty-three percent of the respondents to the ACT Graduating Seniors Survey said that instructors "always" or "frequently" displayed such enthusiasm.


3.5 Perception of Intellectual Stimulation - The outcome of intellectual stimulation centered on two dimensions: (1) challenging and motivating students, and (2) exposing students to new ideas and the free exchange of new ideas. As a university, CSUS has a vested interest in achieving this outcome:CSUS intellectually stimulates and inspires students.

Indicators of Intellectual Stimulation:

1. Developing openness to new ideas and practices
2. Broadening my intellectual interests
3. There was a free exchange of ideas and points of view between professors and students in my classes
4. I was challenged to think about and explore new concepts and theories
5. Overall, the school had an intellectually stimulating atmosphere
6. Courses that stimulate intellectual/interpersonal growth or challenge me
7. This University has helped me meet the goals I expected to achieve
8. My experiences here have helped motivate me to make something of my life
9. I was challenged to think about and explore new concepts and theories
10. I was challenged to think about and explore new concepts and theories

Student perception of CSUS on all indicators of intellectual stimulation was mild in comparison with most of the other outcomes. The overall mean positive response of 63.6% was second lowest among the six outcomes summarized in this chapter. However, none of the indicators scored better than 72% positive. Even the outcome relating to general education had at least one indicator in the 80% plus range. This is clearly an outcome where CSUS could improve.

Unlike the previous outcome of faculty teaching practices, there seemed to be no positive bias on the part of respondents to the ACT Graduating Senior Survey. Both the highest and lowest ranked indicators, respectively, came from the ACT Graduating Senior Survey. The question measuring student perception that the University challenge to think received a 72% positive response. The question measuring the perception that CSUS broadens student's interests received only a 56% positive response.


3.6 Perception of Library and Information Resources - All of the surveys seemed to have one or two items measuring this outcome: CSUS provides students with library and information resources necessary to complete a superior university degree. Indicators came from three categories: library services, library materials and facilities, and library usage. One indicator from the ACT Non-returning Students Survey did double duty and measured both the facilities and services.

Indicators of Library Perceptions:

1. Library/learning resources center services
2. Library services and materials
3. Library materials (books, periodicals, professional journals and informational resources)
4. Library services (e.g. reference desk)
5. Library facilities and services
6. Library use was necessary to complete course assignments (including research papers)
7. How frequently did you use the library last semester for research purposes?

With the possible exception of required library usage, the library outcome scored very well. Were it not for the outcome of overall impression of CSUS, the library outcome would have had the best rating of any of the outcomes. Seventy-four percent of the responses to this outcome were positive. Were one of the indicators not present, the mean percentage of positive responses would have been 78%, which would have made it the highest rated outcome.

Most interesting was the fact that the highest rated indicator came from the ACT Non-returning Students Survey. Typically, scores from this survey were comparatively low. But in this instance, the ACT Non-returning Students indictor achieved a positive score of 84%.

Both services and facilities scored very well. A high percentage of students were happy with the library. The only low score came from an indicator which asked if the library was necessary to complete course assignments. Only 49% of the respondents to the General Education Survey answered that the library was required "almost always" or "frequently." This would appear to be more of an indictment against general education that against the library.

A similar question was not asked regarding use of the library in major courses. Students were asked about the frequency of their library use in the CASPER survey. Here 75% reported that they had used the library five or more times during the semester to complete course assignments such as research papers.


Concluding Thoughts

Much of the data summarized in this chapter points to a teaching culture at CSUS that is appreciated by a good majority of the University's students and alumni. Particularly in the major, teaching effectiveness is reported by students to be generally high--both in terms of general satisfaction and when considered in the context of what the literature has to say about effective teaching practices. The WASC Steering Committee found student perception of experience in General Education more problematic. The table below compares the mean percentage of positive responses for all six outcomes summarized in this chapter. Clearly, the perceived problem area for students is General Education.

Chapter 4

Student Outcomes at CSUS : Perceptual and Objective Measures of Learning

I'm glad someone cares about how well we are learning instead of just how much.

--incoming first-year student

In the late 1980s the California State University system held several assessment conferences for faculty and administrators, and, along with WASC, expected campuses to develop and implement assessment initiatives to assess the effectiveness of their educational programs. In 1993 the CSUS Academic Senate responded by developing a policy, which the President approved, requiring departments to develop assessment plans to measure student outcomes in the major. The 1992 revised General Education program also included an assessment mandate. Faculty and administrators have attended assessment conferences and meetings of the Assessment Forum of the American Association of Higher Education to learn more about assessment.

When the WASC Steering Committee was presented with the opportunity to build assessment into its Self-Study, the selection of the Student Outcomes theme was particularly pertinent. However, when a proposal to have selected academic departments measure student outcomes in the major was challenged, the Steering Committee moved toward a more general assessment of educational outcomes. The Committee was aware that an initiative to examine student outcomes would contribute to future curriculum changes, and, in the long run, might have a positive effect on students, faculty, alumni, employers, and the public. In Phase II student outcomes has been expanded to include more data.

Desired Student Outcomes

The WASC Educational Standards, as well as the liberal arts tradition, require that University graduates demonstrate competence in written and oral communication, quantitative reasoning, critical thinking, and develop an appreciation of cultural diversity. A subcommittee of the WASC Steering Committee examined instruments and studies conducted at other universities designed to measure these competencies. Given the time frame for the Self-Study, the subcommittee realized it would be impossible to develop assessment tools unique to CSUS. Eventually the WASC Steering Committee agreed to examine the performance of CSUS students on existing tests taken by graduating students and to use opinion survey instruments developed by professional testing services that measure student outcomes. Data pertaining to six, largely objective, outcomes are reviewed in this chapter:

4.1 Thinking Skills: Students leaving CSUS think clearly and logically.

4.2 Information Skills: Students leaving CSUS find information and examine that information critically.

4.3 Communication Skills: Students leaving CSUS communicate effectively both orally and in writing.

4.4 Quantitative Skills: Students leaving CSUS reason quantitatively.

4.5 Pre-professional Skills: Students leaving CSUS are prepared to assume career responsibilities, meet credential requirements, and gain admission to graduate or professional schools.

4.6 Retention and Graduation: CSUS retains students who have not yet achieved a credential or degree and graduates students seeking a degree

4.1 Critical Thinking Skills - Perhaps there is no outcome more fundamentalto the purpose of a university than a commitment to produce graduates who are critical thinkers. CSUS is no exception, and expects that Students leaving CSUS can think clearly and logically.

Critical Thinking Skills Indicators:

1. Drawing conclusions after weighting evidence, facts, and ideas

2. Developing problem-solving skills

3. Learning to think and reason

4. Thinking objectively about beliefs, attitudes, and values

5. Defining and solving problems

Graduates and alumni have largely favorable perceptions of the critical thinking skills developed during their years at CSUS. In short, students believed that the University is doing a good job of teaching in this area. Across the five perceptual indicators of thinking skills almost three out of four responses (73%) were positive. Interestingly, both the top-ranked indicator in this category and the bottom-ranked indicator dealt with "problem solving." Seventy-nine percent of the respondents to ACT Alumni Survey (ACT Alum) responded positively to the stimulus "defining and solving problems," By contrast, 68% of the students responding to the ACT Graduating Seniors Survey (ACT Grad) rated "developing problem-solving skills" as positive. The percentage of positive responses for the remaining three indicators fell between 71% and 74%.

To measure critical thinking skills of CSUS students objectively, 600 studentswere invited in Fall 1995 to take the Collegiate Assessment of Academic Proficiency (CAAP) test of Critical Thinking published by the American College Testing Program (ACT). This test measures the ability to clarify, analyze, evaluate, and extend an argument. Fifty-six percent (338) of the students actually completed the test. For the most part, CSUS students performed satisfactorily on the Critical Thinking test. It should be noted, however, that the national norms for the critical thinking test are based on sophomore level skills; students who completed the tests at CSUS were enrolled in courses typically taken by seniors. Because of the low numbers of students taking the test, caution must be exercised in drawing inferences from the results. The Critical Thinking test score for the CSUS sample was 63.6. For comparison purposes, the national mean was slightly less at 62.7. The standard deviation of the national sample was 5.4. This means that roughly two-thirds of the national sample scored between 57% and 68%.
The test mean for the 289 CSUS students whose primary language was English was 64 compared with a mean of 59 for the 40 ESL students. Students majoring in the health professions (n = 19) and home economics (n = 7) scored the best (mean = 66). Students majoring in liberal studies (n = 6) and engineering (n = 32) scored slightly below the mean.

Ethnically, only Asian students at CSUS scored substantially below the campus and national mean. The mean for Asian students was 58. However, Black, Filipino, and Native American students also scored below the CSUS mean, as did ESL students.

4.2 Information Skills - Preparation for participation in the "information age" is central to the purpose of any contemporary university. It is almost unthinkable that students could leave CSUS without a modicum of expertise in accessing, evaluating and applying information. Students leaving CSUS can locate information and evaluate that information critically.

Information Skills Indicators:

1. Locating, screening, and organizing information

2. Accessing and using a variety of information sources

3. Analyzing and drawing conclusions from various types of data

4. How well does the curriculum in your major provide you with research skills required by the discipline?

Perceptual indicators of information skills indicate that most students and graduates believed they received fairly good training in locating and critically examining information. This finding held true across three different surveys with three different populations: alumni, graduating seniors and continuing students. Respondents to the surveys were most complimentary of training in research skills. Seventy-nine percent of the respondents to the CASPER survey reported that the curriculum in their major provided them with the research skills demanded by their respective disciplines. The compliments extended to the full range of skills: locating, accessing, screening, organizing, analyzing, drawing conclusions, and using information.

4.3 Communication Skills - It is hard to find a profession that does not demand communication skills. This demand is particularly acute for those who rise to the top of their profession. Nor is communication skill limited to professional settings. Successful participation in society seems to demand written, oral, and interpersonal communication skills. The University's desire, therefore, is that Students leaving CSUS communicate effectively in both oral and written forms.

Communication Skills Indicators:

1. Improving my writing skills

2. Speaking more effectively

3. Listening to and understanding what others say

4. Learning to be adaptable, tolerant, and willing to negotiate

5. Recognizing and using effective verbal communication skills

6. Recognizing and using effective written communication skills

7. Working cooperatively in groups: working as a team member

8. How many classes taken last semester required you to write at least one paper?

9. How many classes taken last semester required a group presentation orproject?

10. How many classes taken last semester required you to make an individual presentation?

11. How well does the curriculum in your major provide you with communication skills required in the discipline?

The list of indicators for communication skills yielded the largest discrepancy between the top-rated and bottom-rated indicators. The variability among perceptual indicators was amazing. The top rated survey item received an 84% positive rating, compared to a 30% rating for the indicator at the bottom of the list. However, most responses by students and former students were positive. Four out of five respondents rated their CSUS education favorably when it came to providing verbal skills required by their respective disciplines.

One clear finding from the perceptual data was that writing is more commonly demanded by instructors than oral presentations. Seventy percent of the respondents to the CASPER survey reported that most of their courses require written work. Yet only 30% of the respondents to the same survey said that group presentations or individual presentations were a requirement for successful completion of their courses. Consistent with this finding, only 56% responded positively to a question regarding learning to speak more effectively. Students were confident in their CSUS training in interpersonal communication skills. The percentages of positive responses assigned to these indicators were "working in groups" (72% positive), "listening and understanding what others have to say" (64% positive), and learning to negotiate" (60% positive). According to alumni, CSUS offers its best communication education in the area of written communication. Eighty-four percent of the respondents to the ACT Alumni Survey reported a major or moderate impact on their lives for the writing skills they received at CSUS. On the ACT Graduating Senior survey, 65% reported progress toward this outcome as a result of their CSUS experience.

The references to written communication skills so far in this chapter have been fromself-report or perceptual measures. CSUS has its own measure of writing skills, the Writing Proficiency Exam (WPE). In order to graduate from CSUS, all CSUS students must pass the WPE with a score of eight or better. The WPE is to be taken as students begin the first semester of their junior year. Passage rates of the WPE are, therefore, a gauge of the collective writing ability of the upper class student body at any given time.
The graph below portrays the results of the WPE taken in March 1996. In the interest of fairness and reliability, students who first scored a "7" on the first administration are scored again by different faculty. On the second scoring the exam is assigned either a "6," failing or an "8," passing. The distribution of scores for the March exam was roughly normal, with most of the scores clustering one side or the other of the criterion for passing. Fifty-six percent of those taking the exam received a passing score.

As a comparative measure of writing instruction at CSUS, scores from the WPE work very well. The passage rate on the WPE for students taking English 1A at CSUS was 63%. The passage rate was higher only among those students who met their lower division English requirement through Advanced Placement or at one of the UC campuses. Considering the caliber of students who typically pass AP English or who begin their college careers at UC campuses, the superior passage rate of these students was not surprising.
When passage rates on the WPE are compared, other sources of lower division writing instruction compare favorably to English 1A at CSUS. The passage rate for students meeting their writing requirement at other CSU campuses was 57%. Students attending community colleges at the time they took English 1A had a 53% passage rate. Transfer students from outside the CSU, a UC, or from a community college system passed the WPE at a 61% rate.

Sixty-one percent of the Caucasian students passed the WPE in March. Students from other ethnic groups performed less successfully. Language, as expected, also predicted passage rates. Students who met their freshman composition requirements through various English as a second language (ESL) options passed at rates of 40% or lower.

A second objective measurement of writing skill was the Collegiate Assessment of Academic Proficiency (CAAP) test published by the American College Testing Program (ACT). CAAP tests were selected because they measured more than minimum competency in writing, and CSUS results could be compared with similar institutions. The Writing Test was designed to measure skills most commonly taught in college-level writing courses and required in most upper-division coursework. The test measures a student's skills in formulating and supporting assertions about a given issue and in organizing and connecting major ideas.

During Fall 1995, 600 students enrolled in advanced study courses were invited to take the CAAP tests. However, only 288 completed the writing test. The CSUS sample of students was comparable with national samples on all demographic elements except ethnicity. The mean score on the CAAP Writing Test taken by CSUS students was 3.2 (standard deviation = 0.7, range 1 to 4.75)--a mean score identical to the national sample. African American students (2.9), Filipino students (2.55), and Native American students (2.61) scored below the mean.

Mean scores of English as a Second Language (ESL) students on the CAAP Writing Test were lower than those of Native language speakers. The mean on the Writing Test for the 258 students whose primary language was English was 3.26 compared with the mean of 2.54 for the 28 ESL students who took the test. When comparing majors, students scoring above the mean of 3.2 were from general studies (3.35, N=7), education (3.45, N=24), and fine arts (3.32, N=10). The lowest scores were from students majoring in criminal justice and social work (3.08, N=24), computer science (3.05, N=5), engineering (2.94, N=13), and home economics (2.58, N=6). Yet another objective measure of writing skill was the California Basic Education Skills Test (CBEST) required of all students entering the primary and secondary teaching professions. Students must pass the CBEST prior to entering the Basic Teaching Credential Program. The CSUS passage rate for 8/95, 10/95 and 12/95 on the writing portion of the test was 73%. The passage rate for the entire State of California on the writing portion of the exam was 72%.

4.4 Quantitative Skills - Undeniably, the ability to think includes the ability to measure, to count, to compare--to calculate. Success in work and life is best assured if Students leaving CSUS can reason quantitatively.

Among the several surveys that measured student perceptions only one indicator could be found that measured student opinions of quantitative skills taught at CSUS. The ACT Graduating Seniors Survey asked respondents to rate "Understanding and applying math concepts and statistical reasoning." Respondents were asked to rate the progress they have made at CSUS toward attainment of the math and statistics outcome. The response categories for this question were "very much," "much," "moderate (average)," "little," or "none." Only 30% answered with "very much" or "much." CSUS students were not confident in the quantitative education they received. It would have been helpful had multiple indicators been available when drawing this conclusion. It is not know whether these students completed their mathematics and/or statistics courses at CSUS.

Unlike the WPE which is available to measure writing proficiency, CSUS has no test of mathematics proficiency prior to graduation. The campus does have a well- developed system for evaluating mathematics competence of incoming students. Students admitted to the University are required to have completed three years of mathematics in high school. The Entry Level Mathematics (ELM) test, administered on campus at the beginning of the student's first year, assesses student competence in mathematics at entrance. Some students need developmental work in mathematics before they can enroll in college level mathematics courses.

One objective measure of quantitative reasoning is available from the Graduate Record Exam (GRE), an examination required by many graduate programs. Unfortunately, CSUS students performed below national and state averages. The WASC steering committee reported that 73% of the students who took the GRE scored at or below the 50th percentile on the quantitative reasoning component.
Another objective measure of quantitative reasoning is the pass rates of students taking The California Basic Educational Skills Test (CBEST). During 1995 CSUS students on average scored below state averages on the math portion of the CBEST with 422 CSUS students passing at a rate of 75.4%. The statewide passage rate was 78.7%.

For the WASC Self-Study, a mathematics knowledge exam was included in The ACT-COS Graduating Seniors Survey. The exam was sent to all graduating students in Spring 1995. The ten question exam is reproduced in its entirety below. The mathematics assessment consisted of ten questions about mathematics knowledge. While 700 mathematics questionnaires were randomly included in the ACT-COS survey, only 56 (8%) were returned. The respondents answered an average of 4.5 questions correctly out of the 10 (SD 1.8). The scores closely approximate a normal distribution. The performance of the CSUS students was average, but there was only a very small number of respondents. Thus, drawing conclusions based on the sample must be approached cautiously.

Taken together, the one perceptual measure and several objective measures of CSUS quantitative student outcomes is not encouraging. Serious consideration should be given to better measurement of student performance in the quantitative area. Of course, the real issue is not measurement, it is student performance.

4.5 Pre-professional Skills - A university education serves a wide variety of functions. Most CSUS students expect degree programs to prepare them both academically and professionally, so that Students leaving CSUS are prepared to assume career responsibilities, meet credential requirements, and gain admission to graduate or professional schools.

Pre-professional Skills Indicators:

1. Developing effective job-seeking skills (e.g. interviewing, resumeconstruction)

2. Preparation for further study

3. Preparation for career

4. How closely related is our current job to the major/field in which youreceived your most recent degree/certificate/diploma?

5. How closely related was your first, full-timejob to the major/field in whichyou received your first degree/certificate/diploma from this school?

6. How well did your experiences at this school prepare your for yourcurrent job?

7. Preparation for further academic study

Among the many outcomes examined in this chapter, pre-professional skills appeared to be the weakest. The mean percentage of positive responses to the many perceptual indicators was only slightly more than half. Only about three in five of those responding to graduating senior and alumni surveys believed that CSUS had "prepared them well for a career." While one's job is often a matter of choice, only 59% of the Alumni said their major was directly related to their first or current jobs. While the professional glass is more than half full, an institution like CSUS should concentrate on filling the glass a little more, especially in providing assistance and support for students desirous of a career.

Even less satisfactory than professional preparation in the major was the University's seeming inability to provide graduates with job-seeking skills. Only 31% said that CSUS had contributed "very much" or "much" toward the outcome of "developing effective job-seeking skills" (e.g. interviewing, resume construction). The 31% positive response to this indicator was the lowest positive response among any of the indicators in any of the outcomes summarized in this chapter. Departments should explore ways of meeting the needs of students in developing job-seeking skills.

A higher scoring indicator in the area of pre-professional category was preparation for further study. Alumni reported 64% positive impressions of professional preparation. Graduating seniors registered a 62% positive figure.

Fortunately, when we are assessing professional preparation of CSUS students we are not limited to perceptual data. Four of the standardized tests currently being taken by CSUS students to meet credential requirements and to determine admission to graduate and profession schools were reviewed.

Graduate Study

The Graduate Record Examination (GRE) is a general test of verbal, quantitative, and analytical reasoning designed to assess undergraduate achievement or the qualifications of students for graduate study. The test does not and cannot measure all the factors important in predicting success in graduate study or in assessing undergraduate achievement. Although universities are discouraged from using the scores in aggregate form, a review of score ranges at CSUS indicated that:

  • half of the students scored at the 50th percentile or below on the verbal portion and half scored above;
  • 73 percent of the students scored below the 50th percentile on the quantitative reasoning component; and,
  • 56 percent scored at the 50th percentile or below on the analytical portion of the exam.

In 1997 the GRE will be adding new sections to the exam that will provide CSUS with more outcome assessment information. The University has not established an acceptable performance score to use in interpreting the GRE test scores for itself for comparison with other institutions.

Preparation for Teaching Career

Since 1983, prospective teachers have been required to take the California Basic Educational Skills Test (CBEST) to ensure an acceptable level of competence for credentialed K-12 teachers. Data were provided to CSUS for its students and others across the State of California, comparing scores by gender and ethnicity and those of seniors to those with the BA, those with more than the BA, and those with the MA degree.

The overall first-time pass rate for the State on the CBEST was 68% and forCSUS students was 64.7%. CSUS students met or exceeded the State rates in reading and writing, but not mathematics. Those with more than the BA exceeded the State and CSUS rates in reading, mathematics, and writing as did those with the MA. Black and Latino students scored below the State and CSUS rates in reading, mathematics, and writing. Other Latino students scored below the State and CSUS scores in reading. Males scored below the norm in writing while females scored below the norm in reading and mathematics.

Preparation for Medical School

In 1995 two test results were available for CSUS students seeking admission to medical schools. The Medical College Admission Test (MCAT) measures proficiency in verbal reasoning and writing, and competency in biological and physical sciences. Forty-two CSUS students completed the test with the following results:

On the MCAT the students from CSUS compared favorably with the national sample except for writing competency. In August 1995 CSUS students scored in the 50th percentile; in April of the same year, they scored in the 75th percentile.

Preparation for Law School

Similar to the MCAT for prospective medical students is the Law School Admission Test (LSAT). Scores on the LSAT are reported on a scale ranging from 120 to 180. Three sets of scores were reviewed for tests taken in December 1995, February 1996, and June 1966. On all three test dates applicants from CSUS scored, on average, below the 50 th percentile. It is not known, however, whether this result is due to poor preparation of students on the part of CSUS or whether the pool of applicants from numbers of students participating.

4.6 Retention and Graduation - The expectation of degree-seeking students who have been granted admission to CSUS is that within a reasonable period of time they will leave with a diploma. Individual circumstances change, but for the most part it should be the goal of CSUS faculty, staff and administration that CSUS retains students who have not yet achieved a credential or degree and graduates students seeking a degree.

Over the nine-year period from 1986 to 1994, the average one-year retention rate for new students has been about 77% for both freshman and transfer students. Over the same time period, the two-year retention rate dropped to 65% for former CSUS freshmen and to 70% for former transfer students. As the graph below illustrates, there has been a slight trend upward in first year retention rates.

But it is not enough just to retain students for a number of years; the ultimate goal is to graduate students. It appears that timely graduation, however, is simply not happening for far too many CSUS students, particularly if timely graduation is defined in the traditional sense of four years for an entering freshman and two years for a community college transfer student.

Only 16% of entering freshman graduate with the expected four-year degree. The percentage more than doubles to 34% after five years, but this is still only about a third of the entering freshman class. After six years the percentage starts to level off at 42%, reaching 48% after eight years.

Similarly, only 23% of transfer students graduate two years after transferring from a community college or other institution. The percentage takes a big jump to 42% after three years, but is still less than half of the population. After four years the percentage starts to level off at 51%, eventually climbing to 57% after six years.
There could be several possible explanations for the length of "time to degree" at CSUS. First this is a commuter campus. Many students cannot take a full load of classes along with the jobs they hold to pay for their education. Second, the number of units required to complete General Education requirements can be extensive for a student who transfers and does not plan well. In some cases the "time to degree" may be because some majors have large unit requirements. Whatever the reason, reducing length of "time to degree" should be examined further.


Concluding Thoughts

Regarding student outcomes at CSUS, it appears that CSUS performed well in training students to locate information and critically evaluate that information once found. CSUS performed very well in educating students to communicate, particularly written communication, but not as well in quantitative skills. Better measurement of quantitative outcomes is needed. Students want more assistance in developing specific career skills. Whether academicians like it or not, students see a university degree as the door to a career. For most students that career does not, however, commence after the traditional four years of higher education. For a variety of reasons a CSUS degree comes only after an extended number of years for most students.


Chapter 5
Learning Community at CSUS: Perceptual Measures of Learning Community

To the extent that we foster a community on campus, we further our goals
in all areas.

--CSUS Strategic Plan

There has been some confusion generated by the concept of "learning community" in varied contexts throughout the Self-Study. We regret the confusion, but affirm the importance of the concept as critical to our educational mission. Learning occurs amid the multiple relationships and activities of the university. The faculty-student classroom experience is not static or linear, unrelated to the complexity of the university experience. Therefore, the values and expectations, and indeed the relationships, of all the participants are worthy of self-examination when we are trying to assess student learning outcomes. Evidence of the importance of this concept can be found in our use of the term to describe our most innovative attempt at enhancing student learning outcomes--the Learning Community in our General Education undergraduate curriculum--where faculty collaboration is key. At the macro level the Learning Community includes all of us-- students, faculty, staff, and administrators.

This theme (represented in Chapter 5 of Phase I) sought to examine the values, expectations, and relationships of these major groups. In this reflective process we wanted to focus on what management consultants call "organizational health" with a primary focus on the strengths and weaknesses of our communication processes. We feel that organizational and cultural barriers to faculty collegiality, for example, or past histories and misunderstandings between different groups, and the challenges of communication in a diverse community are all worthy of our attention. We want to understand those problems so that, in strengthening our community, we can strengthen the learning outcomes of our students.

Our search for methodologies/measurements to assess the Learning Community at CSUS did not produce comprehensive tools that had been tested and validated in other settings. We did include the 1993 campus climate survey which had sought to better understand miscommunication based on racial and ethnic differences within our student community. Although that survey was not comprehensive, it represented the best "handle" we could initially develop on the "status" of our learning community from the student perspective.

The interest in communication in a diverse learning community resulted in a decision to consider staff and faculty attitudes and experiences along the same dimensions as the student climate survey. Assessment tools originally developed by others were adapted by the campus task force in cooperation with the Academic Senate's Committee on Diversity (CODE). During that process, the campus instruments became too long (staff) and disjointed (faculty) which contributed to an unsatisfactory response rate. As the results were tallied, we became conscious of those problems and hoped that the focus group data would add depth to our initial data. It is unclear that that happened.

We are not satisfied with the totality of our examination of our Learning Community. We have added data in Phase II which draws together extensive student outcome data, not previously organized in Phase I but previously collected, which demonstrates a set of relationships which might properly be set as a baseline for student perception of the Learning Community. We will continue to work on the development of valid instruments for assessing the nature of our relationships as members of our Learning Community seeking to develop a similar baseline. The issues identified for future examination in Chapter 5, Phase I provide the specific directions for that work. Data on Student Outcomes in Phase II is organized around 10 desired outcomes.

Desired Student Outcomes

5.1 Cultural Diversity: Students leaving CSUS appreciate cultural diversity.

5.2 Intellectual Tolerance: Students leaving CSUS have been intellectually stimulated, but not harassed or coerced by, contradictory political, moral, or spiritual beliefs.

5.3 Racial Integration: Students interact with students of other racial/ethnic groups in a variety of settings.

5.4 Non-Discrimination: Students are judged by character, effort, and performance, rather than by gender, ethnicity, or other personal characteristics.

5.5 Sense of Belonging: Students have a sense of belonging to the CSUS campus community.

5.6 Access to Faculty: Students have ample access to faculty.

5.7 Administrative Support: Faculty, staff, and administration handle the administrative details of acquiring an education conveniently, pleasantly and in a helpful manner.

5.8 Career Assistance: CSUS assists students in beginning careers.

5.9 Physical Resources: CSUS provides adequate physical facilities and resources for learning.

5.10 Advising: CSUS provides helpful advising services.


5.1 Cultural Diversity - A GE Assessment Subcommittee was appointed inFall 1993 to determine whether the GE Race and Ethnicity courses were meeting the objective of accomplishing "A significant and useful understanding of the perspectives and contributions to human activities and experiences of peoples from a diversity of cultures and backgrounds, including the contributions and perspectives of Non-Western cultures, and of women and ethnic and other minority groups who have been the objects of prejudice and adverse discrimination within our society." Generalizing this objective from the classroom to the entire process of university education the outcome can be summarized as: Students leaving CSUS appreciate cultural diversity.

Cultural Diversity Indicators:

1. Broaden my awareness of diversity among people, their values and cultures

2. Becoming a more effective member in a multi-cultural society

3. My classes regularly exposed me to the contributions of a variety of groups or peoples

4. Understanding and appreciating cultural and ethnic differences betweenpeople

5. Much cultural/ethnic diversity in student body

6. Multicultural content of courses

7. My classes regularly exposed me to the contributions of a variety of groups or peoples

8. How many classes taken last semester exposed you to the contributions of a variety of racial or cultural groups?

Responses to the cultural diversity questions were generally positive. The mean percentage of positive responses was 55.9%. Leading the list was the indicator of "understanding and appreciating cultural and ethnic differences between people." Almost three out of four respondents to the alumni survey believed that efforts to help students understand and appreciate cultural diversity had a "major" or "moderate" impact on their experiences at CSUS.

Roughly half of the respondents to both the Graduating Senior (ACT Grad) and General Education (Gen Ed) Surveys claimed that classes almost always or frequently exposed them to the contributions of a variety of groups or peoples. Alumni responded about the same in response to a similar question about the multicultural content of courses. Slightly fewer (42%) perceived that courses had made a great contribution to becoming "an effective member in a multi-cultural society."

In Fall 1994, 1,701 students enrolled in "Race and Ethnicity in American Society" courses were surveyed at the end of the semester to determine whether they perceived that the course they were enrolled in met the objectives, which were to develop ". . .a significant and useful understanding of the perspectives and contributions to human activities and experiences of people from a diversity of cultures and backgrounds..."

Overall, the majority of students indicated that the Race and Ethnicity courses were meeting the intended General Education objectives. The proportion of respondents choosing "agree" or "strongly agree" on each question varied from a high of 82 percent on "Having positive attitudes toward other ethnic groups" to a low of 57 percent on "Interacting frequently with other ethnic groups as a result of the course." The average was a 73 percent positive response across all survey questions.

5.2 Intellectual Tolerance - Students' intellectual growth is best accomplished in an environment free of confrontation: through rational discussion they can examine their own and others' beliefs, appreciate diversity, and learn tolerance. Students leaving CSUS have been intellectually stimulated, but not harassed or coerced by contradictory political, moral, or spiritual beliefs.

Intellectual Tolerance Indicators:

1. Freedom from harassment on campus

2. Campus atmosphere of ethnic, political, and religious understanding

3. I felt free to disagree with professors

4. Getting along with people whose attitudes and opinions are different from mine

5. Overall, there was a campus atmosphere of ethnic, political, and religious understanding and acceptance

6. There was free exchange of ideas and points of view between professors and students in my classes

7. There was free exchange of ideas and points of view between professors and students in my classes

8. Professors were responsive to student questions and concerns

9. I felt free to disagree with professors

The scores from the nine indicators of intellectual tolerance exhibited a rather large variability ranging from almost four out of five positive responses to one question, down to one-third positive responses to another question. Seventy-nine percent of the General Education students affirmed that professors were "almost always" or at least "frequently" responsive to student questions and concerns.

A less common observation was that students felt free to disagree with professors. Less than half of the respondents (46% of the graduating seniors and 43% of the General Education students) reported such tolerance was frequently the case. The least impressive indicator of intellectual tolerance was freedom from harassment. Disturbingly, only 33% of the graduating seniors were satisfied that the campus was reasonably free from harassment. Since harassment was not defined, we are left to speculate on the type of harassment.

Nevertheless, better than half of the respondents to a variety of surveys found CSUS to be supportive of a free exchange of ideas and supportive of "getting along with people whose attitudes and opinions are different..."

5.3 Racial Integration - It is one thing to appreciate cultural diversity or to be tolerant of alternative points of view. It is quite another to seek out members of other racial or ethnic groups or to actively engage in discussions with those holding contrary beliefs. Yet a university is precisely the setting where this admirable outcome can be pursued: Students interact with students of other racial/ethnic groups in a variety of settings.

Racial Integration Indicators:

1. Interacting well with people from cultures other than my own

2. Getting along with people from various cultures, races, backgrounds, etc.

3. Racial harmony at this college

Evidence from three indicators drawn from three separate surveys supports the conclusion that students believe they get along well with members of other cultural groups, but they are less successful at integrating others into their daily lives. Seventy-four percent reported a major or moderate impact on their lives of attempts at "getting along with people from various cultures, races, backgrounds, etc." Students do not, however, appear to seek out people from other racial or ethnic groups. Only 48% claimed to have great success at interacting with people from other cultures.

5.4 Non-Discrimination - If the citizenry is to achieve a society without discrimination against minority groups, it will likely be due to influences from institutions such as public universities. But if CSUS is to produce leaders it must first rid itself of discrimination. A desirable outcome, therefore, is that Students are judged by their character, effort and performance, rather than by their gender, ethnicity, or other personal characteristics.

Non-Discrimination Indicators:

1. This college is equally supportive of women and men

2. This college is equally supportive of all racial/ethnic groups

3. Campus acceptance of individuals regardless of their sexual orientation

4. The campus was, generally, free from harassment (e.g. sexual, racial, etc.)

5. This University is equally supportive of women and men

6. This University is equally supportive of all racial/ethnic groups

According to survey results, students believe that the campus is more accepting of some forms of equality than others. There appears to be a hierarchy of non-discrimination, with gender equality at the top of the hierarchy and sexual orientation at the bottom. Students believed the campus community was most supportive of equality between the sexes. Sixty-four percent of the graduating senior sample strongly agreed or agreed with the statement that "this college is equally supportive of women and men." Similarly, three-fifths of the respondents to the SNAPS survey believed that CSUS was equally supportive of women and men.

Students believed CSUS was less supportive of racial equality. Fifty-five percent of the respondents to the SNAPS survey and 54% of the respondents to the graduating seniors survey agreed with the statement that "this university is equally supportive of all racial/ethnic groups." Equality appeared to be least common for differences between sexual orientation. Only one-third of the alumni characterized the CSUS campus as accepting of individuals regardless of sexual orientation.

5.5 Sense of Belonging - CSUS is different from many Universities of its size and stature in that it is largely a commuter campus. Students working off-campus, part or full time, are trying to balance their work time with a part or full time academic load. The challenge for CSUS then becomes one of ensuring that Students have a sense of belonging to the CSUS campus community.

Sense of Belonging Indicators:

1. My sense of belonging on this campus

2. College social activities

3. Opportunities for involvement in campus activities

4. Strong sense of individual belonging on this campus

5. Many opportunities for student involvement in campus activities

6. Concern for me as an individual

7. Recreation programs and/or activities

8. Social and cultural activities

9. Opportunities for personal involvement in campus activities

10. I felt valued as a member of my classes

11. I participated in and/or attended out-of-classroom activities (speakers,presentations, student groups, clubs, etc.)

12. Participation in out-of-class activities contributed to my growth and development

13. How often did you participate in or attend campus activities, other than classes, last semester?

14. Do you feel adequately informed about campus events and activities?

15. How many times last semester did you meet with other students in campus meeting areas such as the pub, library, lounges, etc.?

16. Do the extra-curricular events and activities provided by the University meet your needs?


The pattern that emerged from responses to this set of indicators was entirely consistent with the commuter demographics of CSUS. The majority of students did not feel a sense of belonging to this campus. The mean percentage of positive responses to this set of indicators was 39.8%. This was the second lowest mean among the ten outcomes summarized in this chapter.

Students simply do not participate with much regularity in campus activities. Only 22% of the respondents to the GE survey said they frequently participate in out-of-class activities. Only 31% said they participate in campus activities at least once a week (CASPER phone questions). Furthermore, only 30% said that campus activities contributed to their personal growth and development. The highest percentage of activity was registered in response to a question about meeting other students in campus areas such as The Pub, Library, student lounges, etc. (46%). But for many students, the likely referent in this question may have been academic study groups or course project groups.

Nor do most students feel that CSUS affords them enough opportunity to participate in activities. Three separate surveys found the percentage of students who are satisfied with the availability of activities to be in the low forties.

The lack of extra-curricular activity and the perception that CSUS affords little opportunity for such activity may be the result of the individual circumstances in which commuter students find themselves. Students with a full course load, a full work load and perhaps a full family load may have little time or inclination to search out and avail themselves of campus activities. Indeed, 68% of the respondents to a CASPER survey reported that the extra-curricular events and activities provided by the University do meet their needs. It may be the case that many students do not perceive a need for such activities.

Whether the need for campus activities is felt or not, the effects of such a lack of activity are acutely felt. Only 45% of the respondents to the ACT Graduating Seniors Survey expressed a sense of belonging on the campus. The sense of belonging dropped even further when the graduates had left the campus for a while and became alumni. Only 29% of the alumni sample expressed a strong sense of belonging on the campus.

Even more problematic may be the fact that the lack of connection could translate to feelings about one's self. The General Education Survey found that only 40% of the respondents "felt valued as a member of my classes." Only 56% of the alumni believed that CSUS had been concerned for them as individuals. Certainly there are many causes for feeling disconnected or not belonging, but more frequent participation in activities on campus might help the situation.

If the University decides to confront this issue, the difficult task is how to intervene in what may be a vicious cycle. If students don't have time to participate, or don't participate, or see few opportunities to participate, or do not value the opportunities that are available, they feel isolated from the campus, so they participate even less. And the cycle continues. Indirect support of this cycle comes from the fact that few students positively evaluated the existing activities (30% on SNAPS and 37% on the ACT Graduating Seniors Survey). Although the answers to the problem are not apparent, the problem is real.

5.6 Access to Faculty - One of the defining characteristics of an excellent teaching institution is the ease with which students can access faculty for help with coursework, advising, and assistance with campus activities such as independent research projects, internships and career preparation. The desired outcome for students is that... Students have ample access to faculty.

Access to Faculty Indicators:

1. Availability for faculty for office appointments

2. Informal contact with faculty in non-academic settings

3. I felt comfortable talking to my professors out of class

4. My professors knew my name

5. Many opportunities for student/faculty interaction

6. Most faculty were readily available to students outside of class time

7. Accessibility of faculty

8. Opportunity to meet with faculty outside of the classroom

9. Out-of-class availability of your instructors

10. Major faculty, staff and administration were willing to help me when I had problems

11. I felt comfortable talking to my professors out of class

12. My professors knew my name

13. How many times last semester did you meet with faculty in campus meeting areas such as the pub, library, lounges, etc.?


Although exposure to faculty may not be as frequent as desired, students indicated that faculty were available to them. Sixty-five percent of the respondents to the SNAPS survey rated faculty accessibility as good to excellent. Sixty-six percent of the graduating seniors were satisfied with faculty availability for office appointments. When alumni were asked a similar question, 74% agreed that "most faculty were readily available to students outside of class time." Even 58% of the non-returning students were satisfied with the availability of faculty.

Less frequent, however, was informal contact with faculty in non-academic settings. Only 35% of the graduating seniors viewed the level of informal contact as satisfying. A CASPER survey found that only 26% of the students had met with faculty the previous semester outside of classrooms or faculty offices in areas such as the library or lounges. Exposure to faculty outside of the classroom was rated as excellent or good by only 21% of the students polled.

Despite the limited informal contact with faculty, 74% of the graduating seniors believed that faculty frequently knew their names. Such familiarity was less common for students enrolled in General Education courses (46%). The net result of exposure to faculty was summed up in the alumni survey when, on a 1 to 5 scale, 61% rated student faculty interaction in the 4 or 5 range.

5.7 Administrative /Student Support - The backdrop for faculty interaction with students are the many services provided by CSUS staff and administrators. The typical referent most people have of a college education is a professor standing in front of a chalk board or a table full of lab equipment and a classroom full of students. But that scene simply does not take place without a lot of planning and effort by a wide range of key people who provide the administrative details that make the learning experience possible. Faculty, staff and administration handle the administrative details of acquiring an education conveniently, pleasantly and in a helpful manner.

Administrative/Student Support Indicators:

1. New student orientation services

2. Financial aid services

3. Transfer of course credits from other colleges to this college

4. I was informed when classes were to be canceled

5. High cost for attendance

6. The financial aid available to me was adequate for my needs

7. I encountered few course scheduling or course availability problems

8. Registration procedures

9. Orientation to and instruction in use of campus computer system

10. Financial aid counseling and related services

11. Publications: catalog and schedule of classes

12. Bookstore

13. Financial aid services

14. Campus orientation programs

15. Admission services

16. General registration procedures

17. Availability of the courses you want at times you can take them

18. How accurately does the printed schedule of classes reflect what actually happens when the semester begins?

The Administrative/Student Support Indicators evidenced a wide range in the extent to which students were satisfied with such services. On the positive side, students gave positive marks to publications (71%), the bookstore (66%), and the efforts made to inform them when classes were canceled (74%).

Services students need to enroll and get started at CSUS were evaluated slightly less positively. The SNAPS survey found that only 48% of the students rated admission services as excellent or good. The process of transferring credits was seen in a more positive light with 64% expressing satisfaction. Four indicators relating to the process of registration (printed schedules, registration procedures and scheduling problems) had a mean positive rating of 56%. A related indicator of course availability scored only 43% in the positive column.

A problem area for students coming to CSUS seemed to be campus orientation. Graduating seniors gave campus orientation a 46% favorable rating, SNAPS gave it a 41% positive rating, but only 21% of alumni recalled the experience as satisfying.

But the real administrative headache for students seemed to be the issue of financial aid. Regardless of the survey or the way the question was worded, three-fourths of the survey respondents failed to rate financial aid positively. The performance of financial aid was consistently rated negatively. It was one of the most negative response to any of the indicators measuring any of the outcomes in this chapter. Only 16 percent of the alumni rated "financial aid, counseling and related services" as satisfying.

But the lowest evaluation was directed at remedial and tutorial services. Only 22% of the graduating seniors assigned a satisfactory rating to these services.

5.8 Career Assistance - Despite the protests of traditional liberal education proponents, many students focus their education on a subsequent career. If CSUS is to honor student wishes it must look toward a career outcome: CSUS assists students in beginning careers.

Career Assistance Indicators:

1. Learning to formulate and re-shape my lifetime goals

2. Learning about career options

3. Career planning services

4. Practical work experience offered in areas related to my major

5. Job placement services (e.g. opportunities to link with employers)

6. Learning about existing and emerging career options

7. Career planning and placement services

8. Career planning provided by faculty

9. Career planning office

10. Career planning services

11. Job placement services


CSUS did not perform well on the indicators measuring the outcome of career assistance. The only indicator with a percentage of positive responses above 50% was "learning to formulate and re-shape my lifetime goals." The specific indicators of "career planning services" ranged from a high of 40% positive to a low of 27%. Two references to a specific office scored 25% and 21% for "career planning office" and "job placement services," respectively.

Achieving only slightly better rating were two indicators referring to "career options." Graduating seniors reported a 34% positive response to the indicator "learning about career options." A similarly worded item from the alumni survey yielded a 42% positive response. CSUS should engage itself in a discussion about career assistance. If CSUS believes it is providing adequate career assistance there is a gap between the quality of the service we are providing and the level of service students think they are getting.

5.9 Facilities and Services - Large pieces (literally) of the campus picture are the physical structures on campus and the physical pieces of apparatus that go inside those structures. CSUS provides adequate physical facilities and resources for learning.

Facilities and Services Indicators:

1. Student access to computer facilities and services

2. Developmental remedial, and tutorial services, including writing labs, math labs

3. Extensive computer system services, equipment, labs, etc.

4. General condition of buildings and grounds

5. Computer workstations

6. Classroom and laboratory facilities

7. Parking facilities and services

8. How satisfied are you with laboratory facilities in your major

9. Are campus meeting areas adequate to meet your needs?

Student evaluation of the physical resources tended to fall into two categories: (1) the physical structures on campus and (2) the learning "hardware" that occupies those buildings and grounds. Respondents to the various surveys tended to evaluate the adequacy of the structures more positively than the availability and quality of the "hardware." CASPER surveys found that students were very happy with the campus meeting areas (81% positive). Two-thirds of the alumni evaluated the buildings and grounds positively. The exception to positive evaluations of physical facilities was the reaction of non- continuing students to parking. Only 32% of this group believed parking to be adequate.

Respondents to CASPER phone survey questions positively responded to "laboratory facilities in your major." Seventy-three percent of the students reported they were either very satisfied or satisfied. The percentage of positive evaluations dropped when the sample was non-returning students, however. Among this group only 57% reported that they were satisfied with the classroom and laboratory facilities.

A decline in student enthusiasm arose when the subject changed to computer facilities, services and workstations, however. The percentage of positive ratings in this category dropped to 46% among graduating seniors, 46% among the SNAPS sample and 35% among alumni.

5.10 Advising
- A final desired outcome discussed under the general heading of learning community is student perception of advising: CSUS provides helpful advising services.

Advising Indicators:

1. Quality of academic advising

2. New student placement in reading/writing, math courses

3. Academic advising

4. The University advising center or general studies office

5. Advising centers in my major department or school/college

6. Faculty in my major department

7. Administrative or program staff (e.g., EOP, Adult, Re-Entry, Disabled Student Services, financial aid office)

8. Campus catalog or other department or school publications

9. Orientation and program preparation seminars

10. Academic advising services

11. College orientation program

12. Availability of your adviser

13. How satisfied are you with academic advising in your major?

14. How satisfied are you with peer or faculty mentors in your major?

15. How satisfied are you with academic advising in your major?

16. Major advisor was accurate and helpful

Considering the fact that advising is a relatively narrow topic the variability of the scores for the different indicators was surprising. The percentages of positive responses ranged from a high of 79% (peer or faculty mentors) to a low of 16% (student placement). A clear pattern was present, however. The indicators referencing the respondent's major department tended to be very positive, whereas the indicators referencing advising outside the student's major tended to be less positive. When CASPER users were asked, "how satisfied are you with academic advising in your major?", 77% responded that they were very satisfied or satisfied. By contrast, when the SNAPS survey asked students to rate the "University advising center or general studies office," only 31% offered ratings of "good" or "excellent."

One indicator of advising made no specific reference to either the major or the rest of the university. This item was present on the ACT Non-returning student survey. It asked students to rate the availability of advisors. To this question only 41% said they were satisfied with access to advising. The mean percentage of all responses to the advising indicators was 49.0%.

Concluding Thoughts

Much of this chapter dealt issues of tolerance, diversity, and discrimination. Students were relatively positive about the issue of diversity, particularly in the curriculum. Students tended to be mildly positive in their perceptions of several other learning community outcomes (access to faculty, physical resources, and advising). But students perceived several areas where the campus may want to devote some attention: some areas included under administrative support may need further analysis (particularly financial aid and advising); many students seem to want but did not feel a strong sense of belonging to the campus; and most want more attention paid to career assistance.


Chapter 6

The Ripple Effect: From WASC Self-Study to an
Institutional Commitment to a Culture of Evidence


The purpose of this chapter is to provide a more comprehensive report on the many consequences of the University's initial foray into assessment, prompted initially by WASC. It is illustrative of the University's commitment to use assessment in building a culture of evidence. Of necessity, this description of the multiple ways in which the campus is collecting evidence, and using it to make critical decisions, also documents how the governance and process portion of the infrastructure work. We believe that the infrastructure we are creating will ultimately sustain the changes set in motion by WASC. The chapter begins by providing a description of the overall CSUS institutional commitment to assessment followed by a thorough description of the critical link connecting assessment, planning, and budget at CSUS. It concludes with a detailed description of the many consequences and changes already underway as a result of our commitment to using evidence to improve our effectiveness as an institution of higher education.


The Institutional Commitment: University Assessment Model

The WASC Self-Study commitment to assessment has stimulated an institutional commitment in using evidence to improve institutional quality and effectiveness. The transition from WASC to an institutional commitment is represented by a three dimensional cube (see Phase I University Assessment Model Figure I) that illustrates the University's long term assessment plan. One dimension of the cube represents Initiatives, which are the external and internal forces that call for assessment. [For a complete description of the external and internal stimuli for our assessment efforts (University Assessment Initiatives) see page 72-77 of Self-Study Phase I.] The second dimension represents Assessment Resources, which are the ongoing activities used to gather evidence about institutional and program outcomes. For a complete description of the Assessment Activities and Instruments see pages 77-82 of the Self-Study Phase I. The third face of the cube represents the goals of the institution as represented in the University's Strategic Plan Themes. The Strategic Plan is comprised of eight goals, articulated in a series of theme papers. The critical feature of this conceptual presentation of the long term University Assessment Model is that it demonstrates the University's commitment to assess its progress in achieving the goals of the Strategic Plan using a variety of assessment activities and multiple methods at every level of the University.

The University Assessment Model grew directly out of the WASC Self-Study model. The WASC Self-Study focuses on three themes: Teaching and Learning, Student Outcomes, and The Learning Community. The University's long-term model, an extension of the WASC model, quite naturally replaces the three WASC Self-Study themes with the eight themes of the University Strategic Plan. Otherwise, the model is the same: a variety of initiatives involve the University in assessment, and the instruments and activities provide data to assess our progress in meeting the goals of the Strategic Plan. Not every assessment initiative is expected to relate to all eight themes or to require data from all available instruments. For example, professional accreditation assessment activities would likely focus on just three Strategic Plan themes: teaching and learning, academic programs, and scholarship. Instruments and institutional data designed to provide evidence related to the themes of capital campus and public life might not be useful to a professional accreditation review. Similarly, the NCAA accreditation activity would not be addressing the scholarship theme, but would focus primarily on issues of teaching and learning and enrollment planning.

The predominant assessment initiative that would, by definition, address the goals of the Strategic Plan and its themes, and use all available data and instruments, is the assessment of the Strategic Plan itself. The eight themes, collectively, describe the issues of fundamental significance to the future of the University. A complete assessment of progress in achieving the goals of the Strategic Plan is a major undertaking to which the University is fully committed. We are in the second year of Strategic Plan assessment. This commitment has redefined the role of the Council for University Planning (CUP) and has forged a strong link among assessment, planning, and budget--certainly the goal of every planning process. The next section describes the process used to assess progress toward accomplishing Strategic Plan goals and the link we have established among assessment, planning, and budget as part of that effort. Following that, we present substantial evidence that the University has acted on the basis of the assessment findings as it strives to implement its Strategic Plan.



Link Among Assessment, Planning, and Budget

A common pitfall of university planning is the failure to connect planning and budgeting. When this happens, the budget becomes the de facto plan and the official "Plan" becomes irrelevant. Since 1991 a document called "Resource Priorities" has served as the primary linkage between planning and budget at CSUS. Each year in December, the Council for University Planning (CUP) recommends a set of Resource Priorities to the President for the following year's budget. Once approved by the President, the Resource Priorities are used by the Vice President for Administration in drafting the University budget. In CUP's annual review of the University budget, the Vice President for Administration is asked to explain how the proposed budget responds to the University's Resource Priorities.

For the past two years, the link between planning and budgeting provided by the University's Resource Priorities has been strengthened appreciably, first, by the enactment of the Strategic Plan and second, by the adoption of the Assessment Model described above. The Strategic Plan provided a framework for the Resource Priorities that had been lacking. Beginning with planning for the 1995-96 budget, the Resource Priorities were developed with substantive references to the themes of the Strategic Plan. This process ensures that there is a broad consensus behind the priorities and that the various priorities are part of a coherent plan to address the University's mission and goals. A second, even more critical change, the incorporation of assessment into the CUP planning process, guarantees that we are directing resources toward those areas most in need of attention. This came directly out of our participation in the innovative, evidence-based Self-Study for WASC and the adoption of the University Assessment Model. The model has dramatically changed, and improved, the role of CUP in planning and establishing Resource Priorities for the 1996-97 budget.

This critical new link among assessment, planning, and budget is carried out through the preparation of theme assessment papers for each of the Strategic Plan Themes. These papers use all relevant institutional and assessment data to create a snapshot of where the University is in relation to each theme. The goal and explanatory text in the Strategic Plan are the source for identifying the "key concepts" which provide the structure for assessment reports. A review of theme assessment reports now constitutes the primary agenda for the Council for University Planning in the Fall semester, as it develops Resource Priorities for the following year's budget.

During the 1995-96 academic year theme assessment reports on Teaching and Learning, Academic Programs, Campus Life, and Enrollment Planning were reviewed by CUP in the Fall semester in planning for the 1996-97 budget. These four were chosen in part because they were perceived by CUP as priorities for the University and, in part, because they were the themes with the most available outcomes data. Assessment reports on the other four themes (Public Life, Capital Campus, Pluralism, and Scholarship) were reviewed in Spring 1996. The first four assessment papers had a profound impact on the Resource Priorities established by CUP. In fact, seven of the eight Resource Priorities for 1996-97 were derived from, or influenced by, the assessment papers. In this way, a critical link among planning, assessment, and resource allocation was established.

As with any new planning process, ours is iterative, with changes made as we go along to improve the linkages and the results. The second time around (during Fall 1996) an important change was made in the way the Council considers the theme assessment papers. Rather than have Council staff present the papers, members of the Council, selected from the faculty, students, and staff on the Council, review the papers in advance and report to the Council. This change fostered a much richer discussion of various interpretations of the assessment data than could be generated by a staff presentation. Members are asked to review the papers, answering the following four questions:

1. What, if any, priority areas for next year's University budget allocations does this information suggest?

2. Does it raise any issues or concerns that CUP might refer to other campus groups or offices for attention?

3. Does the report suggest any "action steps" that CUP should consider for recommendation to the President?

4. Is the reader prevented from drawing reasonable inferences about how the University is performing on these aspects of the Strategic Plan because of missing or unavailable data?


Responses to the questions, and subsequent discussions, have helped CUP integrateassessment, planning, and budgeting at CSUS. A discussion of the process, and how it actually worked, follows:

Resource Priorities. Five papers were reviewed during Fall 1996. The review heavily influenced the Resource Priorities--all six of which stemmed from discussions of the theme assessment papers. One Resource Priority in particular demonstrates the commitment that the Council has to the assessment enterprise:
"Continue to build expertise in and support for outcomes assessment for academic and support programs for purposes of improving University programs and services." Referrals to other groups. In addition to influencing the Resource Priorities, the Council's Fall 1996 review of theme assessment papers resulted in written recommendations to the Director of the Center for Teaching and Learning, and the Chairs of the Academic Senate Curriculum, Faculty Policies, General Education, and Academic Policies Committees. In each case, a memorandum was prepared referring to the data that led the Council to identify the issue as one warranting the attention of the University's various governance bodies with a request for a response. By referring planning issues to appropriate governance bodies across the campus, CUP has strengthened the degree to which assessment informs the decision-making processes of the University.

Recommended action steps. A separate list of recommendations were sent to the President addressing issues where administrative action might be taken to implement aspects of the Strategic Plan. This complements another new task for the Council that will result in establishing "action steps" each year to implement the Strategic Plan. During Spring semester the heads of the University's major operating units (called "program centers") will be asked to propose 3-5 actions they will take to implement aspects of the Strategic Plan. CUP will analyze the proposals in the context of the assessment information it has reviewed and recommend to the President action steps based on University priorities. The President and his staff will make the final decision on the action steps for next year.

Development of new measures. For some themes, extensive data are available that allow the Council to come to an informed judgment about the University's progress and condition. For other themes like Public Life the University has little assessment data available. For still other themes like Scholarship there may be data but disagreement about how to use or interpret the data. The Council's review of theme papers during Fall 1996 resulted in a list of areas where data collection needs to be improved. During Spring 1997 the Council will review a plan to expand available outcome measures for the Public Life theme. In addition, staff from the Office of Institutional Studies will meet with an ad hoc faculty group to develop assessment measures for the Scholarship theme. Resource constraints, of course, limit the extent to which outcome measures can be developed, but a process for identifying how measures can be improved using existing resources is in place. While measures can lead to objective findings, evaluation of findings requires judgment, and judgment requires standards. In many cases measures are available, but standards for interpreting the measures are not. The Council is also involved in the evolution of standards to improve its use of assessment information.

The Council's engagement with assessment data has begun to provide valuable guidance regarding the future direction of the University's Strategic Plan. The relationship between the themes and assessment is a dynamic interaction in which the themes shape the University's assessment activities, and the assessment reports reshape the themes. Specifically, based on CUP's work since Fall 1995, a planning committee of the Council is drafting a new theme (titled "Institutional Effectiveness") and substantively revising three others. One of the revisions to the Campus Life theme, for example, is the inclusion of a strategy to build upon students' reported positive identification with their academic disciplines by urging that disciplines find ways to serve as "homes" to make the University experience more satisfying for students. The evolution of the Strategic Plan itself is perhaps the most profound evidence of the University's commitment to reshape itself based on self- assessment.

The linkage we have forged among assessment, planning, and budgeting extends beyond the CUP process. For example, program centers are asked to provide information on outcomes assessment as part of their annual budget submission. Specifically, they are asked what directions they have taken in relation to the Strategic Plan, how they measure their success, and to provide outcome measures for two years. As a sign of the full extent to which outcomes assessment has been embraced, consider the following recent recommendation of the Council to strengthen the degree to which program centers must justify their budget requests with outcomes data:

The Council recommends that program centers requesting funds to address any of the approved Resource Priorities be required to describe the intended outcomes of the expenditure and, later, to provide an assessment of the actual outcomes of the expenditure. The Council further recommends that the Vice President for Administration present information to the Council on (a) allocations made in relation to the resource priorities, (b) the intended outcomes of the allocations, as proposed by the program center receiving the allocation, and (c) any actual outcomes data that may be available. This information should be presented in the Fall, when the Council begins developing resource priorities each year, and in the Spring, when the Vice President presents the recommended University budget. In this way, subsequent resource priorities will be based both on evidence of progress toward strategic plan goals (from theme assessment papers) and specific evidence of the impact of various funding strategies.

The University plans to implement this recommendation in future budget cycles.

The process of applying for funds from the Lottery has been recently changed to strengthen the link among assessment, planning, and budget. Applicants must submit proposals that relate to a select number of categories derived from the Strategic Plan. Moreover, they must identify projected outcomes for the proposed project and later must submit a final report that includes outcomes data relating back to the projected outcomes cited in the original proposal. Failure to submit the appropriate assessment report will make applicants ineligible for future funding from the Lottery.

Another example of strengthened linkages among assessment, planning, and budget is the new Pedagogy Enhancement Awards Program. Formerly the Faculty Mini-Grant program, the goal of the grants program has been realigned with the Strategic Plan emphasis on teaching, learning, and technology. The guidelines have been strengthened with respect to the identification of project goals, the outcomes of the project, and the requirements for a project final report as a condition of future eligibility. Similarly across the campus, the link among project goal setting, outcomes reporting, and funding has become well-established.

The goal of the planning process at CSUS is to integrate planning, assessment, and resource allocation. Clearly we have made great strides in the last two years with the introduction, first, of the Strategic Plan, and second, of assessment designed to create a "culture of evidence". Our processes for making the best use of assessment data continue to evolve and improve. However, assessment at CSUS is in its infancy, and strategic planning is only slightly more advanced. Much work and discussion are needed to determine how to improve planning and how best to use assessment data in the planning process. The next section describes some of the actions the University has taken in its concerted effort to respond to assessment findings.


Using Evidence to Build Institutional Effectiveness:
Actions Taken in Response to Assessment Findings

During the last two and a half years, CSUS has systematically collected assessment data for purposes of the WASC Self-Study and to support its new institutional assessment efforts. Through the integration of assessment, planning, and budget described above, we have been able to respond to much of the evidence with new or modified University initiatives. This section lists (a) issues that have arisen from a review of various assessment information and (b) initiatives taken to address the evidence. It is organized according to the themes of the University's Strategic Plan. We believe the following pages illustrate the University's commitment to build and respond to a "culture of evidence." Clearly more attention has been given to the themes of Teaching and Learning and Campus Life in view of their priority within the University.


Teaching and Learning Theme

Assessment Findings
Assessment information indicated significant student concerns about the scheduling and availability of classes. We learned from a follow-up survey that the majority of the respondents preferred to schedule their classes on two days a week. Issues have emerged from responses to student surveys indicating that some of our students face a teaching and learning environment that does not always accord with knowledge of best practices. Students study alone, have little contact with faculty outside of the classroom and do not participate widely in faculty research and scholarship. Others reported that faculty and staff are not always willing to help when problems arise relating to their educational needs. Particular dissatisfaction was reported with academic and career advising. Graduating students and alumni reported that their coursework at CSUS did not prepare them as well as they would have liked with the technological and communication skills they need. Alumni reported that their education did not meet their expectations in areas of ethics, problem solving, and a commitment to life-long learning. Faculty voiced the concern that innovative teaching is not rewarded as it should be. In addition, only 36 percent of faculty responding to a survey indicated that they were skilled in "using information technology to enhance instruction."

Institutional Responses

Class Schedule. In response to concerns about the class schedule the Academic Senate developed an alternative to the traditional MWF and TTh scheduling of classes that had governed the offering of most lecture classes on the campus since time immemorial. By Fall 1995, the Senate recommended, and the President approved, a new scheduling pattern, which was implemented in Spring 1996. In addition, the President requested that Academic Affairs investigate the feasibility of changing the academic calendar from a two-semester to a trimester system. A group of faculty representing the Academic Senate and each of the Schools, along with selected administrators, met to consider the issues involved in such a change, and an outside consultant was hired to conduct an analysis of the fiscal impact. After much discussion and analysis, a different alternative, the Extended Semester, was recommended to the campus. In the Extended Semester alternative departments can offer courses, funded by general fund dollars, during the months of January, June, July, and August and include them as part of their fall and spring semester offerings. Numerous administrative hurdles need to be overcome in order to implement this program. During Summer 1996 a cohort of graduate students in Educational Administration began a program of study that allows them to enroll in one three-unit course per month on a year-round basis as part of the regular offerings of the department. Other uses of the extended semester are under review.

Learning Communities. In Fall 1995, the Dean for General Education, in consultation with the General Education Committee, initiated a Learning Communities program on a pilot basis for first semester freshmen. In the program 25-30 students enroll in two or three General Education classes as a group; the courses complement each other and are linked by a common theme. Faculty are encouraged to integrate the courses, promote group interaction and provide academic study sessions. In addition to providing some structure to the General Education program, the Learning Communities provide students with a "sense of belonging" to an academic community, a concern noted by students in the survey data. The retention rate for Learning Community participants is 85 percent which compares favorably to the University's retention rate of 70 percent. The program, in its second year of the pilot phase, has also grown from 100 students to 300 in Fall 1996. Of the 100 students who participated in Fall 1995, 85 returned in Fall 1996. While the program is open to all students, during the second year, some Learning Communities were designed for Educational Opportunity Program students admitted under special contract.

Technical, Communication and Research Skills. While a significant majority of students who responded to the graduating student and the alumni surveys are very satisfied with the academic programs of the University, the CASPER phone survey in Fall 1995 revealed some dissatisfaction with the provision of technical, communication and research skills. Since the survey, the number of computer workstations, lab hours, lab assistance and computer training opportunities for students have increased. Academic Affairs has given priority to establishing more discipline-specific computer labs so that students can better relate technology to their educational goals. A newly-adopted plan for ensuring 24-hour student access to computing resources emphasizes the need for a mix of central and discipline-based lab facilities. Through a new $75,000 grant from the Lottery Fund, the University Library expanded its library instruction program by constructing a new 20 workstation lab for individual student and class use. The Library has also increased its staff hours to help students with on-line and traditional research methods. Reference librarians have designed a series of lectures entitled, "Demystifying Library Databases" to meet student needs for information literacy.

Use of Technology in Teaching. The University's Center for Teaching and Learning and the Computing, Communications and Media Services (CCMS) has sponsored a two-day faculty workshop "Integrating Technology Into the Classroom" for the past two years during the January intersession. To meet the demand for training in the use of technology for instruction, CCMS has quadrupled its training programs for faculty. CCMS also received a lottery fund grant from the University to establish a Multi-Media Fellows Program during the 1996-97 academic year. The program is designed to provide training, resource tools and personnel support to a cadre of faculty which will help them develop the basic knowledge, skills, and experiences not only to create instructional multi-media modules, but to become mentors for other faculty. Academic Affairs provided travel grants ($150 each) to faculty who participated in instructional computing workshops and a limited number of grants for operating expense ($5,000) to departments for developing and implementing a plan for integrating instructional technology into the curriculum. The University has established a Teaching, Learning and Technology Roundtable to involve faculty in campus planning for the use of information technology. An ad hoc committee with representatives from Academic Affairs and CCMS is working to improve the degree to which instructional technology resources are dedicated to meet the needs of the faculty.

Service Learning. A new Office of Community Collaboration was established, within Academic Affairs, to provide expanded opportunities for students to engage in service learning in the community and to identify opportunities for student/faculty collaboration in community-oriented projects. A major goal of the new office is to promote civic engagement and social responsibility among CSUS students.

Other responses:

  • An Academic Advising Intern program was funded to train students to provide academic advising services in their departments as well as the Academic Advising Center
  • Funds were provided to allow students in the Cooper-Woodson College Enhancement Program to conduct research in collaboration with a faculty member and to present research findings at a campus symposium
  • A new project called "Building a Community of Writing Classrooms" brings CSUS faculty together with community college faculty to improve student readiness for college writing as well as the teaching of writing at CSUS.



Campus Life Theme

Assessment Findings

Data from student and alumni surveys indicated two broad areas of concern. One area was the low levels of participation by students in what might be called "traditional campus life" activities. Students reported spending little out-of-class time on campus other than to study. They study alone, and primarily at home. They do not attend formal campus events with any frequency. They do not report a high level of identification with the University as a whole. Some students cited safety concerns; 39% expressed an interest in living on campus if apartment-style housing were available. Students report a much higher sense of belonging at the level of the discipline; they indicate high levels of interest in participating in department clubs; they report considerable interaction with students of various backgrounds in their classes; they report very high levels of satisfaction with the quality of their academic program and faculty. The second broad area of concern focused on student services. Significant percentages of students reported dissatisfaction with some service offices, with student financial aid at the top of the list. More than half indicated they do not feel well-informed about campus events and 24% responded that they do not believe the University welcomes student input in decision making.


Institutional Responses

Student/University Relationship. The Campus Life theme of the StrategicPlan is being revised to align the University's goals more closely with student needs and concerns. It calls for increased attention to establishing a stimulating set of campus life activities, making the campus welcoming to all students, streamlining the provision of student services, and providing more opportunities for students to participate in activities at the level of the academic discipline. To address the concern that students did not have a "sense of belonging" to the campus, a larger student life component was added to the University's outreach, orientation and new student programs. Freshmen who participated in our Summer Orientation Program were able to register for classes while on campus. Continuing students were hired to counsel and advise new students; more faculty participated in the summer program. Summer Bridge, a summer transition program designed to assist underprepared students develop skills in writing and mathematics, was expanded to include more students. A new Student Access Center was opened to provide information and support to students about the full range of opportunities and services that are available to them at CSUS. The Center trains and employs student assistants to help fellow students obtain information and resolve problems. In a major initiative to change the character of student life at CSUS, a housing plan is under consideration for one and two bedroom apartment-style units with private baths.

Student Centeredness. A review of assessment data indicated a number of areas where improvement was needed in the quality of services provided to students. In response, Academic Affairs made "student centeredness" its highest priority during the 1995-96 academic year. The Academic Deans and other administrators developed new activities or expanded existing ones that put the needs of students at the center of their plans for the year. Deans, in turn, sought additional information and ideas from the students, faculty, and staff in their schools. Already a number of new programs and activities have been put in place, including:

  • a service manual for students in one school containing all forms and necessary program information;
  • special events at graduation that accord recognition to students for program- and club-based accomplishments;
  • the involvement of students in the recruitment of new students;
  • a complete evaluation of the academic policies in one school with a commitment to rewrite policies to be clearly understandable from a student viewpoint;
  • an e-mail advising system in one school whereby students can obtain academic advising services in a very convenient manner;
  • a new automated telephone answering system in one school to make it easier for students to reach the desired office;
  • the administering of "customer satisfaction" surveys to students for purposes of improving student support services;
  • expanded opportunities for internships, cooperative education placements, and service learning experiences to respond to student desire to more closely connect their coursework to the world of work;
  • peer advising;
  • an evaluation of all regular correspondence to students from Academic Affairs units to achieve "student friendliness";
  • the initiation of a comprehensive review of all academic policies (University, school, and department levels) to ensure that they best meet the needs of students;
  • the holding of open office hours by all school deans for purposes of being available to meet with students;
  • an increase in social activities designed to increase student/faculty contact outside of the classroom.


Attention to "student centeredness" continues to be at the top of the University's agenda and the term has been incorporated into the vernacular of the campus. In addition to the above activities, the expansion of opportunities for student development outside of the classroom, for department-based activities to improve student retention, and for student/faculty interaction have been identified as priorities for the allocation of funds for both the 1996-97 and 1997-98 budgets.

Information on Campus Events. A number of new initiatives, aimed at improving information about campus events, have been implemented to address student concerns. The University Union Programming Board worked with the State Hornet, the student newspaper, to increase reporter awareness of the importance of timely and complete coverage of campus events. A section of the State Hornet is now dedicated to campus- wide programs and events. The Office of Student Activities increased the production of its publication The Campus Calendar by 34 percent, in an attempt to increase the awareness of scheduled events by all members of the campus community. In addition to providing information on student life and campus programs to prospective students through the mail, information is also made available electronically through the CSUS Home Page. Information on campus clubs and organizations and opportunities to participate in student and faculty governance are now regularly shared with students. Electronic kiosks, a 24- hour computerized information line, which provide students with information on more than 250 campus topics, made their debut in Spring 1995.

Student Input to Decision-making. To address the student perception that the University does not use student feedback, President Gerth invited students and other members of the campus community to nominate any policy, procedure, or rule which might be considered cumbersome or unfair. Since its inception the program has produced changes in parking policies, improved the grade change policy, modified signs in the Library, and increased student accessibility to campus computer labs. Student participation in the life of the campus was also a major consideration in the decision to affiliate with a larger, more comprehensive and prestigious athletic conference. In a general election students gave the University a clear message that they value a viable athletic program.

Campus Environment. More than 50 percent of the respondents in the SNAPS survey indicated that safety was an important factor influencing their choice of a university. The topic of campus safety is an ever present concern of parents who attend the University's Parent Orientation program each summer. The University has taken a proactive approach to safety on the campus. With the support of the Division of Student Affairs, the Department of Public Safety instituted police bicycle patrols in an effort to increase officer visibility and approachability. In response to concerns about lighting on the campus, Public Safety initiated a complete lighting assessment of the campus each month. The Office of Housing and Residential Life provided funds to support a Public Safety Officer to provide in-service training and consultation for residence hall staff members on a part-time basis. A night shuttle service was re-instituted by University Traffic and Parking Services. Each of these initiatives is designed to make CSUS a more responsive, safer campus for students.

Student Support Services. Survey data indicated a low level of student satisfaction with the University's Financial Aid Office. A Voice Response System and three electronic kiosks were installed in Spring 1995 which permit students to access their Financial Aid documents, award status and disbursement information. A Satisfactory Progress Computer Program was implemented in Spring 1996, which notifies students of their status immediately after grades are posted. All-Calc, a software program designed to provide timely eligibility estimates, was placed in service during the 1995-96 academic year for prospective students. In addition to extending hours of service, the Financial Aid Office hired peer counselors and additional professional and support staff to improve service to a increasing number of financial aid applicants. A Financial Aid Management System, which will link the University's Student Information System (SIS) with financial aid records, will improve the ability of the Office to process financial aid applications in a more timely manner. Finally, efforts are underway to upgrade the infrastructure in the Financial Aid Office with an electronic document imaging system. For student services generally, evening services, and hours of operation, for both undergraduate and graduate students were expanded during the 1995-96 academic year. A system was also put in place whereby evening students could request virtually any service on campus and be guaranteed a response from that office within 24 hours of the request. In Fall 1995 the University worked with the Associated Students, Inc. to establish a Student Access Center on the first floor of the University Library where students can receive information on a broad range of University programs and services.


Enrollment Planning Theme

Assessment Findings

The major concern arising from the Enrollment Planning theme assessment paper was student retention. Although it is not clear what external standards would be most appropriate for purposes of comparison, there is consensus across the University that our retention rates are too low. Rates vary considerably across departments and sub- populations, but data show that particular trouble areas are undeclared students and African American students. For all students, the largest loss occurs in the first two years. It appears from student survey responses that improvements in class scheduling and some support services such as advising might improve retention rates. A related concern was student preparedness. Data showed that 27% of Fall 1995 entering first time freshmen and 22% of new transfers were placed on academic action by the end of the first term. In addition, students are not taking the English Placement Test in a timely manner and those who do are failing at higher rates. Increasing numbers of students are requiring remediation. Student enrollment trends are also problematic, particularly for undergraduate transfers. (Graduate applications, admissions, and enrollments are at all time highs.) There is some basis for concluding that slow admissions and financial aid processing contribute to lower enrollment rates. There are a number of academic programs that have been targeted for growth that have not accomplished growth. Campus enrollment balance among class levels is in line with University policy.

Institutional Responses

University enrollment management. The principal initiative taken in response to concerns about enrollment and retention has been the hiring of a senior administrator into a new position called Associate Vice President for Enrollment Management. This administrator, located within Student Affairs, supervises the Office of Admissions and Records and University Outreach. Major efforts are underway to better coordinate University Outreach activities with school and department outreach and enrollment goals and to improve the processing of undergraduate admissions.

School-based retention. Academic departments and schools have implemented a variety of retention strategies appropriate to their own programs. Retention rate data, by program, has been distributed to schools which will be, for the first time, setting retention rate targets by program. This new responsibility for retention relates closely to the new directions that deans are taking with respect to student-centeredness. A review of assessment data indicates that discipline-based activities that focus on connecting students to other students and faculty should have positive effects on student retention.

Entrance requirements. As part of a California State University systemwide effort to better articulate with community colleges, CSUS will phase in enforcement of the requirement that transfer students complete their English, math, and critical thinking requirements before being admitted to CSUS. Senior administrators at CSUS have been meeting with local community college administrators to ensure that this transition is smooth and that students are given appropriate notice of the change.

Writing Proficiency Examination. CSUS is enforcing the requirement that students pass the WPE before taking certain upper division courses. The intent is to ensure that students be equipped as soon as possible for the rigors of a University education, thus improving their chances of success.

Schedule Improvements. The changes to the class schedule described above under the Teaching and Learning theme are also related to efforts to improve student retention.


Pluralism Theme

Assessment Findings

The major concern arising from the Pluralism Theme Assessment Paper was the lower retention and graduation rates for African American, Hispanic, and American Indian students. African American students had the lowest rates. A second concern relates to the campus climate with regard to racial tolerance and respect. Two-thirds of graduating students surveyed indicated that they had grown "very much" or "much" in the area of interacting with people from other cultures and most attributed the growth to their college experience. In addition, most students reported that faculty appear comfortable teaching students of all racial and ethnic groups. However, enough students disputed this claim and reported a climate of discrimination for the University to conclude that issues of race continue to be salient. Students reported a high degree of interaction with students of different race and ethnic groups in the classroom, indicating that efforts to improve relations might be directed at the instructional program rather than only at support services and special programs. Among staff surveyed, 44% indicated a concern about affirmative action hiring policies, mirroring the debate that is occurring across the state and the nation and indicating a need for continued campus dialog and staff development. Most faculty surveyed reported having participated in conferences and workshops that increased their respect for diversity.

Institutional Responses

Retention. Despite the changing climate around Affirmative Action in California, the President and Provost have made it clear that the University will continue to pursue its goals of diversity and pluralism as laid out in the Strategic Plan. School and department- based retention efforts and enhanced student-centeredness are expected to partially address low retention rates among these groups. Funding for School-based Educational Equity programs has been secured through lottery dollars.

Campus climate. The President has vowed to continue a campus dialog on issues of race and pluralism. To this end, a project called "Towards a Race Dialogue" was funded as a collaborative effort among faculty and students in several departments to educate the CSUS community about the issue of affirmative action in higher education. This project was assigned to the Center for Teaching and Learning, which itself offers workshops and materials to increase faculty competence in working with diverse student populations. The Multicultural Center was given funds to sponsor a new lecture series called "Politics of the Americas: Past, Present, and Future" to promote cultural diversity within the University. Several departments across the campus were involved with identifying guest speakers and with integrating speech topics into classroom discussions.

Staff development. The University has stepped up efforts to address staff development through a focus on communication. New workshops have been developed covering issues around staff interactions with students and other staff, customer service, effective listening, mutual respect, and conflict resolution. At this time workshops are voluntary. We are moving to target certain audiences for staff development of this nature and are considering mandatory training.


Academic Programs Theme

Assessment Findings

Few major issues arose from the review of the Academic Programs Theme Assessment paper. Students generally reported satisfaction with their academic course of study at CSUS. Some of the issues from the Teaching and Learning theme relate here as well, such as concern with the effectiveness of the current General Education program and the education students receive in areas of technology and communication. The more detailed assessment of academic program quality occurs through the academic program review process, discussed elsewhere in this chapter. One issue discussed in the Council's review of the assessment paper concerned the quality of departmental assessment plans and the need to increase faculty expertise in student outcomes assessment. Other university- level issues arising from the assessment data are the need to expand distance learning offerings and to expand the regional emphases of our curriculum.

Institutional Responses

Distance education. Academic Affairs has appointed a faculty member, half- time, to take the lead in Distance Education. This area had been one of numerous responsibilities of an academic administrator and was not getting the attention it deserved.
Outcomes assessment. A group of faculty have been given release time beginning Spring 1997 to work as a team on improving student outcomes assessment in their departments. They are expected not only to develop departmental assessment plans for their department's program review but to also serve as resources to other faculty across the campus as they work on assessment plans.

Regional emphasis of programs. The new Office of Community Collaboration (OCC) is working with several academic departments to involve students and faculty in joint activities with state and local agencies and community organizations. One major initiative in which the OCC is involved--Sacramento Enriches--has been located on campus. It is a regional effort to improve the health and welfare of children and youth in the Sacramento region. Faculty and students from several University programs are involved. A new Human Services Collaboration Group has been formed and supported with University funds to promote interaction among different departments in the University that train human services professionals so that students would be trained with the collaborative skills that are needed in the professional world. Partnerships have been forged with school districts and several community agencies to carry out the goals of this project.


Public Life and Capital Campus Themes

Assessment Findings

Assessment papers for both these themes as well as for the Campus Life theme make it readily apparent that the CSUS public life falls short of the traditional ideal. A glaring shortcoming is the lack of community support for Intercollegiate Athletics. Also apparent was the subordinate role that CSUS has typically played in regional economic development issues. The contributions of the campus to the region's cultural life are more impressive, through the Festival of New American Music, the Spring Arts Festival, two public radio stations, high-profile public lectures, offerings of the Theatre Arts, Music, and Art departments, and public events sponsored by the Multicultural Center. In addition, all of our professional schools have a variety of activities involving regional constituencies, virtually all departments offer internship opportunities, and there are numerous Centers and Institutes serving a variety of regional needs. The primary issue raised by a review of this plethora of activity was the minimal coordination across the University. Members of the University community cannot easily know what each other is doing and members of the external community cannot gain easy access to those at CSUS with whom they can build constructive relationships. In addition, the survey of graduating students indicated that learning is generally limited to the classroom, implying that too much of the above- mentioned activity goes on without substantial student participation as part of their academic programs. Finally, the sheer paucity of information available to assess progress toward the goal of the Capital Campus theme indicated that more must be done to generate activity in a broad base of departments that reflect the opportunities and responsibilities associated with being a "capital campus".

Institutional Responses

Intercollegiate Athletics. In an attempt to increase the quality and visibility of CSUS Intercollegiate Athletics in the region, we have joined the Big Sky and the Big West conferences and funded more scholarships. In addition, the historic Memorial Auditorium in downtown Sacramento has been obtained as the home site for men's and women's basketball games.
Regional partnerships. CSUS has become a member of Campus Compact established to assist universities in forming partnerships with local community service agencies to expand opportunities for students and, at the same time, provide much needed volunteer hours to the community. The Office of Student Activities created the Community Service Volunteer Agency to collect information and coordinate volunteerism on behalf of more than 250 registered student organizations. A faculty/student affairs partnership fostered the development of a Service Learning Program at CSUS with the goal of integrating community service into classroom activities. In Fall 1996, Academic Affairs established the Office of Community Collaboration (OCC) to expand opportunities for students and faculty in the community. A central task of the OCC will be to develop a data base of faculty interests and community needs to facilitate identification of potential partners both on and off campus. A faculty member was appointed to the newly-created position of Executive Director of University Economic Development to represent the University in regional economic development issues. These new initiatives will provide the opportunity for students and faculty to use the community as a laboratory for learning and to contribute to the economic and social development of the region.

State capital partnerships. The University recognizes the tremendous advantages it has from its location in the state capital. In March 1995 an Institute for Educational Reform was started at CSUS, headed by a well-respected, long-time member of the California State Senate who had recently retired. In late 1996 an offshoot of that effort--the Center for the Improvement of Reading Instruction--opened on campus as well, to address one of the most salient issues on the state policy agenda. The Center for California Studies continues to expand its scope of activity. Its legislator training programs are becoming more central as term limits increases the number of new members of the Legislature. A new seminar series has been started by the Graduate Program in Public Policy and Administration within the School of Social Sciences and Interdisciplinary Studies to spur increased awareness and interest across the disciplines in public policy issues.


Faculty Scholarship Theme

Assessment, at the level of the Council of University Planning, of the faculty scholarship issue, has been more difficult than for other themes. All other themes were assessed once, in the 1995-96 academic year, and again in the 1996-97 academic year. The faculty scholarship theme assessment paper that was developed in 1995-96 suffered from lack of consensus not only about the definition of scholarship but about how one assesses scholarship at the University level. For example, is it enough to tally the numbers of faculty submitting grant proposals and publishing research, or should we assess its impact on teaching and learning? An ad hoc group of faculty will be assisting staff in the Office of Institutional Studies to develop the outline for the faculty scholarship assessment paper to be presented to the Council at a later date. Certainly much is occurring across the campus in the domain of enhancing faculty scholarship, but it would be inaccurate to attribute these efforts to the University assessment model.

Chapter 7

Creating an Infrastructure to Support Assessment


The purpose of this chapter is to provide more information about three particular consequences to the campus of the WASC Self-Study. (1) The development of assessment plans at the program (academic and non-academic) level that will document student outcomes and student learning outcomes; (2) A critical area of faculty activity on the campus, the faculty evaluation process, that has been significantly shaped by faculty participation in the WASC Self-Study activities; and (3) The issue of building and sustaining faculty expertise and involvement in assessment as a key measure necessary to assure that an infrastructure that sustains the "culture of evidence" is built and nourished at CSUS.


Academic Program Assessment Initiatives

One of the first consequences of the campus commitment to do an accreditation Self-Study based in evidence was a revision to the campus Academic Program Review policy. Beginning with the 1994-95 academic year, every department was mandated to develop an assessment plan for its programs as part of the process of an Academic Program Review. This plan for the assessment of student outcomes was to be developed two years before the actual program review. One year before the review, as the department conducted its Self-Study, data on student outcomes is to be collected and incorporated into the departmental Self-Study activities.

During the December visit, WASC team members met with representatives of selected departments to learn more fully about efforts within the academic programs. This section summarily describe their efforts.


Efforts in the Government department to assess student outcomes are very preliminary. The Department's assessment plan describes several different approaches to gathering information regarding its students. The Department believes assessment should focus its attention first on what effort faculty are putting forward to make sure students have theopportunity to learn. We are also interested in knowing what other circumstances in students' lives affect their opportunities to learn. To gather this information the Department revised its course evaluation form to seek information on how many hours a week students work, to see whether faculty provide the opportunity for students to improve their writing skills, and to gauge how regularly students actually attend class.

The Department will engage in discussions this semester surrounding our Self- Study, that will take us into the area of trying, as specifically as possible, to define and generalize our expectations for students and to share with one another how our courses are attempting to promote the realization of the goals we have identified.

The Department administered a survey to students who submitted graduation petitions in Fall 1996 to graduate in May 1997 (and are administering it currently to students now submitting such petitions) asking a variety of questions about their experience in the Government major. Already the Department has learned that it needs to revise some aspects of the survey, which will be done this semester. Survey responses will give the department an opportunity to make some preliminary judgments as to what students believe is important to them and how well the Department is meeting their perceived expectations. It hopes to learn something about how to educate students about the Department's expectations.

During Spring 1997, the Department plans to have interviews with a small, randomly selected sample of graduating seniors who submitted the survey last Fall semester in order to probe more deeply their experiences in the Government major. Alumni are being surveyed by the Office of Institutional Studies as part of the University's assessment effort. Finally, the Department will consider, as these efforts continue, how it may refine its assessment efforts. The Chair is establishing contacts through the political science "network" to learn more about how others in our discipline are approaching this task.


The objectives of the French major are tested by a comprehensive exam, the French Major Graduating Assessment. The purpose of this exam is twofold: to let students know their strengths and weaknesses in French, and to inform the French faculty of the strengths and weaknesses of the French degree program.

The French program's progress toward its objectives is also gauged by student opinion polls. Do the current students at CSUS and (more important) the alumni with a degree in French consider that the French program is doing what it should to train its students in all the necessary skills?

From the numerical results, it is clear that the students-feel that the French program has allowed them to become fluent in French (65% "agree" or "strongly agree" on question 3) and has given them sufficient knowledge to understand French culture (79% "agree" or "strongly agree" on question 1). The written comments amplify these responses by commenting on the quality of the teaching and teachers. To choose a few samples: "Great Prof's" (sic), "French major program's greatest strengths - excellent, caring faculty," "A dedicated and inspirational faculty," "I think there are some great professors but not enough."

The latter combines praise and blame in one comment, praise for the teachers and blame for not having enough teachers or classes. The French program's lowest score (42% "agree" or "strongly agree") on the statement "The University provides adequate faculty and resources" reflects the reduction in course offerings in upper division French because of budget cuts and low enrollment.

Other student comments suggest other ways in which the French program might be improved. Several students expressed the need for classes oriented towards business and science and a reduction in the number of literature classes, others want more "poetry - French renaissance" classes. A few students from EDTE 385 - Teaching Methods in Foreign Languages, a class staffed by the Department, have suggested some changes in the focus of that class. These suggestions are now being considered by the faculty.


The success of the German program in training its majors is measured by student performance in German 192, the capstone course for this program. It is the designation as "seminar" that differentiates it from other upper-division courses and qualifies it as a "capstone" course.

The German program's progress toward its five objectives is also gauged by student opinion polls. Do the current students at CSUS and (more important) the alumni with a degree in German consider that the German program is doing what it should to train its students in all the necessary skills?

It is clear from both the tabulated scores and the written comments that the students hold the German program in esteem. In the Departmental questionnaire (DATA pp. 175- 177) the comments for German were on the whole more approving than those of other language programs. A majority of the students consider the faculty to skilled, caring, and scholarly (questions 3, 4, 5; scores "agree" or "strongly agree"). The faculty is available for consultation and advising (questions 2, 8). The program teaches German culture well (79% "agree" or "strongly agree" on question 7). The students feel that they understand linguistic phenomena (64% "agree" or "strongly agree" on question 9). They feel that they have become proficient in German (71% "agree" or "strongly agree" on question 6). Comments reinforce this impression: "The instructors...are the greatest strengths." "German teachers...believe in their students and are very excited and personally interested in the subjects..." "The German Department has professors that give wonderful personal attention to their students and their needs." Students made suggestions for improvement: the favorite "less literature" and more grammar and conversation classes; classes at more convenient times; and then-as in all language programs-more classes. The graduate student questionnaires raised some problems: too much English used in undergraduate classes; problems with financial aid, and so on. The Department will respond to these suggestions as far as possible.


The Spanish program's success in meeting its objectives is gauged by student performance in upper-division Spanish classes and by performance on the Spanish program's "G" exam, a comprehensive exam required of all graduates in Spanish and of all entering graduate students. (The scores of entering graduate students give the Department a standard by which to judge its own graduates: if the entering graduate students were to do significantly better or worse than the CSUS graduating seniors, then we could deduce some facts about the CSUS Spanish program. In fact, the scores are similar.) In addition, classroom questionnaires administered in 1995-96 reveal the students' perception of the Spanish program's quality. As an illustration, the following discussion concentrates on the "G" exam and the questionnaires. The G-Exam is required of all Spanish majors, graduate students and students seeking the Single Subject Credential in Spanish. The exam comprises nine sections: 1) Listening Comprehension; 2) Reading Comprehension; 3) Vocabulary; 4) Grammatical Structures; 5) Verbs; 6) Grammatical Theory; 7) Spelling and Accentuation; 8) Composition and 9) an Individual Oral Interview. Students must receive a score of 70% (14 out of 20 points) in order to pass each section. When students are notified of sections they have failed, they must wait at least one semester before retaking those sections. Students are encouraged to make an appointment with the exam administrator right away in order to go over their results and plan strategies for a successful second attempt. Those who choose not to take advantage of this opportunity increase their changes of failing sections again. The results of the G Exam govern the topics covered and/or emphasized in the Spanish major classes. Since the administrators of the G Exam and the Spanish faculty are one and the same, there is constant feedback about the students' needs in their major program.

The Spanish program's progress toward its four objectives is also gauged by student opinion polls. Do the current students at CSUS and (more important) the alumni with a degree in Spanish consider that the Spanish program is doing what it should to train its students in all the necessary skills?

From the numerical results, it is clear that the students feel that the Spanish program has allowed them to become fluent in Spanish and has given them sufficient knowledge to understand Spanish culture (74% "agree" or "strongly agree" on question 6 and 78% "agree' or "strongly agree" on question 7 respectively). The students feel that they understand linguistic phenomena (69% "agree" or "strongly agree" on question 9). The written comments amplify these responses by commenting on the quality of the teaching and teachers. To choose a few samples: "...a feeling of community," "... the greatest strengths...are the professors," "... greatest strengths are the teachers," "The professors are very helpful in advising us." As the last comment indicates, the students are satisfied with advising (questions 2 and 8), with the performance of the faculty (questions 3, 4, 5, the highest scores), and with the overseas programs (question 12). They think that more classes should be offered at off-campus locations (question 10), and they feel that CSUS should offer more Spanish classes (question 11, the lowest score). None of these reactions is surprising.

Biological Sciences

Prior to designing instruments for outcomes assessment in the Biological Sciences major, the faculty developed both content and skills objectives for the Department core curriculum. Assessment instruments were then designed to determine if these objectives were being met by the current curriculum. The various means of assessment were aimed at several populations: current students, graduating senior, alumni, and employers of alumni.

For skills objectives assessment, the Department utilizes poster session based on senior laboratory projects. Students in BIO 121 Cell Physiology design, execute, and gather and analyze data for a laboratory project. Students prepare a written project proposal, design and present a poster which includes photographs and/or graphs of results for examination by students and faculty, and orally present the research outlined in the poster. The posters are judged by both students and faculty, and the three best are selected for display in the Science building. BIO 121 serves and has served for several years as a capstone course. An additional goal for skills assessment in BIO 121 is to develop a means by which to measure objectively the level of quality of the posters sessions and project proposal papers. This will allow the Department faculty to chart progress in meeting skills objectives.

Assessment of progress in meeting content objectives will be assessed by inserting "GRE-type" questions into examinations taken by students in upper division core courses. The Department plans to begin this assessment in the first and penultimate core courses taken by majors, BIO 160 General Ecology and BIO 184 General Genetics. These courses are taken by very different class levels of students and cover vastly different content areas in biology. This addresses the recommendation of a WASC team member who suggested assessing students at various class levels instead of just seniors. Student responses will be scored on the number of correct responses for the class as a whole. This will give an indication of the success in meeting content skills.

Additionally, the Department hopes to continue to annually administer a questionnaire to graduating seniors to obtain their perceptions in the Department's success in meeting content goals. This questionnaire was administered in Spring 1996. Forty-four students responded. Of these, 80% felt that the core curriculum prepared them exceptionally well or more than adequately in 8 of the 10 curriculum areas identified in the questionnaire. The exceptions were preparation in evolution which the faculty is addressing with the introduction of a new course, and preparation in organic and biochemistry, which is not under the control of the Department. Yearly administration of the questionnaire would allow the Department to chart progress in meeting content goals.
In Fall 1996, the Department sent a questionnaire to Fall graduates which was designed to provide information on both progress in meeting curricular goals and on the future plans of graduates. Few responses were received. In Spring 1997, the Department will begin a pilot project utilizing exit interviews in which graduating seniors chosen randomly will be interviewed on such topics as future plans and student perceptions of strengths and weaknesses of the Department.

Alumni questionnaires similar to those administered to graduating seniors were administered to alumni as part of the Self-Study process. Data on this questionnaire is pending and has not yet been analyzed. A questionnaire for employers of alumni was included in the alumni survey in which the employer was asked to assess the CSUS Biological Sciences alumna's preparation in five areas. Twenty responses were received, and they indicate that the level of preparation was better or as good as that of graduates of other universities.

Outcomes assessment is viewed as an ongoing endeavor by the Biological Sciences faculty, and assessment instruments will continue to evolve as curricular goals are modified and as the faculty's knowledge of assessment instruments grows.

Electrical and Electronic Engineering

"Outcomes Assessment" is now an important part of the accreditation process in Engineering. ABET (Accreditation Board for Engineering and Technology) is presently considering a new set of guidelines for the evaluation and accreditation of engineering programs that focuses on outcomes assessment. The Department of Electrical and Electronic Engineering at CSUS will be visited by ABET during Fall 1997 for an accreditation evaluation. The internal program review will take place during the upcoming Spring 1997 semester. The EEE Department included an assessment plan in its June 1996 Self-Study document, portions of which have already been implemented. The goal is to develop a "non-intrusive" outcomes assessment process for the department that is tied to our mission that can be used to guide decision making, maintain accountability and foster understanding. This draft will focus on the "Student Outcomes" portion of our assessment plan which we plan to implement during Spring 1997.

The objectives of the assessment plan are to determine the students' technical ability/knowledge, ability for life-long learning, laboratory competence, leadership/teamwork ability, and communication skills. Another goal is to develop a set of evaluation measures that can account for students' development after entering the program. This is especially important considering the substantial number of transfer students entering the Engineering program with varying levels of preparation. Ultimately, longitudinal studies of significant scope will be required to relate professional performance to expected outcomes, and validate the evaluation assessment measures. During the Spring semester the department plans to survey the major employers of our graduates and use the data for improving program design and delivery. Also the department chair will conduct exit interviews of all graduating students to determine their satisfaction with the program and their views on possible improvements.

The department plans to evaluate student outcomes with an ongoing outcomes assessment process in the following courses:

  • Capstone Senior Design Project Course (EEE 190B/EEE 191B)
  • Electronics Laboratory (EEE 118L)
  • Electric Networks Laboratory (EEE 117L)

A detailed description of the evaluation process we propose to employ appears in our Self-Study document. Briefly, we have developed performance criteria for each objective (What will students be able to do, or be, or possess when the goal is accomplished) and the techniques used to achieve the goals. Through timely feedback, both during the process and at the end, determine if the performance criteria were met and the objectives were achieved. For instance, laboratory competence and communication skills are assessed through evaluation of written and oral reports collected from EEE 117L, EEE 118L, and EEE 190B/EEE 191B. These courses are required of all EEE students. Technical ability, and leadership/teamwork ability is evaluated through an assessment of a portfolio of final exams, laboratory exams, design reviews and reports, and the Senior Design Final Report. The department is also discussing a standardized exam to be administered to all second semester juniors, that covers the fundamentals and core electrical engineering areas and concepts. For starters and for benchmarking purposes, the department plans to use a test that was administered to a similar group of students at Cal Poly, SLO to all students enrolled in EEE 118.

Special Education, Rehabilitation, and School Psychology

The department's assessment plan is quite comprehensive and very driven by accreditation requirements. Feedback is received on candidate performance throughout the training process and after employment in the discipline. Sources of feedback/evaluation include direct observation of candidate performance with reference to professional competencies (both during training and after training), candidate evaluation of the effectiveness of program components in preparing them for the profession, and employer evaluation of the effectiveness of program components in preparing entry-level professionals. A variety of methods and instruments is used to collect this information. As a result of this developmental assessment of outcomes for our students, program adjustments are routinely made for individual students (e.g., recommending a different pattern of coursework or additional field hours) and major program changes in order to render our programs more effective for all students (e.g., requiring early field experience for all school psychology candidates; adding lab sections to the methods courses in special education).


Nonacademic Program Assessment Initiatives

The non-academic program review process contains three main components. First, the Self-Study is conducted by the management and staff of the department under review. A broad ranging series of questions addressing such topics as administrative placement in the university structure, human and fiscal resources, physical space, and technical support are to be discussed and written answers provided.

The second component is an audit by the Department of Management Services. At a scheduled period of time, this department's reviewer assembles key personnel from the department under review and conducts an introduction to the audit. The reviewer describes the purpose and scope of the review and compiles a list of key personnel to be interviewed. This list contains a sample of those with the department under review as well as the names of individuals within the university community who are knowledgeable of the services this department provides. The subsequent individual interviews cover issues dealing with management structure; resource management; adherence to an accomplishment of goals and objectives, and a review of staffing and technical support.

Procurement Services

In 1995-96 a non-academic program review of Procurement Services was conducted. The Self-Study was completed early in the review cycle. The management review,, which was conducted soon thereafter, left only the decision of how to evaluate services provided by the office.

The Chair of the non-academic review process, in consultation with the Director of Procurement and the Vice President for Administration, decided a comprehensive user's survey would be appropriate. Conducted by the Institute for Social Research, the survey was done in two phases. First, on-campus clients were surveyed with respect to their level of satisfaction with procurement requests. On-campus constituents were asked to rate the department's quality with respect to efficiency, completeness, demeanor, helpfulness, and understanding of the specification of requests, among others. Finally, the second survey was sent to off-campus vendors. In addition to the areas above, vendors also responded to issues such as notification of requests for bids and the bid process itself. The results of these surveys were included in the final report.


Faculty Evaluation

Perhaps the most clear-cut and potentially significant initiative growing out of WASC concerns rethinking of faculty evaluation. For some time there has been growing interest on the Sacramento campus in rethinking how the performance and effectiveness of faculty ought to be evaluated. While periodic efforts to grapple with parts of the puzzle (e.g., a Senate inquiry several years ago into the nature of scholarship and to reframe expectations in this area) have come and gone with varied degrees of success, nothing of a sustained or comprehensive nature has taken place on the campus in at least twenty years. By identifying teaching effectiveness as one of the core areas to be explored for accreditation, WASC set in motion a series of steps that has already opened a conversation long overdue and has provided conditions that show promise of leading to significant policy change.

The very act of establishing a WASC Steering Committee subgroup on teaching effectiveness was in itself a significant development. For one thing, it was the first such campus-wide group in anyone's memory. For another, it served as the catalyst to mount the first inventory of campus studies on teaching. Perhaps most important, it launched the first major survey of student and alumni attitudes in the university's history. At the same time, the WASC process provided a "hot house" for ideas that diverse faculty had already been contemplating in relative isolation. The momentum given to reconsidering the RPT process by WASC cannot be overstated.

As a direct result of WASC shorter-term efforts, the Faculty Policies Committee (FPC) of the Academic Senate has assumed the longer-range task of continuing the conversation on teaching effectiveness. Using data provided by the WASC Steering Committee and drawing on the Steering Committee's counsel, the FPC established a Working Group on Faculty Evaluation that has had wide membership from across the campus community and which met regularly during the 1995-96 academic year. In Spring 1996, the Working Group got Academic Senate endorsement in principle of its direction, and then set about devising a new direction for Retention, Tenure, and Promotion (RTP) at CSUS, which it articulated in a Position Paper written in Fall 1996. The paper called for active consideration of a series of reforms in the RTP process, including (1) the spelling out of expectations for student development in each discipline; (2) the spelling out of expectations/standards for faculty performance within the context of the discipline for each of the areas of evaluation (i.e., teaching, scholarship and creative activity, and service/governance); (3) the devising of a development plan for each faculty member that gives tentative intentions for the next three or four years; (4) a teaching/scholarship portfolio that would highlight and document the best of the candidate's work as a supplement to the conventional Personnel File, which lists accomplishments; (5) a reflective statement that would address the candidate's performance in one or more of the areas under evaluation, depending on the department's inclination. The Working Group's Position Paper on Faculty Evaluation was widely distributed to the campus in November 1996, and comments solicited. The comments have since been compiled and will be distributed to the campus by mid-February 1997. The Senate Working Group's next step is to consider ways to encourage reform and to recommend them to the Faculty Policy Committee, which in turn will make recommendations to the full Senate, hopefully no later than the end of Spring 1997. Early in their deliberations, the Working Group's thought was that a package would be presented to the Senate for debate and possible adoption as policy. Increasingly, though, as working group members became more familiar with the literature and examined successful change models on other campuses, the consensus has changed. Indeed, the thinking now tends to lean toward recommending modes of change that are "department up" rather than "university down." Put another way, the group increasingly became concerned that they were concentrating on political/policy solutions for cultural/academic problems, rather than exploring ways that are (a) discipline or domain specific and (b) have faculty ownership from the beginning. Using this approach, the Working Group is leaning toward exploring ways to strengthen and expand the Center for Teaching and Learning (CTL) to serve as a clearing house and incubator for new directions in faculty evaluation. Following this approach, the Working Group would emphasize (a) opening and sustaining a conversation at the department and school level of RPT using its Position Paper as a starting point; (b) working with the CTL to devise and implement an action plan for such activity; and (c) requesting resources to carry out such an action plan. The concern, of course, is that efforts of the sort described here will evaporate once WASC is completed. However, there is no reason to believe that the momentum will necessarily ebb as individuals involved in WASC or the Working Group move on to other interests if agencies such as the Faculty Policies Committee and the Center for Teaching and Learning, in cooperation with the university- wide administration, provide the institutional continuity and leadership.


Current Infrastructure to Support Student Outcomes Assessment
in Academic Programs

While the University has as assessment policy requiring departments to develop and implement assessment plans, the infrastructure to support and monitor progress continues to evolve. The Associate Vice President for Academic Affairs, who is the accreditation liaison officer to WASC, has been assigned this responsibility. The Office of Institutional Studies both supports assessment activities for the Council for University Planning, and provides consultation and support to departments as they develop assessment plans.

Role of the Associate Vice President for Academic Affairs
For the past several years, the Associate Vice President has held workshops on student outcomes assessment in the major for departments one year prior to their scheduled Self-Study phase of the academic program review process. For the past several years, representatives from these departments have attended the AAHE Conference on Assessment supported by the Office of Academic Affairs. For two years Professor Diane Halpern from CSU, Bernardino and colleagues of hers assisted the University in its effort to educate faculty about assessment.

Departments develop their assessment plans in consultation with the Associate Vice President and Institutional Studies; they continue to work with these offices as they implement their plans. While the first group of departments in the program review process is implementing plans, the second group is developing its plans. Because we perceive assessment to be a continual process (not something that is done once every six years before the department's Self-Study), we understand the need to build a strong, accessible infrastructure to ensure that departments receive support and assistance with their assessment efforts.

Associate Vice President Gray invited Don Farmer, Vice President of Academic Affairs at King's College, and three of his faculty to present a workshop on Student Outcomes Assessment in the Major. On March 3, 1997 all departments were invited to this workshop; 60 chairs and faculty participated. While previous workshops have been helpful, our department chairs want to learn how their peers are actually assessing student learning in the major. It was our attendance at the AAHE Assessment Conference last June that exposed us to King's College assessment team. It is our plan to continue bringing faculty with experience in assessment to CSUS to share their strategies and results.

Role of Institutional Studies
The staff in the Office of Institutional Studies provides support to academic programs in several ways. Institutional Studies has a repository of information on assessment of student outcomes. Its staff provide suggestions and ideas to departments as they develop their assessment plans.

As part of the University-level assessment of its strategic themes, Institutional Studies is also responsible for administering the ACT-Alumni Survey to alumni. The Council for University Planning endorsed the plan of sending the alumni survey to graduates from departments engaged in developing assessment plans. Department chairs are asked to write a letter to alumni and develop 10 additional survey questions to include with the ACT-Alumni Survey. The information collected from alumni in each department is sent back to the departments for use in preparing their self-studies. Similarly, results of the graduating student survey of majors in these departments are forwarded to the department. Institutional Studies administers, summarizes and forwards results of both surveys to the department along with other university data on the department, e.g., retention and graduation rates, grade distribution. The data from these surveys of graduating seniors and alumni provide departments with perceptions of the "outcomes" of their education and provide general information that guide departments in developing more specific outcome measures for their academic programs.

General Education
While Academic Affairs and Institutional Studies have played primary roles in supporting the development and implementation of assessment plans in academic departments, the General Education Committee, with the support of the former Dean for General Education and the current Faculty Coordinator for General Education, has designed and implemented its own assessment plan. The Committee has completed the assessment of "Race and Ethnicity" courses and are now engaged in assessing Area B: Physical and Life Sciences, and have initiated an evaluation of the University's writing requirements. The Faculty General Education Coordinator, who reports to the Associate Vice President, and the Committee will continue to be responsible for assessing the General Education Program and CSUS graduation requirements.



The Infrastructure: Additional Plans for the Future

Now that the WASC Self-Study and review process is drawing to a close, we recognize that it is important to maintain the momentum generated over the past several years. The link between establishing priorities and resource allocation and assessment is strong at the University level. Chapter 6 demonstrates the role that assessment now plays in setting resource priorities and shaping decisions at the institutional level. In preparation for the 1997-98 budget cycle, Deans were asked to relate their requests to the University's Strategic Plan goals.

In an effort to link assessment to priorities and allocations at the School level, Academic Affairs will be working with the Deans to develop stronger linkages between the results of academic program reviews and the School's priorities for resource allocation. In the future as evidence from assessment initiatives becomes an important aspect of the program review process, the results of assessment will be used by the Dean and Department Chairs to improve and enhance academic programs. The Deans will be expected to use the evidence in setting School priorities. Strengthening the link between program based assessment and the decision-making process at the level of the Dean and, ultimately, Academic Affairs will be a challenge for us over the next several years. However, the Academic Senate's Curriculum Policies Committee and the Associate Vice President for Academic Affairs are at this time developing plans to strengthen the program review process and involve the Deans more in the process itself.
It is impossible for one administrator, the Associate Vice President, and one office, Institutional Studies, both with other University responsibilities to carry the full responsibility for assessing student outcomes in Academic Programs. Academic Affairs set aside resources in the 1996-97 budget to allow the Associate Vice President and to support departments as they developed and implemented their assessment plans. Funds were used to: 1) support faculty from five departments to develop and implement assessment plans, and 2) develop expertise among the faculty that can be shaped with other faculty. We selected departments who are committed and eager to get serious about the use of student outcomes to improve their programs: Biological Sciences, Computer Science, Business, Communication Studies, and Electrical and Electronic Engineering. Each department received assigned time for one faculty member for the Spring 1997 semester. If this pilot program is successful, we will extend this opportunity to more programs next year. As we identify faculty who are interested and willing to become a resource for other faculty, we envision two additional alternatives for strengthening the infrastructure for assessment of student outcomes in academic programs.

a. The Center for Teaching and Learning at CSUS could become a home for faculty with experience in assessment, who would be available to consult with other faculty. The leadership of the Center could be responsible for providing workshops and other initiatives to encourage and support for faculty. Before any decision is made to move in this direction, the University's Faculty Policies Committee of the Academic Senate will need to consider modifying the mission of the University's Center for Teaching and Learning.

b. Another alternative would be to build the infrastructure for student outcomes assessment in each school. This alternative requires a commitment of the school dean and faculty. Because of disciplinary commonalities in each school, the sharing of similar assessment strategies could be quite effective. Because academic programs with external, professional accreditations are increasingly expecting programs to assess student outcomes, Schools will of necessity become involved in the assessment effort.

In both scenarios, the Associate Vice President for Academic Affairs and the Office of Institutional Studies will continue to provide leadership and support in their respective roles in the assessment effort. Academic Affairs is committed to following through on these alternatives and perhaps others, as appropriate.

While an administrative infrastructure to support assessment of student outcomes in academic programs is necessary, it is also critical that the Academic Senate and its committees continue efforts already underway and to study the results of assessment initiatives, such as those generated by the WASC Self-Study, where the issues are within the purview of the Senate.

The Faculty Policies Committee will very soon be coming to some decision about its proposal to change faculty evaluation process, particularly for retention, tenure and promotion. Other Senate committees, Academic Policies, Curriculum Policies, General Education, and Graduate Policies, have been asked to address relevant issues generated from the WASC Self-Study during the Spring 1997 semester. The WASC faculty coordinator will spend the spring semester monitoring these discussions. The more supplemental data relevant to our three WASC Self-Study themes (Chapters 3, 4, 5 in the Self-Study Phase I and II) will be helpful to these committees.

The WASC Self-Study process has set in motion changes that will make available to the next WASC visiting team evidence of student outcomes and learning outcomes in our academic programs and in General Education.