Academic ProgramsTeaching and LearningResearch and Creative ActivitiesUniversity and Community ServicesSpecial Accomplishments

Introduction

Mission

Curriculum

Assessment Plan

Learning Goals

Data

Self-Study

Collective Portfolio Assessment

Assessment rests on two legs: student perceptions of their experience and actual student work of graduating seniors. This is why the department’s assessment plan also includes samples of student work in a "Collective Portfolio." This portfolio is currently being constructed. In the near future this site will present an analysis of the quality of student work of graduating seniors.

The Collective Portfolio is composed of a sample of papers from majors in all of the core, required upper division courses in the major. Each year, the Assessment Committee solicits samples of A, B, and C work from faculty who teach these courses. The selected papers are then put into the department’s "Collective Portfolio." The papers reviewed are from the academic year 1998-1999. These papers represent the first set of papers collected in the portfolio. The Sociology Department’s Committee on Assessment decided to initially look at the papers in the Department’s Collective Portfolio by focusing on only two of the department’s eleven learning goals. The Committee selected only two goals to start with, since it has no track record of evaluating papers in a collective portfolio. The thought was to start slowly to see how the assessment of actual student papers might work and what could be learned from doing this exercise. In future years, the department will select other learning goals for analysis in the Collective Portfolio. The two selected goals represent a disciplinary goal (#5) and a general education, university goal (#11). As the Committee read the papers, the members focused on the patterns that were present in the students’ work.

Learning Goals Assessed in the Portfolio

The two learning goals selected for this initial assessment –
(#5) The sociology major at CSU Sacramento will be expected to study, review, and reflect on basic concepts in sociology and their fundamental theoretical interrelations. Under this learning goal, the student should be able to define, give examples, and demonstrate the relevance of concepts such as, culture, socialization, stratification, social structure, institutions, differentiations by race/ethnicity, gender, age, and social class, etc.

(#11) The sociology major at CSU Sacramento will be expected to have the writing skills necessary to communicate effectively with persons whom they encounter in their work, civil obligation, and personal life. Under this goal, the student should be able to a) write a clear and concise report of the findings from sociological data analysis; and b) write a clear and concise sociological analysis and account of a social event, topic, issue, or problem.

Assessment Categories or Rubrics

Before reading the papers, the Committee decided to use a simple set of categories to start with. Over time, these categories will probably be expanded and become more focused. The assessment categories used for the initial assessment are –

a. What do sociology majors do that is outstanding?
b. What do sociology majors do that is satisfactory?
c. What do sociology majors do that needs improvement?
d. What might the department do to help students improve?

Assessment Results

a. Outstanding Work

For the learning goal on writing a research report, a few students did outstanding work. These students were able to elaborate their ideas and present a more substantial account both of the concepts they were using as well as the literature review. For example, in the literature review, students were able to go beyond merely describing a published piece of research as it relates to their topic, but were also able to criticize the research they were reviewing. In other words, they went beyond description to critical thinking about the literature This is an outstanding accomplishment for an undergraduate. Few of our students are able to do this, in part because it requires a good deal of knowledge about the topic and previous research done on the topic. These papers also show the ability to move between levels of analysis in sociology–between the micro and macro, between the self and social structure, and so forth.

The outstanding papers were also organized very well. The paragraphs were logically ordered, there were transitions between paragraphs, each paragraph had a thesis sentence, and paragraphs covered only one topic. Less complex sentences tended to be used, which made the writing more clear and concise. Literature reviews, library research, and citations were done well. They were complete, up-to-date, and related to the topic and thesis. These papers reflect precision in the use of sociological concepts too. The concepts were stated explicitly and defined reasonably well, and these papers provided clear empirical referents for the concepts used. In other words, the authors provide an empirical account of what the concept means. The outstanding work represents the ability to move from the descriptive level to the analytical level.

b. Satisfactory Work

For the leaning goal on writing a research report, most students performed satisfactorily. They were able to communicate their ideas and write a report based on findings from sociological data analysis. Most cited current literature, most used proper citation methods, most were able to explain the independent and dependent and control variables in their research paper (the research design), and most presented clear conclusions. Even so, satisfactory work often contains errors in writing, but they tend to be minor and don’t interfere with communication of the topic or ideas. In addition, the satisfactory papers were organized in terms of defining the research question, the literature review, research design, data analysis, conclusions, and shortcomings of the study. In brief, they were able to follow the acceptable organization for writing a research paper. This is what we would expect given their course work in research methods.

For the learning goal on the application and understanding of basic concepts, most students were able to adequately describe the empirical world of their observations, but were less able to clearly link those observations to a clearly presented and concisely defined sociological concept. In addition, in some of the papers that were satisfactory, the student tried to cover too many concepts and consequently, failed to provide either a clear conceptualization of the concept or link the concept to the empirical world of their observations. When there are too many, some of them are covered superficially. The satisfactory work is represented by good, sometimes excellent work at the descriptive level, detailing observations in the natural laboratory of society. This work is weaker at the analytical or conceptual level. The student authors don’t move easily from the descriptive to the analytical level.

However, satisfactory work often does show that the student has learned to think sociologically, to use the sociological imagination. These papers do show that the student has taken empirical observations and reflected on them through the eyes of sociological analysis, even though the examples and the writing don’t reveal this as well as they could. An example of this is a satisfactory paper which involved observing an African American church ceremony. Although the scripture and the sermon were about the "brotherhood of Christ," the student was observing the inequalities in the church from the point of view of the power structure within the church. This student was able to give some examples that illustrated the concept of the power structure, and the student did discover that she was being a sociologist, seeing the event, at least in part, as a sociologist would. But there could have been more empirical examples and the concept of power structure could have been more adequately conceptualized.

c. Work that Needs Improvement

For the learning goal on writing, some sociology majors do not write as well as they should or could. They often do not have good transitions between paragraphs, cover more than one topic in a paragraph, have some grammatical problems (wrong verb tense). Some sentences can be too complex and tangled, so that the idea conveyed is not clear. Citations are also not consistently done in the research report. Sometimes the thesis of the paper is not stated clearly. Conclusions are not complete. Sometimes there are factually incorrect statements. Subheadings in the paper tend not to be present, which leads to loose organization of the ideas. One of the characteristics of student work that needs improvement is that it tends to be superficial. There is not a deep account of the topic or the research problem or the literature review. Sometimes the problem is not stated too clearly or is too oversimplified. The work tends to skim the surface, although the authors do have basic ability at following the design for writing a research report.

For the goal of understanding and applying sociological concepts, some of the sociology majors do not have a clear idea or understanding of the concept they are writing about. Hence, their writing tends to lack precision. Their use of concepts is too broad, not precise. The concepts tend to be "spoken about" not carefully defined or conceptualized. They tend to be confused about the meaning, or implicitly think they know the meaning without making the meaning explicit, or keep hidden in their writing what they do know about the meaning of the concept. As a consequence, their work is often at the descriptive level, pointing out what they observed, saw, heard, and so forth. Even at the descriptive level their accounts tend to be too brief and superficial.

d. What can be done to improve the student’s work?

One way sociology faculty may be able to improve the writing of students who do not write well is to try to have them write less complicated, shorter sentences. Assignments that are longer, need to require students to use subheadings within the paper that might help them to organize their ideas more clearly and coherently.

Even the satisfactory work of students could be improved on this point.
One way to improve students work with sociological concepts is to provide them with opportunities ( several assignments) that require them to provide clear empirical referents and examples for the concepts. If students can point to the empirical world and use characteristics of that world as a way to understand abstract concepts, they would be in a better position to both understand the concept more clearly and to write about it more concisely. Another way to possibly improve students’ ability to use and apply sociological concepts is to provide assignments that are specifically structured. The assignments may need to be shorter so that more of them can be required. An example of such an assignment may require the student to take one concept, go to the literature to look up various conceptualizations of the concept, choose one definition and then show explicitly how the empirical observations the student is making can be seen through the concept. If students had several of these briefer assignments, they might improve their ability to move from observation to analysis.

Still another way to help student improve their work writing and applying sociological concepts is to provide them with opportunities for writing and then revising and rewriting. This would have the benefit of allowing students to see faculty or graduate assistant feedback and then learn from that feedback in the revision. This would provide students with practice and with the possibility of receiving help. If students had more opportunities for doing this, they would also learn how difficult it is to do this well. They would also have the chance to see themselves improving as they do the assignments. Providing more opportunity for our students to write and revise should help them move from description to analysis with more clarity.

The department may need to see if there would be resources to hire some of its excellent graduate students or one of the part-time faculty to help faculty who teach the core courses–especially sociology 150, 125, 193 and 194–with the writing and rewriting options. Faculty in the research methods courses already have most students write and rewrite parts of their research projects over the course of two semesters. This is possible in those courses because the department attempts to limit enrollment to no more than 25 in each section.

Summary Reflections on the Collective Portfolio

The process of assessment is still under review as much as the student learning the department is seeking to assess. How we arrive at our judgments, the criteria we use to assess the collective portfolio, and the committee’s suggestions for improving how we teach and how students might learn better are still open for discussion and negotiation. Still, the process itself, however imperfect, has moved us an important step forward.

The biggest problem our students face in reaching the two learning goals assessed in the collective portfolio must be dealt with on many levels and will require patience and reason. It is clear, nonetheless, that problems with writing and discipline specific analytical reasoning will require resources. These include, but are not limited to, faculty time necessary to integrate and effectively evaluate written analytical projects, especially in the core courses with larger enrollments.

Regardless of resources available for implementing pedagogical and curriculum changes, a major issue that will have to be considered is how we address these changes as a department and not as individual faculty members. This collective effort will have to be sensitive to the intellectual integrity of individual faculty and also to the fact that there are many different ways to effectively teach the discipline of sociology. The worst thing we could do is to force one approach over the other.

Finally, this is the first set of papers in the first department collective portfolio. Over time the department will be able to see more clearly trends in patterns in student work. The results of this initial analysis may, in part, reflect the sample of papers from 1998-1999 and not recurring patterns over a period of time.

 

 

 

 

 

 

 

 

California State University, Sacramento