CALIFORNIA STATE UNIVERSITY, SACRAMENTO
Research in Public Policy and Administration
Meeting time and place:
Office hours: Tuesday,
I hear, and I forget;
I see, and I remember;
I do and I understand.
This course focuses on the design of social science research. The main goal is to enhance students' understanding of how best to fashion studies to draw valid inferences. Additionally, PPA 205 aims to promote knowledge of the characteristics and potential pitfalls of various data gathering approaches. As well, PPA 205 is intended to facilitate appropriate choice of quantitative analysis techniques, effective presentation of study results, and ethical conduct of research projects.
While PPA 205 concentrates especially on applications in public policy and politics, it draws from other social science disciplines as well. For example, we will consider the implications of the controversy about the consequences of "low self-esteem" in psychology.
This is not primarily a course on statistical analysis of quantitative information. Another course, PPA 207, serves that purpose. The discussion of statistical techniques in PPA 205 is aimed mainly at enhancing understanding of methods not discussed in depth in the other course, especially those appropriate for analyzing categorical data (e.g. religion). We will focus especially on cross-tabulation, and its relationship to regression analysis. We will also consider the practical reasons for choosing one technique or another.
CONDUCT OF THE COURSE
This course differs from traditional research methods courses in that a relatively heavy emphasis is placed on classroom discussion. I believe it is not enough for students to listen carefully to a lecture; instead, the concepts must be used to analyze real world studies and information. Accordingly, a typical class will begin with assessment of an applied reading or some other means of putting course ideas to work. In the latter part of the class I will draw lessons and further elaborate upon important points from the readings.
Especially because of the emphasis on discussion, it is essential that students come to class having done the week's reading. Students should also be prepared to accept special discussion related assignments, such as leading the analysis of a particular study.
Required course readings include a course packet containing articles and book chapters, as well as the following books.
Richard L. Cole, Introduction to Political Science and Policy Research (New York: St. Martin's Press, 1996).
Jean M. Converse and Stanley Presser, Survey Questions: Handcrafting the Standardized Questionnaire, SageUniversity Paper series on Quantitative Applications in the Social Sciences, 07-063 (Beverly Hills: Sage Publications, 1986).
James A. Davis, The Logic of Causal Order, Sage University Paper series on Quantitative Applications in the Social Sciences, 07-055 (Beverly Hills: Sage Publications, 1985).
Gary King, Robert O. Keohane and Sidney Verba, Designing Social Inquiry: Scientific Inference in Qualitative Research (Princeton: Princeton University Press, 1994).
Mark S. Litwin, How to Measure Survey Reliability and Validity (Thousand Oaks: Sage Publications, 1995)
All readings are available from the Hornet Bookstore.
There will be a mid-term examination and a final examination. The mid-term is scheduled for October 12; the date of the final exam will be forthcoming. Both tests will be taken in the classroom and will be "open book, open note" exams, meaning that you are welcome (indeed, encouraged) to bring to class any course materials you think may be helpful.
PAPERS AND HOMEWORK
There will be two short (1,000 word) individual papers on specific course topics. Detailed guidelines for these assignments will be provided. Additionally, toward the end of the semester there will be a group assignment requiring a class presentation and preparation of a two page (i.e., "short and sweet") memo.
Additionally, there will be two one page homework assignments. Unlike the case for other assignments, only three homework grades will be used. Homework assignments that answer the questions adequately will receive an "A" grade; those with minor errors will receive a "B" grade, while those with significant flaws will receive a "C" grade.
LATE ASSIGNMENTS AND MISSED CLASSES
Late assignments will not be accepted. At my discretion, a student who misses a deadline may be given a make-up assignment. Whether or not a penalty will be assessed depends on the reason (e.g. a family emergency constitutes a good reason; a competing requirement for another course does not).
A student with more than one unexcused absence from class will be penalized one full class participation grade. A student who misses more than three classes for any reason should drop the course.
Grading will be determined as follows:
DETAILED CLASS INFORMATION
Note: In general, applied readings are listed first.
I. DESIGNING SOCIAL INQUIRY
David King, "Notes on the Uniform Distribution" (typescript, 1994)
Cole, Introduction to Political Science and Policy Research, ch. 1
King, Keohane and Verba, Designing Social Inquiry, pp. 1-13
1. What are the key characteristics of social science research?
2. Why did I assign the article by David King?
3. What is the difference between normative and empirical research?
B. THEORIES, HYPOTHESES, AND VARIABLES
Homework Assignment #1 Due
Baumeister, Roy, Laura Smart and Joseph Boden, "Relation of Threatened Egotism to Violence and Aggression: The Dark Side of High Self-Esteem," Psychological Review, 103 (1996), pp. 5-8, 12, 15-20, 26-29
Cole, Introduction to Political Science and Policy Research, ch. 3
King, Keohane and Verba, Designing Social Inquiry, pp. 14-33, 99-114
- What does it mean to have a falsifiable theory?
- Is the theory about the connection between self-esteem and violence falsifiable?
- What is the difference between an experimental and non-experimental research design?
C. THINKING ABOUT CAUSALITY
Kevin B. Smith and Jeremy Eccles, "Buying a Better SAT Score: A Renewed Search for the Elusive Link between Education Expenditures and Outcomes," State and Local Government Review, 30 (1998), pp. 42-51
Davis, The Logic of Causal Order (entire)
- What is the conventional causal model with respect to how teacher salary influences student performance? How does the model advanced by Smith and Eccles differ?
- How can researchers determine the correct causal order for the variables they are examining?
D. MEASUREMENT, RELIABILITY, AND VALIDITY
Paper #1 Due
Cole, Introduction to Political Science and Policy Research, ch. 6
Litwin, How to Measure Survey Reliability and Validity, pp. 1-45
- How can you tell if you have a good measure of some underlying concept?
- What is the difference between validity and reliability?
E. A PROGRAM FOR ENTERING AND ANALYZING SOCIAL SCIENCE DATA: SPSS
Most of this class will be devoted to an overview of how to use SPSS to enter, code, and analyze data. Those who are unfamiliar or uncomfortable with SPSS may want to "play" with the program ahead of class, drawing on the practically oriented guide by Ed Nelson et al., which will be on 2 hour reserve at the library (the Nelson book was written for SPSS version 7.5, but there have been only small changes to the program in the most recent version).
Bruce E. Keith, et al., The Myth of the Independent Voter (Berkeley: University of California Press, 1992), pp. 1-5, 9-22
Litwin, How to Measure Survey Reliability and Validity, pp. 47-58
- What difference does it make whether "leaners" are coded as Democrats/Republicans or independents? Why?
- How should one approach coding decisions?
II. DATA COLLECTION
A. FINDING AND USING ARCHIVAL DATA
Web Sites to Visit
Come to class having perused the web sites for the Social Science Data Base Archive (SSDBA) at CSU Los Angeles, and the Inter-University Consortium on Political and Social Research (ICPSR) at the University of Michigan:
Ted Lascher, "The Moderately Skilled Layperson's Guide to Using the CSU System's Social Science Databases," typescript, 1998
Cole, Introduction to Political Science and Policy Research, ch. 5
- What kind of social science data sets are widely available to people within the academic community?
- How would you go about finding and obtaining a relevant social science data set?
Note: The class time remaining after the exam will be devoted to preparation for the group assignment due on November 30
B. INDIVIDUAL VERSUS AGGREGATE LEVEL DATA
Deborah Franklin, MPPA thesis, selections (forthcoming)
William S. Robinson, "Ecological Correlations and the Behavior of Individuals," American Sociological Review, 15 (1950), 351-357
Cole, Introduction to Political Science and Policy Research, 237-238
- What exactly is the difference between individual and aggregate level data? Why is the distinction important?
- Why might one use aggregate data to draw inferences about individuals? What should you be concerned about in doing so?
C. CONDUCTING SURVEYS: THE BASICS
Converse and Presser, Survey Questions, pp. 9-31
Richard Seltzer, Mistakes that Social Scientists Make (1996), pp. 90-100
- What common problems face people who construct surveys? What can "go wrong?"
- How can surveys be made conceptually clear?
C. ASSESSING SURVEY DESIGN
Paper #2 Due
Converse and Presser, Survey Questions, pp. 31-end
Litwin, How to Measure Survey Reliability and Validity, pp. 59-71
Come to class prepared to discuss your paper.
D. CASE STUDIES
Lyn Kathlene, "Power and Influence in State Legislative Policymaking: The Interaction of Gender and Position in Committee Hearing Debates," American Political Science Review, 88 (1994), pp. 560-576
Edward L. Lascher, Jr., "Loss Imposition and Institutional Characteristics: Learning from Automobile Insurance Reform in North America," Canadian Journal of Political Science, 30 (1998), pp. 143-164
King, Keohane and Verba, Designing Social Inquiry, chs. 4, 6
- How convincing are the inferences Lyn Kathlene draws?
- In attempting to determine the influence of governmental systems, why is it appropriate to focus on American states and Canadian provinces?
- What is selection bias? How should the researcher address potential selection bias?
- How does one "maximize leverage" (a la King, Keohane and Verba)?
III. DATA ANALYSIS
A. DATA ANALYSIS: THE BASICS
Homework Assignment #2 due
Cole, Introduction to Political Science and Policy Research, chs. 8-9
- What is the difference between statistical significance and the strength of a relationship?
- How do the chi-square, Cramer�s V, and Lambda statistics differ?
B. DATA ANALYSIS: MULTIPLE VARIABLES AND THE LOGIC OF ELABORATION
David Reynen, MPPA thesis, selection (forthcoming)
Cole, ch. 10
- What is the difference between a spurious effect and an intervening effect?
- What techniques can be used to control for the effects of different independent variables on a dependent variable?
- What is the logic of elaboration?
IV. PRESENTING RESULTS/ETHICS OF SOCIAL SCIENCE RESEARCH
A. PRESENTING RESULTS
Group Memo Due; Group Presentations in Class
Robert Putnam, "The Strange Disappearance of Civic America,"The American Prospect (Winter 1996), pp. 34-48
Cole, Introduction to Political Science and Policy Research, ch. 11
What are the characteristics of an effective presentation using social science data?
B. ETHICAL ISSUES/COURSE WRAP-UP
Earl Babbie, Survey Research Methods (1973), pp. 347-357, 362-364
What are researchers' ethical obligations to the subjects of their research? To their discipline and colleagues?
To the larger public?