Domain Descriptions

Domain 1: Research Methodology & Data Analysis

General Description

Individuals seeking the PhD in the Department of Educational Psychology are expected eventually to be able to discover new knowledge. In the interim, they will be expected to evaluate critically the efforts of those who claim to have discovered new knowledge. The discovery of new knowledge can be modestly viewed as a reduction in uncertainty about what to believe. Domain 1 deals with a collection of ideas, concepts, and skills that seekers of new knowledge frequently use to reduce their own uncertainty and/or to justify their conclusions to others. In general, the content of Domain 1 covers:

The planning for and acquisition of data relevant to a problem where uncertainty exists.
The selection and defense of a model that provides an arguable justification for proceeding from observed data to conclusions.
Reduction of the observed data to summary indicators and the use of inferential statistics to make probability statements about conclusions based on the indicators.

Courses

Currently, the only primary course that will satisfy the Domain 1 requirement is Experimental Design and Statistical Inference. Enrollment in this course requires knowledge equivalent to that obtained in a one-semester introduction to statistics course (EDP 371 or EDP 380E.1)

Numerous secondary courses satisfy the rest of the Domain 1 requirement. (It should be noted that the three courses identified as “Secondary—1 or 2” will satisfy either the Domain 1 or the Domain 2 requirement, but not both.)

Detailed statements of the contents, objectives, and assigned reading for each course can be examined in the departmental office (SZB 504) or on the course descriptions web page. Additional information can also be obtained from the instructor who teaches the course.

P/S	EDP #	Topic	Usual Instructor
Primary	482K.1	Experimental Design & Statistical Inference	Borich, Pituch, Emmer
Secondary 1 or 2	380P.4	Evaluation Models & Techniques	Borich
Secondary	382K.2	Correlation & Regression	Koch, Beretvas
Secondary 1 or 2	382K.3	Factor Analysis	Beretvas
Secondary	382K.4	Survey of Multivariate Method	Koch
Secondary	384	Educational Research Methodology	Dodd
Secondary 1 or 2	384.4	Introduction to Survey Research	Beretvas
Secondary	394.8	Qualitative Research Methods	Emmer

Topics

Brief descriptions of the topics covered in Domain 1 courses are given below. Although redundancies occur among the topics listed in the descriptions, they have been retained in order to allow students to gain a better understanding of the relationships among some of the topics.

Correlation and Regression Methods. Topics covered in this course include: Review of correlation topics in introductory course; two-variable linear regression theory; correlation ratio, standard error of estimate, test for linearity of regression, and relation of correlation ratio to analysis of variance; relationship between byx and ryx, coefficients of alienation, nondetermination, and significance tests for b and r; biserial correlation, point-biserial correlation, and the phi coefficient; tetrachoric correlation, rank correlation (and tau), and the G index of agreement; Fisher’s Z and significance test for r, differences between r’s, and effect size; partial correlation, restriction-in-range, and spurious correlation; three-variable R and principles of R; coefficient of concordance (and its relation to analysis of variance) and intraclass correlation; R with more than 3 variables; and testing models.

Educational Research Methodology. This course is an introduction to educational research methodology. Topics covered in this course include: the steps in the research process: identifying the research problem, statement of problem and research questions, quantitative studies, qualitative studies, and research reports. The course also covers the following research designs: experimental, correlational, survey research, grounded theory, and mixed. Students are expected to develop a research proposal to address a research question in their own field of study by the end of the course. Prerequisites: EDP 482K or equivalent.

Evaluation Models and Techniques. This course traces the contemporary development of evaluation in education and human service and identifies milestones and existing directions. The relationship between research and evaluation, as well as identification of distinguishing and overlapping characteristics is also included. Students are required to present and synthesize evaluation strategies and concepts; identify practical and theoretical models for evaluation; provide a description of evaluation methodologies; identify and apply quantitative and qualitative tools for evaluation; present critical considerations on evaluation design, criteria for judging evaluations, and measurement problems related to evaluation; and distinguish adequate from inadequate evaluation designs. Topics include: parallels and contrasts between research and evaluation, evaluation models and approaches, behavioral objectives, needs assessment techniques, program modeling and decomposition, evaluation methodology, outcome evaluation, and evaluation instruments.

Experimental Design and Statistical Inference. Topics covered in this course include experimental, non-experimental, and quasi-experimental designs; sampling distributions; randomization and random sampling; normal, t, and F distributions; hypothesis tests; types of errors; power; hypotheses about single means and pairs of means; the analysis of variance for designs with one or more variables of classification; random effects and mixed models; comparisons among means; randomized block designs; designs with repeated measures , including split-plot designs; multiple regression; testing hypotheses about R2 and R2 differences; relationships between analysis of variance and multiple regression analysis; and analysis of covariance.

Factor Analysis. Course content includes: General introduction, factor analysis models, elementary matrix algebra, rotation of axes, and vector representation. Basic formulas of common factor analysis and principal components analysis. Test for significance of correlation matrix, criteria for number of factors, principal-axis, image analysis, and alpha methods of factoring; rotation models, simple structure, orthogonal and oblique axes, transformation matrix, graphical rotation; criteria for analytical orthogonal and oblique rotation (including Procrustes); interpretation of factors, factor scores, computer programs; and confirmatory factor analysis including higher order and hierarchical models, tests of model fit.

Introduction to Survey Research. This course is an overview of the survey research process, addressing various topics in survey research within the context of the development of a survey study. Accordingly, topics are sequenced to follow the creation of a survey study, proceeding from conceptualization, to measurement, to administration, to data management and analysis, and finally, to writing the research report. Issues in quasi-experimental design, sampling, questionnaire construction, administration, and the use of computers in survey research are covered within this framework. The relative advantages, disadvantages, and appropriateness of the different modes of survey administration (i.e., face-to-face interviews; telephone interviews; mail and other self-administered questionnaires) for different research questions are also examined.

Meta-Analysis. Requirements for this course include a miniature primary research study, a small meta-analysis project, and a short-essay final exam. Topics covered include recent developments in the synthesis of quantitative research findings from multiple studies. Omnibus tests for combining probabilities are covered, but the main focus is on effect size measures: standardized mean differences and correlations. Effect size variation and the influence of moderator variables are also considered in depth. Corrections for unreliability and a small library of computer programs are also discussed. Students present their own meta-analyses in the last two class periods.

Qualitative Research Methods. This course examines research methods that are descriptive, field-based, interpretive, and discovery-focused, in contrast to methods that use quantitative summaries of data in order to test null hypotheses. Topics covered include varieties of qualitative research (emphasizing grounded theory, but also including case studies, ethnography, phenomenology), identifying questions and phenomena for research, planning and conducting qualitative research, coding and other analytic procedures, developing an interpretation, and trustworthiness issues in qualitative inquiry.

Survey of Multivariate Methods. The content of this course may vary slightly depending on the instructor, but topics usually covered include: Fundamentals of vector and matrix algebra needed for multivariate analysis, including notation, operations, rank, inverse, and solving equations; multiple regression, including tests of hypotheses, coded predictors, relationship to ANOVA and ANCOVA; use of the multivariate normal distribution; Hotelling’s T2; multivariate analysis of variance; the Wilks, Rao, and other criteria for significance of differences among groups; post hoc comparisons; discriminant function analysis; classification functions; canonical correlation; principal components analysis; and introduction to factor analysis.

Domain Exam

For students who elect this avenue for fulfilling the Domain 1 requirement, the three-hour domain examination consists of two parts of equal length. The first half is required of all students and covers the content of the primary course in Domain 1, EDP 482K.1: Experimental Design & Statistical Inference. It consists of 60 objective items and one essay item.

In the second part of the exam, the student chooses to answer questions pertaining to the topics covered in one secondary course in Domain 1. This second part of the exam consists of several sets of questions, each set covering topics included in one of the secondary courses in Domain 1. The student chooses one set and responds only to those questions. There are both objective and essay items in each of these sets.

Each essay response is evaluated independently by two readers. Each reader then considers the scores on both the objective items and the essay items and recommends either Pass or Fail. If the Pass/Fail recommendations of the two readers differ, a third reader will evaluate the student’s responses and resolve the discrepancy. Students may not retake the exam if they do not pass: a student who fails the examination must satisfy the domain requirement by coursework.

Readings

The following listing includes texts required or recommended for the Domain 1 primary and secondary courses listed above:

Babbie, E. (1990). Survey research methods (2nd ed.). Belmont, CA: Wadsworth.

Borich, G. (1982). Programs and systems: An evaluation perspective. New York: Academic Press.

Cattell, R. B. (1978). The scientific use of factor analysis in behavioral and life sciences. Plenum Press.

Converse, J.M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire. Beverly Hills, CA: Sage.

Cooper, H., & Hedges, L. V. (1994). Handbook of Research Synthesis. New York: Russell Sage Foundation.

Edwards, A.E. (1984). An introduction to linear regression and correlation (2nd ed.). New York: Freeman.

Glass, G. & Hopkins, K. (1996). Statistical methods in education and psychology (3rd ed.). Boston: Allyn & Bacon.

Gock, R.D. Multivariate statistical methods in behavioral research. McGraw-Hill.

Gorsuch, R. L. (1983) Factor analysis (2nd ed.). Lawrence Erlbaum Associates.

Guilford, J.P., & Fruchter, B. (1978). Fundamental statistics in psychology & education (6th ed.). McGraw-Hill.

Hays, W. (1994). Statistics (5th ed.). New York: Holt, Rinehart, & Winston.

Hempel, C. Philosophy of natural science. Prentice-Hall.

Jaeger, R. (1984). Sampling in education and the social sciences.

Kalton, G. (1983). Introduction to survey sampling. Beverly Hills, CA: Sage.

Kirk, R. (1995). Experimental design (3rd ed.). Belmont, CA: Brooks/Cole.

Pedhazur, E.J. (1982). Multiple regression in behavioral research. New York: Holt, Rinehart, & Winston.

Rummel, R.J. (1970). Applied factor analysis. Northwestern University Press.

Stevens, J. (1992). Applied multivariate statistics for the social sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Thorndike, R.M. (1978). Correlation procedures for research. New York: Gardner Press, Inc.

Ward, J. & Jennings, E. (1973). Introduction to linear models. Prentice-Hall.

Weber, R. P. (1990). Basic content analysis (2nd ed.). Beverly Hills, CA: Sage.

Wickens, T.D. (1989). Multiway contingency analysis for the social sciences. Hillsdale, NJ: Earlbaum.