Research Methods in Education

Research Methods in Education

2

 

 

For my wife Angela, my loving supporter and most honest advisor, always.

—Joseph Check

To my wife, Elizabeth.

—Russell Schutt

3

 

 

Research Methods in Education

Joseph CHECK Russell K. SCHUTT

University of Massachusetts Boston

4

 

 

FOR INFORMATION:

SAGE Publications, Inc.

2455 Teller Road

Thousand Oaks, California 91320

E-mail: order@sagepub.com

SAGE Publications Ltd.

1 Oliver’s Yard

55 City Road

London EC1Y 1SP

United Kingdom

SAGE Publications India Pvt. Ltd.

B 1/I 1 Mohan Cooperative Industrial Area

Mathura Road, New Delhi 110 044

India

SAGE Publications Asia-Pacific Pte. Ltd.

33 Pekin Street #02-01

Far East Square

Singapore 048763

Copyright © 2012 by SAGE Publications, Inc.

All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Printed in the United States of America

Library of Congress Cataloging-in-Publication Data

Check, Joseph W., 1947-

Research methods in education / Joseph Check, Russell K. Schutt.

p. cm.

Includes bibliographical references and index.

ISBN 978-1-4129-4009-2 (pbk.)

1. Education—Research—Methodology. I. Schutt, Russell K. II. Title.

LB1028.C523 2012

370.72—dc232011032014

This book is printed on acid-free paper.

11 12 13 14 15 10 9 8 7 6 5 4 3 2 1

Acquisitions Editor: Jerry Westby

Editorial Assistant: Erim Sarbuland

Production Editor: Catherine M. Chilton

5

 

 

Copy Editor: Gillian Dickens

Typesetter: C&M Digitals (P) Ltd.

Proofreader: Annette R. Van Deusen

Indexer: Sheila Bodell

Cover Designer: Candice Harman

Marketing Manager: Katherine Winter

Permissions: Karen Ehrmann

6

 

 

Detailed Contents

Preface Acknowledgments Part I. Foundations of Research

1. Science, Schooling, and Educational Research 2. The Process and Problems of Educational Research 3. Ethics in Research 4. Conceptualization and Measurement 5. Sampling

Part II. Research Design and Data Collection 6. Causation and Research Design 7. Evaluation Research 8. Survey Research 9. Qualitative Methods: Observing, Participating, Listening 10. Single-Subject Design 11. Mixing and Comparing Methods and Studies 12. Teacher Research and Action Research

Part III. Analyzing and Reporting Data 13. Quantitative Data Analysis 14. Qualitative Data Analysis 15. Proposing and Reporting Research

Appendices Glossary References Author Index Subject Index About the Authors

7

 

 

Preface

This is an exciting time to begin learning about educational research. For many years, the study of education was slow to change and regularly criticized—by practitioners for not being practical enough, by researchers from other disciplines for not being theoretical enough (Lagemann, 2000). Research that met rigorous “scientific” standards was accused of staying on library shelves and being of little use to teachers. Research that documented the daily life of students and teachers was derided as mere storytelling and “not scientific enough” (Walters & Lareau, 2009, pp. 1–2). Overall, education research was characterized by “continuing contests among different groups, especially scholars of education, scholars in other fields and disciplines, school administrators, and teachers” (Lagemann, 1997, p. 5).

Despite criticisms and contention, in the early years of the new century—the 2000s—it became clear that things were changing, that we were seeing “an explosion of new methodologies and approaches to inquiry” (Biesta & Burbules, 2003, p. 1) that could make educational research an evolving and exciting field, both practical and theoretical, both rigorous and context sensitive.

Educational research today is marked by three qualities. It is eclectic, dynamic, and essential. By eclectic, we mean that many disciplines and schools of thought contribute to educational research. By dynamic, we mean rapidly evolving, with new philosophies and methods emerging on a regular basis. By essential, we mean necessary, and recognized as necessary, for the optimal development of schooling.

8

 

 

Electic

Eclectic means coming from many sources. Because learning is affected both by schooling and by factors outside schooling, educational research must be interdisciplinary, meaning that it involves more than one academic discipline or field of study. An “educational researcher” might be a professor of education, a sociologist, an anthropologist, an economist, a psychologist, or a medical doctor. In some forms of educational research, teachers are themselves researchers of their own classrooms.

Many common techniques in educational research originated outside the field of education. Anthropology gave us ethnography, the study of culture and cultural processes, now used to study classrooms and the culture of schools. Sociology gave us survey research; survey techniques are now used throughout education to study large populations of teachers and students. Psychology gave us tools for understanding the intellectual and emotional development of children. Quantitative disciplines such as mathematics and economics contribute multiple tools based on statistical analysis, including recent work in areas such as effect sizes, which measure the strength of relationships between variables. Many academic disciplines have scholarly journals that deal specifically with the intersection between their discipline and education, including Anthropology and Education, Education Economics, Sociology of Education, and the Journal of Educational Psychology.

9

 

 

Dynamic

Educational research is a continually developing field. New theoretical approaches and data-gathering techniques emerge regularly. This is due in part to education’s dynamic relationship with other fields of study, mentioned above. Examples of techniques that have been developed or refined in recent years include multilevel analysis and narrative inquiry. Multilevel analysis is an advanced statistical technique that enables a researcher to accurately analyze “nested” or “hierarchical” data sets at different levels, such as an individual student, a classroom of students, and a whole school (Hox, 2010). Narrative inquiry concentrates on collecting and telling the life stories of a single individual (Clandinin, Pushor, & Orr, 2007). A second kind of dynamic tension is created by the current federal push for scientifically based research and the simultaneous questioning of what that means in a naturalistic setting such as a classroom, particularly in classrooms with culturally diverse and disadvantaged populations.

Education researchers also have their own energetic and up-to-date professional organization, the American Educational Research Association (AERA), which provides a platform for discussion and new inquiry techniques and is open to participation from anyone involved in educational inquiry, whatever their philosophical or methodological approach.

10

 

 

Essential

Accountability-based school reform efforts, particularly federal and state legislation, have made test scores, dropout rates, and other measurements an essential component of life in schools. Regular data collection, one of the key elements of research, is now the norm. New professional standards in fields such as school counseling and special needs teaching stress the importance of evidence-based practice. In addition, the growing popularity of formats such as teacher research and action research means that teachers are no longer just subjects of research conducted by others. Classroom teachers now have the tools to become researchers of their own practice and change agents in their own buildings. Taken together, these developments make research an essential dimension of life in schools today, both for the students, teachers, and administrators inside them and the parents, legislators, and policy makers outside.

At its heart, research starts with questions; every chapter of this book begins with a research question. Examples of published research studies related to the opening question are then used as examples throughout the chapter. By the end of the book, you will have encountered more than 30 published research studies that take different theoretical approaches and use many different methods to gather and interpret data.

11

 

 

Acknowledgments

Our thanks to Jerry Westby, Acquisitions Editor, SAGE Publications. Jerry has encouraged and supported this project continuously from the idea stage through final production. His patience, confidence, and human touch have smoothed the rough spots and helped us always keep the final goal in view. No author could ask for a better publisher.

We also thank the personable and highly creative SAGE professionals who turned a typescript into a scrupulously edited, highly attractive book: editor Denise Simon, production manager Catherine Chilton, copy editor Gillian Dickens, marketing manager Katharine Winter, associate editor Megan Krattli, editorial assistant Erim Sarbuland, and assistant editor Rachael Leblond. Kathryn Stoeckert Sabella did a superb job of developing the online exercises, and Gina Cook and Michelle Turner Mangan developed the fine set of ancillary materials.

We feel gratitude and friendship for the unique, cross-disciplinary support group of SAGE authors of related methods texts who have extended their wisdom and expertise: Ronet Bachman (University of Delaware), Dan Chambliss (Hamilton College), Ray Engel (University of Pittsburgh), and Paul Nestor (University of Massachusetts, Boston).

We are also indebted to the first-rate group of reviewers whose thoughtful suggestions and cogent insights have helped improve every chapter. They are Kimberly Alkins, Queens College; Maureen Angell, Illinois State University; Scott Bauer, George Mason University; Calvin Brown, Tennessee State University; Elaine Bukowiecki, Bridgewater State College; Laurie Carlson, Colorado State University; Dick M. Carpenter II, University of Colorado; Stephen E. Cramer, University of Georgia; Patrick Dilley, Southern Illinois University; Paul Erickson, Eastern Kentucky University; Chester Fuller, Central Michigan University; Gerry Giordano, University of North Florida; Carol Ann Gittens, Santa Clara University; Kristine Hogarty, University of South Florida; John Huss, Northern Kentucky University; Richard M. Jacobs, Villanova University; Lori Kim, California State University, Los Angeles; Lydia Kyei-Blankson, Illinois State University; Marie Kraska, Auburn University; Maria K. E. Lahman, University of Northern Colorado; Brian Lawler, California State University San Marcos; Alar Lipping, Northern Kentucky University; Jeffrey Lorentz, University of Houston, Clear Lake; Crystal Machado, Indiana University of Pennsylvania; Jerry R. McMurtry, University of Idaho; Steven Miller, Loyola University Chicago; Ralph Mueller, George Washington University; Victor Nolet, Western Washington University; Jeffrey Oescher, Southeastern Louisiana University; Gary W. Ritter, University of Arkansas; Charles Thomas, George Mason University; Jacqueline Waggoner, University of Portland; Scott Walter, Washington State University; Carol Wickstrom, University of North Texas; Diane Wilcox, James Madison University; and Nadia Zabtcheva, Montclair State University.

Joe thanks Russ greatly for the opportunity to co-create this book and for his constant concern not just for high-quality content but for straightforward writing to make the content accessible to students. Joe also

12

 

 

thanks Karen Daniels and Margo Moore, the two hard-working graduate assistants who helped with this project; departmental administrative staffer Molly Pedriali, who was always needed and always there; his

faculty colleagues, for their invaluable expertise not just as researchers but as teachers; and his students, on whom many of the approaches in this book were tried out. Finally, Joe thanks his wife Angela for her constant and unstinting encouragement and support, both emotional and editorial, and his children, Joe and Pietra, for their suggestions and support, and for enriching his life just by being who they are.

Russ thanks Joe for his skill, dedication, and good cheer throughout this project, and he thanks his wife Elizabeth for her love and support, and his daughter Julia, for the joy she brings to his life and for all that she contributes to the social world.

13

 

 

Part I Foundations of Research

14

 

 

Chapter 1 Science, Schooling and Educational Research

Research Question: How Do Early Childhood Experiences Affect Schooling?

15

 

 

Chapter Contents

Learning About the Educational World The Educational Research Approach Educational Research Philosophies

Can watching too much television hurt toddlers, putting some children at a disadvantage before they enter preschool? Are preschool children whose mothers work outside the home academically behind when they go to school? Understanding early childhood influences is a key task for educational researchers and forms the research focus for this chapter. Dimitri A. Christakis, Frederick J. Zimmerman, David L. DiGiuseppe, and Carolyn A. McCarty (2004, pp. 708–713) studied more than 1,200 children and found that 1- and 3-year- olds who watched more than 3 hours of television daily had a substantially increased risk of developing attention problems by age 7. This was one of the first large-scale, systematic investigations that linked difficulty concentrating, restlessness, and impulsive behavior to watching television as a young child.

The Christakis et al. (2004) study appeared in the medical journal Pediatrics, yet it deals with an educational question that parents and early childhood teachers face every day. The study took a scientific, research-based approach to a complex, emotional question: Does too much TV harm very young children?

Have you thought about TV’s effects on your own learning or that of children you have contact with? No one wants to feel that children are threatened by television, yet TV is widely used as an “electronic babysitter.” In this chapter, you will learn how the Christakis et al. (2004) study and other, more recent investigations are helping to answer important questions about early childhood learning.

16

 

 

Learning about the Educational World

Just one research question about learning raises so many more questions about education. Take a few minutes to read each of the following questions and jot down your answers. Don’t worry about your responses: This is not a test; there are no “wrong” answers.

1. Do you think you watched too much television as a child? 2. How has television watching affected your learning? 3. How many hours of television does the average American child watch every day? 4. Does the content of what young children watch make a difference? Is educational programming such as

Sesame Street better for a 3-year-old than Saturday cartoons? 5. How much does quality of early child care in general affect learning? 6. Most television viewing takes place outside school, but its effects show up in the classroom. How do you

think other forces outside school affect how children will perform in school?

We’ll bet you didn’t have too much trouble answering the first two questions, about your own experiences, but what about the others? These four questions concern “the educational world”—the educational experiences and orientations of people in addition to ourselves. Usually, when we answer such questions, we combine information from many different sources. We may also recognize that our answers to the last four questions will be shaped by our answers to the first two—that is, what we think about the educational world will be shaped by our own experiences and by the ways we have learned about the experiences of others. Of course, this means that different people will often come up with different answers to the same questions. Studying research methods will help you learn what criteria to apply when evaluating these different answers and what methods to use when seeking to develop your own answers.

Take a bit of time in class and share your answers to the six questions. Why do some of your answers differ from those of your classmates? Now, let’s compare your answers to Questions 3 through 6 to the findings of researchers using educational research methods.

Question 3: Preschool children in general watch anywhere from 0 to 30 hours of television a week, but the average is more than 4 hours per day. Four-year-olds average 50 to 70 minutes of viewing daily, most of which is cartoons (Huston, Wright, Rice, Kerkman, & St. Peters, 1990).

Exhibit 1.1 shows the amount of television watched by 18-month-olds in the Christakis et al. (2004) study mentioned earlier. Notice that the highest bar is at the zero level—no TV watching—and that more than half of the children studied watched 0 to 2 hours per day of television.

Question 4: Television viewing at an early age affects learning in school at a later age, but the connection is not a simple one. Important considerations include the type of programs viewed (commercial or educational), whether parents talk with children about what they’re seeing, and how well children understand the difference between cartoon-type fantasies and real life (Peters & Blumberg, 2002).

17

 

 

Question 5: Conditions of child care, particularly the amount and quality of adult attention, can make a large difference in how television viewing affects later learning. Researchers at the National Institute of Child Health and Human Development (2002) studied the effects of early child care on more than 1,000 children. The study found that “children’s development was predicted by early child-care experience” (p. 133). Educational development was connected to quantity, quality, and type of child care just prior to the child going to school. Higher quality care was associated with better language skills for 4½-year-olds, and after a certain point, the more hours children spent in care, the more behavior problems they showed.

Exhibit 1.1 Hours of Television Viewing for 18-Month-Old Children

Source: Christakis et al. (2004, p. 711).

Question 6: Many factors outside school affect learning in school. Physical and cognitive disabilities, for example, affect in-school learning for many children. Later in this chapter, you will learn of research on a legally blind preschool child in a classroom with 13 physically normal students. You will also learn of research that looks at cognitive effects on children when mothers work during the first year of the child’s life. Economic and social factors play a large role in educational success or failure, and the federal government has created “compensatory” programs to level the educational playing field. One such program, Early Head Start, seems to be succeeding, and you will also learn of this research.

How do these answers compare with the opinions you recorded earlier? Do you think your personal experiences have led you to different answers than others might have given? Do you see how different people can come to such different conclusions about educational issues?

We cannot avoid asking questions about our complex educational world or trying to make sense of our position in it. In fact, the more that you begin to “think like an educational researcher,” the more such questions will come to mind. But as you’ve just seen, our own prior experiences and orientations, particularly

18

 

 

our own experiences as learners and teachers, can have a major influence on what we perceive and how we interpret these perceptions. As a result, one person may see television as a way to extend learning to millions of children, another person may think television for preschoolers should be completely banned, and others may think the entire issue is overblown. In this chapter, you will begin to look at research results in an analytic way, asking what questions have been researched, what the results of these studies are and what they mean, and how much confidence we can place in them.

19

 

 

Errors to Avoid

Educational research relies on analytic thinking, and one important element of analytic thinking is avoiding errors in logic. As readers and consumers of educational research, we have a right to expect rigorous thinking in research articles. Errors in thinking can occur in the way a research question is constructed, the methods used to carry it out, or the conclusions the researcher draws. Becoming aware of some of the most common errors in thinking will give you a head start on becoming a smart reader of educational research.

Four common errors in reasoning occur in the nonscientific, unreflective talk and writing about education that we encounter daily. Our favorite examples of these “everyday errors in reasoning” come from a letter to Ann Landers. The letter was written by someone who had just moved with her two cats from the city to a house in the country. In the city, she had not let her cats outside and felt guilty about confining them. When they arrived in the country, she threw her back door open. Her two cats cautiously went to the door and looked outside for a while, then returned to the living room and lay down. Her conclusion was that people shouldn’t feel guilty about keeping their cats indoors—even when they have the chance, cats don’t really want to play outside.

Do you see this person’s errors in reasoning? She was guilty of the following:

Selective observation. She observed the cats at the outside door only once. Overgeneralizing She observed only two cats, both of which previously were confined indoors. Yet she drew a conclusion about cats in general. Illogical reasoning. She assumed that others feel guilty about keeping their cats indoors and that cats are motivated by feelings about opportunities to play. Resistance to change. She was quick to conclude that she had no need to change her approach to the cats.

If you recognize these errors for what they are and make a conscious effort to avoid them, you can improve your own reasoning. You will guard against stereotyping people, avoid jumping to conclusions, and look at the big picture. These are errors in observing, generalizing, reasoning, and reevaluating that the methods of educational research help us avoid.

Observing

One common observing mistake is selective observation—choosing to look only at things that are in line with our preferences or beliefs. When we are inclined to criticize individuals or institutions, it is all too easy to notice their every failing. For example, if we are convinced in advance that all television viewing by children is harmful, we can find many confirming instances. But what about educational programs such as Sesame Street and educational software games that use television to teach basic concepts in language and arithmetic? If we acknowledge only the instances that confirm our predispositions, we are victims of our own selective observation.

Our observations can also simply be inaccurate. If a child says she is “hungry” and we think she said she is

20

 

 

“hunted,” we have made an inaccurate observation. If we think five students are standing in a hallway when seven actually are, we have made an inaccurate observation.

Such errors occur often in casual conversation and in everyday observation of the world around us. In fact, our perceptions do not provide a direct window onto the world around us, for what we think we have sensed is not necessarily what we have seen (or heard, smelled, felt, or tasted). Even when our senses are functioning fully, our minds have to interpret what we have sensed (Humphrey, 1992). The optical illusion in Exhibit 1.2, which can be viewed as either two faces or a vase, should help you realize that perceptions involve interpretations. Different observers may perceive the same situation differently because they interpret it differently.

Exhibit 1.2 An Optical Illusion

Generalizing

Overgeneralization occurs when we conclude that what we have observed or what we know to be true for some cases is true for most or all cases. We are always drawing conclusions from our experience, but sometimes we forget that our experiences are limited. The educational world is, after all, a complex place. We have the ability to interact with just a small fraction of the individuals who inhabit the educational world, especially in a limited span of time. Some people feel that television watching can’t hurt young children because it exposes them to ideas and information they would never have otherwise encountered. Would their experience generalize to yours? To others?

Exhibit 1.3 depicts the difference between selective observation, which we have already discussed, and

21

 

 

overgeneralization.

Exhibit 1.3 The Difference between Selective Observation and Overgeneralization

Reasoning

When we prematurely jump to conclusions or argue on the basis of invalid assumptions, we are using illogical reasoning. For example, it is not reasonable to propose that children who watch no television will have no attention problems in school because there are sources of attention problems other than television. On the other hand, an unquestioned assumption that every child who watches television will have some attention problems overlooks some important considerations, such the type of programs that are being watched and the amount of parent interaction around the child’s TV viewing. Logic that seems impeccable to one person can seem twisted to another—the problem usually is reasoning from different assumptions rather than just failing to “think straight.”

22

 

 

Reevaluating

Resistance to change, the reluctance to reevaluate our ideas in light of new information, may occur for several reasons:

Ego-based Commitments

We all learn to greet with some skepticism the claims by leaders of companies, schools, agencies, and so on that people in their organization are happy, that revenues are growing, and that services are being delivered in the best possible way. We know how tempting it is to make statements about education that conform to our own needs rather than to the observable facts. It can also be difficult to admit that we were wrong once we have staked out a position on an issue.

Excessive Devotion to Tradition

Some degree of devotion to tradition is necessary. Learning both in and out of school can be traditional in many ways, both open and hidden. Some skepticism about the effects of television on children can be a healthy antidote to panic about media’s effects, which neither parents nor teachers can control. But too much devotion to tradition can stifle adaptation to changing circumstances. When we distort our observations or alter our reasoning so that we can maintain beliefs that “were good enough for my grandfather, so they’re good enough for me,” we hinder our ability to accept new findings and develop new knowledge.

Uncritical Agreement with Authority

If we do not have the courage to evaluate critically the ideas of those in positions of authority, we will have little basis for complaint if they exercise their authority over us in ways we don’t like. And if we do not allow new discoveries to call our beliefs into question, our understanding of the educational world will remain limited.

Now take just a minute to reexamine the opinions about learning that you recorded earlier. Did you grasp at a simple explanation even though reality is far more complex? Were your beliefs influenced by your own ego and a tendency to overgeneralize your own learning experiences? Did you weigh carefully both the positive and negative effects that television has on learning? Do you see some of the challenges faced by educational researchers?

23

 

 

The Educational Research Approach

Educational research is designed to reduce potential sources of error in reasoning about the educational world. Educational research builds on the methods of science, so it relies on logical and systematic methods to answer questions, and it does so in a way that allows others to inspect and evaluate its methods. In the realm of educational research, these methods are not so unusual. After all, they involve asking questions, observing social groups, and counting people, which we often do in our everyday lives. However, educational researchers develop, refine, apply, and report their understanding of the educational world more systematically, or “scientifically,” than the general public does.

24

 

 

Science and Educational Research

What is “scientific” about educational research methods? Consider this:

Science: A set of logical, systematic, documented methods for investigating nature and natural processes; the knowledge produced by these investigations.

Educational research: The use of scientific methods to investigate teaching and learning, both inside and outside school—the educational world; the knowledge produced by these investigations.

Educational research methods reduce the likelihood of overgeneralization by using systematic procedures for selecting individuals or groups to study that are representative of the individuals or groups to which we wish to generalize. Educational researchers use explicit criteria for identifying causes and for determining whether these criteria are met in a particular instance to avoid illogical reasoning. Educational research methods reduce the risk of selective or inaccurate observation by requiring that we measure and sample phenomena systematically. Because they require that we base our beliefs on evidence that can be examined and critiqued by others, educational research methods lessen the tendency to develop answers about the educational world from ego-based commitments, excessive devotion to tradition, and/or unquestioning respect for authority.

25

 

 

Motives for Educational Research

Research begins with questions—in this chapter, questions about television viewing and early childhood development. What motivates selection of a research question or focus? Usually it’s one or more of the following reasons:

Personal Motivations

Some educational researchers who explore early childhood learning feel that by doing so, they can help to improve the lives of children, the effectiveness of schooling, or the conditions of disadvantaged groups in their communities or countries. Educational researchers may become interested in early childhood as a result of watching their own children or grandchildren or after teaching or working with young children. A teacher- researcher who teaches second graders might want to find out how much television the children in her class watch and what kinds of programs they spend the most time on. Many research questions spring from the researcher’s own life, experiences, and values.

Academic Motivations

Academic questions about influences on educational processes have stimulated much educational research. Early childhood researchers want to understand the strength and meaning of family and other outside influences on the school performance of young students. Do lower income, family disintegration, or other factors that often lead to increased day care mean that some children enter school at a large disadvantage? Can public interventions in the early years help to equalize this disadvantage? The desire to gain a better understanding of questions such as these is motivation enough for many educational researchers.

Policy Motivations

Many government agencies, elected officials, and private organizations seek better descriptions of the effects of television so they can advocate for policies that use television for the public good. School officials may need information for planning curriculum, teaching approaches, and media awareness programs. Parent groups may want to inform their members to limit television viewing in the best interests of their children. These policy and program management needs can stimulate numerous research projects.

Quantitative methods: Methods such as surveys and experiments that record variation in educational life in terms of categories that vary in amount. Data that are treated as quantitative are either numbers or attributes that can be ordered in terms of magnitude.

Qualitative methods: Methods such as participant observation, intensive interviewing, and focus groups that are designed to capture educational reality as participants experience it, rather than in categories predetermined by the researcher. Data that are treated as qualitative are mostly written or spoken words or observations that do not have a direct numerical interpretation.

26

 

 

Quantitative and Qualitative Orientations

One of the most common divisions in educational research is the distinction between quantitative research and qualitative research. Both quantitative and qualitative researchers collect data and then use them to tell a meaningful story—data analysis and findings, in research terms—but the data they collect and the methods they use to analyze them differ substantially. Here’s a very basic way of understanding the difference. Quantitative researchers collect numbers and quantities as basic data and employ a whole array of statistical procedures to analyze those data. Qualitative researchers record words, pictures, or video as data and identify patterns and themes in those data that result in narrative interpretations that create meaning.

The distinction between quantitative and qualitative data is not always sharp. Qualitative data can be converted to quantitative data, when we count the frequency of particular words or phrases in a text or measure the time elapsed between different observed behaviors. Surveys that collect primarily quantitative data may also include questions asking for written responses, and these responses may be used in a qualitative, textual analysis. Qualitative researchers may test explicit explanations of educational phenomena using textual or observational data. We’ll examine such “mixed method” possibilities in Chapter 11.

Triangulation: The use of multiple methods to study one research question.

Educational researchers often combine these methods to enrich their research. The use of multiple methods to study one research question is called triangulation. The term suggests that a researcher can get a clearer picture of the educational situation being studied by viewing it from several different perspectives. Each will have some liabilities in a specific research application, and all can benefit from combination with one or more other methods (Brewer & Hunter, 1989; Sechrest & Sidani, 1995).

The distinction between quantitative and qualitative methods involves more than just the type of data collected. Quantitative methods are most often used when the motives for research are explanation, description, or evaluation. Exploration is most often the motive for using qualitative methods, although researchers also use these methods for descriptive, explanatory, and evaluative purposes. Chapters 9 and 14 present qualitative methods in much more detail, and most other chapters include some comparison of quantitative and qualitative approaches. The next section discusses four primary types of educational research and their relationship to qualitative and quantitative orientations.

27

 

 

Types of Educational Research

There are four types of educational research projects. This section illustrates each type with examples from educational research about early childhood:

Descriptive Research

Defining and describing education-related phenomena is a part of almost any research investigation, but descriptive research is often the primary focus of the first research about some issue. One of the early descriptive questions researchers asked about the extent of television viewing by young children was, “What patterns of viewing do children have in early childhood and how do they differ from the viewing habits of older children?” (Huston et al., 1990). Measurement (the topic of Chapter 4) and sampling (Chapter 5) are central concerns in descriptive research. Survey research (Chapter 8) is often used for descriptive purposes.

Exploratory Research

Exploratory research seeks to find out how people get along in the setting under question, what meanings they give to their actions, and what issues concern them. The goal is to learn “What is going on here?” and to investigate educational phenomena without explicit expectations. This purpose is associated with the use of methods that capture large amounts of relatively unstructured information. For example, researchers investigating young children’s learning have had to look closely at the educational effects of social context and peer interaction. Exploratory research frequently involves qualitative methods.

Explanatory Research

Many consider explanation the premier goal of any science. Explanatory research seeks to identify causes and effects of educational phenomena, to predict how one phenomenon will change or vary in response to variation in some other phenomenon. Early childhood researchers adopted explanation as a goal when they began to focus on factors that influence children’s development and behavior. Their explanatory questions have included “Is maternal employment in the first year of life associated with negative child outcomes in the first three years of life?” and “Are these effects … mediated by the quality of child care or the home environment?” (Brooks-Gunn, Han, & Waldfogel, 2002, p. 1052). We focus on ways of identifying causal effects in Chapter 6. Explanatory research often involves experiments or surveys (see Chapter 8), both of which are most likely to use quantitative methods.

Evaluation Research

Seeking to determine the effects of an educational program or other type of intervention is a type of explanatory research because it deals with cause and effect (see Chapter 7). However, evaluation research differs from other forms of explanatory research because evaluation research considers the implementation and effects of educational policies and programs. These issues may not be relevant in other types of explanatory research. For example, concern over the impact of Early Head Start, an extensive federal early childhood program, provided the impetus for a major federal evaluation study that involved more than 3,000 families

28

 

 

(U.S. Department of Health and Human Services, 2002).

Certain types of research are often used with certain research methods because those methods yield a kind of data that are helpful for answering that type of question. Exhibit 1.4 shows some common correspondences between type of research, the goal of the research, and methods and techniques that match. The correspondence between types and methods should not be regarded as a hard-and-fast rule—it’s possible to find exploratory research that uses some quantitative techniques, for example—but the chart gives a handy way of thinking about relationships between types and methods.

We’ll now summarize four actual early childhood research projects that exemplify these four types of research.

Description: What Types of TV Programs do Young Children Watch?

In the 1980s, when this research was conducted, researchers knew that children watched a lot of television but knew little about what types of programs they were actually watching. A team of researchers followed 326 children for 2 years to determine how much television and what kind of programming the children were watching. The children were in two age groups, 3 to 5 and 5 to 7 years old.

The researchers found that the children watched, on average, 2 to 3 hours of television per day. Children’s viewing changed with age, and boys tended to watch more cartoon and action programs than girls. Boys were also more interested in adult informational programs. As they grew older, children of both sexes began watching more comedies meant for general audiences. As the children matured, they moved from informative programs aimed at children to entertainment programs for a general audience (Huston et al., 1990).

Exploration: How does Classroom Social Context Affect a Disabled Preschool Child?

Elizabeth Erwin, Elizabeth Alimaras, and Nikki Price (1999) knew that research going back to the 1930s shows the importance of social context and peer interaction to learning. They also knew that there was little

29

 

 

research on peer interactions in early childhood settings that included children with disabilities. They designed a small pilot study of socialization experiences in a preschool class that included Ryan, a 3-year-old with identified disabilities (Ryan had detached retinas in both eyes with no light perception in his right eye and a visual acuity of 20/600 in his left eye, no cognitive impairments, but other medical complications, including kidney disease). To conduct the study,

qualitative methods (i.e. participant observations and semi-structured personal interviews) were used … in an effort to provide a rich portrait of events, experiences, and perspectives. Data were gathered across daily classroom routines and natural settings within the school such as the music room, playground, hallways, and classrooms. (p. 57)

Observations began in September and were conducted approximately once a month until school closed in June. Observational data were supplemented by 1-hour personal interviews with the classroom teacher, the teacher assistant, and Ryan’s mother and father. Because this was an exploratory study, the findings included narrative descriptions of the types of interactions that occurred between Ryan and other children. Overall, the authors felt that their exploratory study, rich in description, resulted in more questions than answers, and they urged other researchers to further explore the skills and knowledge practitioners needed to successfully support disabled students. This type of conclusion, which points the way for further research, is not uncommon in exploratory studies, which are typically undertaken when a question or research area is just starting to be investigated.

Explanation: When Mothers Work during the Child’s First Year, Do Children Suffer Cognitively?

One of the large changes in our society over the past 40 years has been the increasing number of mothers who work outside the home. Questions have been raised about whether this helps or harms young children as they prepare to begin formal schooling. Researchers at the U.S. National Institute of Child Health and Human Development (Brooks-Gunn et al., 2002) tried to find out if maternal employment during a child’s first year had positive or negative cognitive effects later, when the child was almost ready to go to school. They used quantitative methods to analyze a large body of information (called a “data set”) that the agency had already collected.

The researchers studied 900 European American children. The number of hours that mothers worked was compared to their children’s scores on a standard test of cognitive skills. They concluded that

maternal employment by the ninth month was found to be linked to lower Bracken School Readiness Scores at 36 months, with the effects more pronounced when mothers were working 30 hours or more per week and with effects more pronounced for certain subgroups (i.e., children whose mothers were not sensitive, boys, and children with married parents). (Brooks-Gunn et al., 2002, p. 1052)

30

 

 

The researchers were careful to note limitations of their study. These included the fact that they had studied only one population, European Americans, and the idea that factors other than working also made a difference. These factors included “quality of child care, home environment, and maternal sensitivity.” However, even when these additional factors were taken into account, “the negative effects of working 30 or more hours per week in the first 9 months were still found” (Brooks-Gunn et al., 2002, p. 1052).

The researchers also made policy recommendations based on their findings, which is not unusual in explanatory research in education. They recommended, for instance, that “it would be prudent for policy makers to go slow on measures … that would require mothers to enter the labor force (full-time) early in the first year of life and to consider measures (such as the proposed Family and Maternal Leave Act extensions) that would allow more mothers to choose to delay their return to the labor force and/or to work part-time until late in the first year of life” (p. 1068).

Evaluation: Is the Early Head Start Program Working?

Because large- and small-scale programs are a common feature of schools, many educational research studies try to measure program effectiveness. In 1995, Congress created Early Head Start, which was the only federal program specifically designed to improve the early education experiences of low-income babies and toddlers. In August 2002, the federal government published the first full evaluation study of the program, which aims to improve infants’ and toddlers’ later school success by supporting prenatal health and improving children’s cognitive, social, and emotional development (U.S. Department of Health and Human Services, 2002).

The researchers used a large, randomly assigned sample—more than 3,000 families—in 17 sites across the country and a wide range of evaluation measures of cognitive, language, and social-emotional development. Evaluation results showed that the program was working across the whole range of indicators. These included cognitive functioning, interaction with parents, movement toward self-sufficiency, child-father interactions, and greater school readiness (U.S. Department of Health and Human Services, 2002).

Exhibit 1.5 shows some of the indicators used in the Early Head Start study to measure parents’ knowledge of and involvement with their children. The last three items refer to a hypothetical scenario presented to parents in which their child’s behavior required parental intervention. Unfortunately, one of the other findings of the evaluation was that the program currently reached only 3% of those eligible to receive services (Stark, 2003).

Source: U.S. Department of Health and Human Services (2002, p. 16).

31

 

 

32

 

 

Educational Research Philosophies

Different educational researchers are guided by different research philosophies. A philosophy, in this case, means a viewpoint on what constitutes educational reality. Naturally, how you think about reality has implications for what methods you use to investigate that reality. In this section, we will describe and explain two alternative research philosophies that are prevalent in educational research, positivism and interpretivism. Positivism—and its descendent postpositivism—is more closely linked to quantitative research approaches. Interpretivism—and its descendent constructivism—is more closely linked to qualitative approaches.

33

 

 

Positivism and Postpositivism

Researchers with a positivist philosophy believe that an objective reality exists apart from the perceptions of those who observe it and that the goal of research is to better understand this reality. This is the philosophy traditionally associated with natural science (e.g., biology, chemistry, physics), with the expectation that there are universal laws of human behavior and with the belief that scientists must be objective and unbiased to see reality clearly (Weber, 1949, p. 72). Positivists believe that a well-designed test of a theoretically based prediction can move us closer to understanding actual educational processes.

Positivism: The belief, shared by most scientists, that there is a reality that exists quite apart from our own perception of it, that it can be understood through observation, and that it follows general laws.

Postpositivism: The belief that there is an empirical reality, but that our understanding of it is limited by its complexity and by the biases and other limitations of researchers.

Intersubjective agreement: An agreement by different observers on what is happening in the natural or educational world.

The philosophy of postpositivism is closely related to positivism. Postpositivists believe that there is an external, objective reality, but they are very sensitive to the complexity of this reality and to the limitations and biases of the scientists who study it (Guba & Lincoln, 1994, pp. 109–111). As a result, they do not think we can ever be sure that scientific methods allow us to perceive objective reality. Instead, postpositivists believe that the goal of science is to achieve intersubjective agreement among scientists about the nature of reality (Wallace, 1983, p. 461). For example, postpositivists may worry that researchers’ predispositions bias them in favor of a certain theory. Therefore, they remain skeptical of research results that support that theory until a number of researchers report such evidence. A postpositivist has much more confidence in the community of researchers than in any individual researcher (Campbell & Russo, 1999, p. 144).

34

 

 

Interpretivism and Constructivism

Qualitative educational research is often guided by a different, interpretivist philosophy. Interpretivist approaches have become increasingly influential in educational research, and their growing prevalence since the 1990s, sometimes referred to as the “interpretive turn,” has changed the educational research landscape (Howe, 1998). Interpretivist researchers believe that educational reality is socially constructed and that the goal of educational research is to understand what meanings people give to reality, not to determine how reality works apart from these interpretations. This philosophy rejects the positivist belief that there is a concrete, objective reality that scientific methods help us to understand (M. Lynch & Bogen, 1997). Instead, interpretivists believe that people construct an image of reality based on their own preferences and prejudices and their interactions with others and that this is as true of scientists as it is of everyone else. This means that we can never be sure that we have understood reality properly, that “objects and events are understood by different people differently, and those perceptions are the reality—or realities—that social science should focus on” (Rubin & Rubin, 1995, p. 35).

Interpretivism: The belief that reality is socially constructed and that the goal of social scientists is to understand what meanings people give to that reality.

Constructivist paradigm: A perspective that emphasizes how different stakeholders in educational settings construct their beliefs.

The constructivist paradigm extends interpretivist philosophy by emphasizing the importance of exploring how different stakeholders in a social setting construct their beliefs (Guba & Lincoln, 1989, pp. 44–45). It gives particular attention to the different goals of researchers and other participants in a research setting and seeks to develop a consensus among participants about how to understand the focus of inquiry. From this standpoint, “Truth is a matter of the best-informed and most sophisticated construction on which there is consensus at a given time” (Schwandt, 1994, p. 128).

35

 

 

Taking a Balanced Approach

It is tempting to think of positivism as representing an opposing research philosophy to interpretivism and constructivism. Then it seems that we should choose the one philosophy that seems closest to our own preferences and condemn the other as “unscientific,” “uncaring,” or perhaps just “unrealistic.” But there are good reasons to prefer a research philosophy that integrates some of the differences between these philosophies (J. Smith, 1991). Researchers influenced by a positivist philosophy should be careful to consider how their research approaches and interpretations are shaped by their own social and educational background —just as we are cautioned to do by interpretivist researchers. Researchers influenced more by an interpretivist philosophy should be careful to ensure that they use rigorous procedures to check the trustworthiness of their interpretations of data (Reissman, 2008, pp. 185–199). If we are not willing to ask “hard questions” about our projects and the evidence we collect, we are not ready to investigate the educational world (Reissman, 2008, p. 200). The educational phenomena we study are often complex, and we must take this complexity into account when we choose our methods and interpret our results.

But even in areas of research that are fraught with controversy, the quest for new and more sophisticated research has value. What is most important for improving understanding of the educational world is not the result of any particular study but the accumulation of evidence from different studies of related issues. By designing new studies that focus on the weak points or controversial conclusions of prior research, educational researchers contribute to a body of findings that gradually expands our knowledge about the educational world and resolves some of the disagreements about it.

36

 

 

Conclusions

We began this chapter with the question, “How do early childhood experiences affect schooling?” Throughout the chapter, you saw examples of ways in which researchers approached this topic, including looking at the effects of television on attention span and aggressive behavior; the effects of social dynamics in a preschool classroom with 14 students, one of whom was severely disabled; the possible cognitive effects on very young children when mothers work; and the evaluation of Early Head Start, a federal preschool program. Each study examined the question from a different perspective, used different methods, and reached different conclusions. Taken together, though, they begin to build a research-based answer to the original question. They show, in a small way, how the aggregation of individual studies can eventually help us to attain big answers.

We hope this first chapter has given you an idea of what to expect in the rest of the book. Our aim is to introduce you to educational research methods by describing what educational researchers have learned about the educational world as well as how they have learned it. The substance of educational research inevitably is more interesting than its methods, but the methods become more interesting when they’re linked to substantive investigations.

This book’s first six chapters deal with Foundations of Research. We have focused attention on early childhood research in this chapter; in Chapter 2, we use studies of reading instruction to illustrate the research process. Chapter 2 outlines the research process as a whole and also presents specific techniques for becoming a savvy reader and user of educational research. We also introduce the process of writing research proposals, which is then continued in special end-of-chapter exercises throughout the book. Chapter 3 explains research ethics, particularly the importance of the ethical treatment of human subjects.

To complete our overview of the foundations of research, in Chapters 4, 5, and 6, we introduce a variety of measurement approaches, different ways of sampling larger populations, and specific techniques to maximize the validity of causal assertions. Methods of designing research and collecting data are the focus of the book’s second section (Chapters 7–12). Evaluation research and educational assessment, the subjects of Chapter 7, are conducted to identify the impact of educational programs or to clarify educational processes involving such programs. Experimental methods are often part of evaluation research, and so they are discussed in this chapter. Survey research (Chapter 8) can be used to collect data from a large population of students, teachers, or community members.

Chapter 9 shows how qualitative methods can uncover aspects of the educational world that we are likely to miss in experiments and surveys and sometimes result in a different perspective on educational processes.

Chapters 10, 11, and 12 introduce data collection approaches that can involve several methods. Chapter 10 focuses on single subject designs, which can be extremely useful in investigations in the classroom and in counseling, where a single student can be studied. Chapter 11 covers mixed methods. As you might suspect, mixed-method approaches combine two or more of the other methods. Chapter 12 gives special attention to methods of inquiry—teacher research and action research—that emphasize the concerns and insights of

37

 

 

practitioners—classroom teachers and others directly involved in education.

The third section of this book, comprising the final three chapters, deals with Analyzing and Reporting Data. Chapter 13 is not a substitute for an entire course in statistics, but it gives you a basic idea of how to use statistics when analyzing research data and reporting or reviewing research results. In Chapter 14, we examine in some detail the logic and procedures of qualitative data analysis. You will be struck by the differences between qualitative data analysis techniques and the quantitative data analysis techniques of Chapter 13. Chapter 15 deals with reporting research; our research efforts are really only as good as the attention we give to our research reports. In Chapter 15, we finish the discussion of research proposals started in Chapter 2.

Each chapter ends with several helpful learning tools. Lists of key terms and chapter highlights will help you to review. Discussion questions and practice exercises will help you to apply and deepen your knowledge. Special exercises guide you in developing your first research proposal and finding information on the World Wide Web.

38

 

 

Key Terms

Constructivism 15 Descriptive research 11 Evaluation research 12 Explanatory research 11 Exploratory research 11 Illogical reasoning 8 Inaccurate observations 6 Measurement 11 Qualitative methods 10 Quantitative methods 10 Positivism 15 Postpositivism 15 Resistance to change 8 Science 9 Selective observation 6 Triangulation 11

39

 

 

Highlights

Educational research cannot resolve value questions or provide permanent, universally accepted answers. Empirical data are obtained in educational research investigations from either direct experience or others’ statements. Four common errors in reasoning are overgeneralization, selective or inaccurate observation, illogical reasoning, and resistance to change. These errors result from the complexity of the educational world, subjective processes that affect the reasoning of researchers and those they study, researchers’ self- interestedness, and unquestioning acceptance of tradition or of those in positions of authority. Educational research is the use of logical, systematic, documented methods to investigate individuals, processes, contents, and educational systems, as well as the knowledge produced by these investigations. Educational research can be motivated by personal preferences, academic issues, and policy concerns. Educational research can be descriptive, exploratory, explanatory, or evaluative—or some combination of these. Quantitative and qualitative methods structure research in different ways and are differentially appropriate for diverse research situations. It is possible to mix qualitative and quantitative methods to gain accurate knowledge of particular questions.

40

 

 

Student Study Site

To assist in completing the web exercises, please access the study site at www.sagepub.com/check, where you will find the web exercise with accompanying links. You’ll find other useful study materials such as self- quizzes and e-flashcards for each chapter, along with a group of carefully selected articles from research journals that illustrate the major concepts and techniques.

41

 

http://www.sagepub.com/check

 

Discussion Questions

1. Select an educational issue that interests you, such as television watching or charter schools. List at least four of your beliefs about this phenomenon. Try to identify the sources of each of these beliefs.

2. Find a report of an educational research in an article in a daily newspaper. What were the major findings? How much evidence is given about the methods the researcher used? What additional design features might have helped to improve the study’s validity?

42

 

 

Practice Exercises

1. Review letters to the editor and opinion pieces in your local newspaper. Identify any errors in reasoning: overgeneralization, selective or inaccurate observation, illogical reasoning, or resistance to change.

2. Read the abstracts (initial summaries) of each article in a recent issue of a major educational research journal. (Ask your instructor for some good journal titles.) On the basis of the abstract only, classify each research project represented in the articles as primarily descriptive, exploratory, explanatory, or evaluative. Note any indications that the research focused on other types of research questions.

43

 

 

Web Exercises

1. Prepare a 5- to 10-minute class presentation on the ERIC System. Go to the ERIC site at http://www.eric.ed.gov/ to view some of the research. Pick a study listed on ERIC and write up a brief outline for your presentation, including information on study design, questions asked, and major findings.

2. Is the Pediatrics study’s perspective representative of other researchers? Check out the research reports on early childhood and television for the last 5 years at ERIC. How many studies did you find? Write up some information regarding the research and its goals, methods, and major findings. What do the researchers conclude about the impact of television on young children? How do these conclusions compare to each other and to those of the Pediatrics study?

44

 

http://www.eric.ed.gov/

 

Developing a Research Proposal

Will you develop a research proposal in this course? If so, you should begin to consider your alternatives.

1. Think of three or four topic areas you might like to study. What are your motives for studying each topic?

2. Develop four questions that you might investigate about two of the topics you just selected. Each question should reflect a different research motive: description, exploration, explanation, or evaluation. Be specific.

3. Which question most interests you? Would you prefer to attempt to answer that question with quantitative or qualitative methods? Why?

45

 

 

Chapter 2 The Process and Problems of Educational Research

Research Question: How Does a Child Learn to Read?

46

 

 

Chapter Contents

Educational Research Questions Educational Research Basics The Role of Educational Theory Educational Research Goals Educational Research Proposals, Part I

Reading has formed the core of the elementary school curriculum since the beginning of public education in the United States. In this chapter, we will examine different educational research strategies and how they have been used to answer questions about reading. We will consider in some detail the techniques required to begin the research process: formulating research questions, finding information, reviewing prior research, and writing a research proposal. Appendices A, B, and C provide more details on these key techniques. By the chapter’s end, you should be ready to formulate a research question, design a general strategy for answering this question, critique previous studies that addressed this question, and begin a proposal for additional research on the question.

47

 

 

Educational Research Questions

An educational research question is a question about the educational world that you seek to answer through the collection and analysis of firsthand, verifiable, empirical data. It is not a question about who did what to whom but a question about individual cases, people in groups, general educational processes, or tendencies in school change. What kinds of responses does a disabled preschooler get from classroom peers? Is language development innate or environmental? How much have techniques for teaching reading changed over the past 40 years? So many research questions are possible that it is more of a challenge to specify what does not qualify as an educational research question than to specify what does.

But that doesn’t mean it is easy to specify a research question. In fact, formulating a good research question can be surprisingly difficult. We can break the process into three stages: identifying one or more questions for study, refining the questions, and then evaluating the questions.

48

 

 

Identifying Educational Research Questions

Educational research questions may emerge from your own experience—from your “personal troubles,” as C. Wright Mills (1959) put it. One experience might be based on your own learning to read, another on your son’s or daughter’s preschool experiences, a third on a show about high schools you saw on television. You may find yourself asking a question such as, “In what ways do students tend to transfer home reading experiences to school?” or “Do 3-year-old boys play differently than 3-year-old girls?” or “Does the high school day begin too early for effective learning by adolescents?”

The research literature is often the best source for research questions. Every article or book will bring new questions to mind. Even if you’re not feeling too creative when you read the literature, most research articles highlight unresolved issues and end with suggestions for additional research. For example, Anderson, Hiebert, Scott, and Williamson (1985, p. 23) reviewed a large body of previous research and concluded that reading aloud to children was the most important thing parents could do to ensure their children’s reading success in school. However, later researchers challenged this conclusion, particularly in regard to family literacy practices in nonmainstream homes (Auerbach, 1995; Moll, Amanti, Neff, & Gonzalez, 1992). A new study could focus on the effects of social context on a child’s emergent literacy: How do home literacy activities in widely different settings influence future school literacy performance? Any research article in a journal in your field is likely to have comments that point toward unresolved issues.

Many educational researchers find the source of their research questions in educational theory. For example, you may have concluded that children who have been read aloud to at home have a big advantage in learning to read in school, so you may ask whether reading theory can explain how listening affects reading.

Finally, some research questions have very pragmatic sources. You may focus on a research question posed by someone else because it seems to be to your advantage to do so. Some educational scientists conduct research on specific questions posed by a funding source in what is termed an RFP, a request for proposals. (Sometimes the acronym RFA is used, meaning request for applications.) Or you may learn that the action research team in the school where you work needs help with a survey to learn about parents’ home reading practices, which becomes the basis for another research question.

49

 

 

Refining Educational Research Questions

It is even more challenging to focus on a problem of manageable size than it is to come up with an interesting question for research. We are often interested in much more than we can reasonably investigate with limited time and resources. In addition, researchers may worry about staking a research project (and thereby a grant or a grade) on a single problem, and so they may address several research questions at once. Also, it might seem risky to focus on a research question that may lead to results that conflict with our own cherished assumptions about the educational world. The prospective commitment of time and effort for some research questions may seem overwhelming, resulting in a certain degree of paralysis.

The best way to avoid these problems is to develop the research question one bit at a time. Don’t keep hoping that the perfect research question will just spring forth from your pen. Instead, develop a list of possible research questions as you go along. At the appropriate time, you can look through this list for the research questions that appear more than once. Narrow your list to the most interesting, most workable candidates. Repeat this process as long as it helps to improve your research questions.

50

 

 

Evaluating Educational Research Questions

In the third stage of selecting a research question, we evaluate the best candidate against the criteria for good educational research questions: feasibility, given the time and resources available; educational importance; and scientific relevance (King, Keohane, & Verba, 1994).

Feasibility

We must be able to conduct any study within the time and given the resources we have. If time is short, questions that involve long-term change may not be feasible. Another issue is what people or groups we can expect to gain access to. Observing educational interaction in Tahiti may be impractical if you are a student in Maine. Next we must consider whether we will have any additional resources, such as research funds or other researchers to collaborate with. Remember that there are severe limits on what one person can accomplish. On the other hand, we may be able to piggyback our research onto a larger research project. We also must take into account the constraints we face due to our schedules and other commitments, as well as our skill level.

Educational Importance

Educational research is not a simple undertaking, so it is hard to justify the expenditure of effort and resources unless we focus on a substantive area that is important. Besides, you need to feel motivated to carry out the study. Nonetheless, “importance” is relative, so for a class assignment, university students’ memories of learning to read and write might be important enough.

For most research undertakings, we should consider whether the research question is important to other people. Will an answer to the research question make a difference for society or for educational relations?

Educational researchers are not wanting for important research questions. A typical recent issue of the journal American Educational Research (2009) included studies of the role of courts in educational policy; of college students majoring in disciplines related to science, technology, and math; of the study habits of academic versus vocational track high school students; and of the use of grade-level teaching teams in elementary schools. All of these articles addressed research questions about important educational issues, and all raised new questions for additional research.

Scientific Relevance

Every research question should be grounded in the educational research literature. Whether we formulate a research question because we have been stimulated by an academic article or because we want to investigate a current educational problem, we should first turn to the educational research literature to find out what already has been learned about this question. You can be sure that some prior research is relevant to almost any research question you can think of.

Different studies of reading aloud to children and other home literacy practices have led to contradictory

51

 

 

conclusions about the impact of home life on learning to read. This particularly has been the case in studying

students from nonmainstream families or families where English is not the home language. Exploring the educational research literature on a given topic will not usually tell you that someone has already found “the answer.” It will inform you, though, about who has begun to ask good questions, how they have attempted to answer those questions, and what areas have and haven’t been explored. So you always need to connect new research questions to prior research investigations.

52

 

 

Educational Research Basics

How do we find prior research on questions of interest? You may already know some of the relevant material from prior coursework or your independent reading, but that won’t be enough. When you are about to launch an investigation of a new research question, you must apply a very different standard than when you are studying for a test or just seeking to “learn about how kids learn to read.” You need to find reports of previous investigations that sought to answer the same research question that you wish to answer, not just those that were about a similar topic. If there have been no prior studies of the same research question on which you wish to focus, you should seek to find reports from investigations of closely related research questions. Once you have located reports from prior research similar to the research you wish to conduct, you may expand your search to include investigations about related studies that used similar methods.

Sometimes you’ll find that someone else has already reviewed the literature on your research question in a special review article or book chapter. For example, Melanie R. Kuhn and Steven A. Stahl (2003) published an excellent review of the research on fluency in reading instruction in the Journal of Educational Psychology. They began with two basic questions: “How do children become fluent readers?” and “What instructional strategies are effective in promoting literacy among beginning readers?” (p. 3). Their article summarizes a range of literature in these areas, covering both theoretical and practical approaches.

You will not always be lucky enough to find a recent, comprehensive literature review article in the area of your question, but most of the research articles you find will include a literature review. These reviews can help a lot, but they are no substitute for reviewing the literature yourself. No one but you can decide what is relevant for your research question and the research circumstances you will be facing—the setting you will study, the timing of your study, the new issues that you want to include in your study, and your specific methods. And you can’t depend on any published research review for information on the most recent work. New research results about many questions appear continually in scholarly journals and books, in research reports from government agencies and other organizations, and on websites all over the world; you’ll need to check for new research such as this yourself.

53

 

 

Finding Information

Conducting a thorough search of the research literature and then reviewing critically what you have found is an essential foundation for any research project. Fortunately, much of this information can be identified online, and an increasing number of published journal articles can be downloaded directly to your own computer (depending on your particular access privileges). But just because there’s a lot available online doesn’t mean that you need to find it all. Keep in mind that your goal is to find reports of prior research investigations; this means that you should focus on scholarly journals that choose articles for publication after they have been reviewed by other educational researchers—“refereed journals.” Newspaper and magazine articles and Wikipedia entries just won’t do, although you may find some that raise important issues or even that summarize educational science research investigations.

Every year, the Web offers more and more useful material, including indexes of the published research literature. You may find copies of particular rating scales, reports from research in progress, papers that have been presented at professional conferences, and online discussions of related topics. We will review in this section the basic procedures for finding relevant research information in both the published literature and on the Web, but keep in mind that the primary goal is to identify research articles published in refereed journals. Appendix C provides more detailed instructions.

Searching the Literature

The educational research literature should be consulted at the beginning and end of an investigation. Even while an investigation is in progress, consultations with the literature may help to resolve methodological problems or facilitate supplementary explorations. As with any part of the research process, the method you use will affect the quality of your results. You should try to ensure that your search method includes each of the following steps:

Specify your Research Question

Your research question should be neither so broad that hundreds of articles are judged relevant nor so narrow that you miss important literature. “Does building vocabulary improve reading comprehension?” is probably too broad. “Does weekly vocabulary testing strengthen word attack skills for second-grade boys?” is probably too narrow. “Does the vocabulary level of early childhood students affect their growth as readers?” provides about the right level of specificity.

Identify Appropriate Bibliographic Databases to Search

The Educational Resources Information Center (ERIC) may meet many of your needs. Sponsored by the U.S. Department of Education, ERIC is a comprehensive, searchable, digital library of education-related resources, including both abstracts and full-text articles, going back to 1966. However, if you are studying a question about education and disability, you should also search in Medline, the database for searching the medical literature. If your focus is on counseling or mental health, you’ll also want to include a search in the

54

 

 

online psychological abstracts database, PsycINFO, or the version that also contains the full text of articles

since 1985, PsycARTICLES. There are a wide variety of other specialized databases you may wish to investigate, such as Ebsco Business Source Complete, which covers research pertaining to adult learners in the workplace.

In addition, the search engine Google now offers “Google Scholar” (which indexes journal articles) and “Google Print” (which indexes books) for anyone with Web access.

Choose a Search Technology

For most purposes, an online bibliographic database that references the published journal literature will be all you need. In addition to the databases already mentioned, your college library can probably give you access to scholarly databases covering education such as Academic Search Premier, Educator’s Reference Complete, Expanded Academic ASAP, and JSTOR. However, searches for obscure topics or very recent literature may require that you also search websites or bibliographies of relevant books.

Create a Tentative List of Search Terms

List the parts and subparts of your research question and any related issues that you think are important: “reading comprehension,” “vocabulary instruction,” “emergent literacy,” and perhaps “early childhood reading.” List the authors of relevant studies. Specify the most important journals that deal with your topic.

Narrow your Search

The sheer number of references you find can be a problem. Depending on the database you are working with and the purposes of your search, you may want to limit your search to English-language publications, journal articles rather than conference papers or dissertations (both of which are more difficult to acquire), and materials published more recently, say, in the past 5 years.

Refine your Search

Learn as you go. If your search yields too many citations, try specifying the search terms more precisely. If you have not found much literature, try using more general terms. Whatever terms you search first, don’t consider your search complete until you have tried several different approaches and have seen how many articles you find. A search for reading research in ERIC on June 2, 2010, yielded 59, 424 hits, but searching quantitative reading research and then qualitative reading research reduced the number of hits to 1,892 and 1,351, respectively.

Use Boolean Search Logic

It’s often a good idea to narrow your search by requiring that abstracts contain combinations of words or phrases that include more of the specific details of your research question. Using the Boolean connector AND allows you to do this, while using the connector OR allows you to find abstracts containing different words that mean the same thing. (This is explained further in Appendix C.) Exhibit 2.1 provides an example of a Boolean search using AND plus keywords reading and qualitative in the Advanced Search option in ERIC.

55

 

 

Use Appropriate Subject Descriptors

Once you have found an article that you consider appropriate, take a look at the “descriptors” field in the citation (see Exhibit 2.2). You can then redo your search after requiring that the articles be classified with some or all of these descriptor terms.

Check the Results

Read the titles and abstracts you have found and identify the articles that appear to be most relevant. If possible, click on these article titles and generate a list of their references. See if you find more articles that are relevant to your research question but that you have missed so far. You will be surprised (we always are) at how many important articles your initial online search missed.

Read the Articles

Now it is time to find the full text of the articles of interest. If you’re lucky, some of the journals you need will be available to patrons of your library in online versions, and you’ll be able to link to the full text just by clicking on a “full text” link. But many journals, specific issues of some journals, or both will be available only in print; in this case, you’ll have to find them in your library or order a copy through interlibrary loan.

Refer to a good book for even more specific guidance. Arlene Fink’s (2005) Conducting Research Literature Reviews: From the Internet to Paper is an excellent guide.

Exhibit 2.1 Boolean Connectors in ERIC

Exhibit 2.2 ERIC Search Result with Descriptors

56

 

 

You may be tempted to write up a “review” of the literature based on reading the abstracts or using only those articles available online, but you will be selling yourself short. Many crucial details about methods, findings, and theoretical implications will be found only in the body of the article, and many important articles will not be available online. To understand, critique, and really learn from previous research studies, you must read the important articles, no matter how you have to retrieve them.

If you have done your job well, you will now have more than enough literature as background for your own research, unless it is on a very obscure topic (see Exhibit 2.3). At this point, your main concern is to construct a coherent framework in which to develop your research question, drawing as many lessons as you can from previous research. You may use the literature to identify a useful theory and hypotheses to be reexamined, to find inadequately studied specific research questions, to explicate the disputes about your research question, to summarize the major findings of prior research, and to suggest appropriate methods of investigation.

Exhibit 2.3 A Search in ERIC on “Early Childhood Reading” on May 26, 2011, Showing the First of 13 Items

Be sure to take notes on each article you read, organizing your notes into standard sections: theory, methods, findings, and conclusions. In any case, write the literature review so that it contributes to your study in some concrete way; do not feel compelled to discuss an article just because you have read it. Be judicious. You are conducting only one study of one issue; it will only obscure the value of your study if you try to relate it to every tangential point in related research.

Do not think of searching the literature as a one-time-only venture—something that you leave behind as you move on to your real research. You may encounter new questions or unanticipated problems as you conduct your research or as you burrow deeper into the literature. Searching the literature again to determine what others have found in response to these questions or what steps they have taken to resolve these problems can yield substantial improvements in your own research. There is so much literature on so many topics that it often is not possible to figure out in advance every subject you should search the literature for or what type of search will be most beneficial.

Another reason to make searching the literature an ongoing project is that the literature is always growing.

57

 

 

During the course of one research study, whether it takes only one semester or several years, new findings will be published and relevant questions will be debated. Staying attuned to the literature and checking it at least when you are writing up your findings may save your study from being outdated.

Searching the Web

The World Wide Web provides access to vast amounts of information of many different sorts (Ó Dochartaigh, 2002). You can search the holdings of other libraries, download the complete text of government reports, and find descriptions of particular research projects. It is also hard to avoid finding a lot of information in which you have no interest, such as commercial advertisements, third-grade homework assignments, or college course syllabi.

After you are connected to the Web with a browser such as Microsoft Internet Explorer, Firefox, or Chrome, you can use three basic strategies for finding information: direct addressing—typing in the address, or URL, of a specific site; browsing—reviewing online lists of websites; and searching—the most common approach. Google is currently the most popular search engine for searching the Web. For some purposes, you will need to use only one strategy; for other purposes, you will want to use all three. Appendix C contains information on all three methods.

Exhibit 2.4 illustrates the first problem that you may encounter when searching the Web: the sheer quantity of resources that are available. It is a much bigger problem than when searching bibliographic databases. On the Web, less is usually more. Limit your inspection of websites to the first few pages that turn up in your list (they’re ranked by relevance). See what those first pages contain and then try to narrow your search by including some additional terms. Putting quotation marks around a phrase that you want to search will also help to limit your search—searching for “early childhood reading” on Google (on May 25, 2011) produced 105,000 sites, compared to the more than 13 million retrieved when the quotes were omitted—so Google searched “early” and “childhood” and “reading.” Notice also in Exhibit 2.4 that the first two results, as well as all of the entries down the right-hand side of the page, are labeled “ads,” meaning that Google was paid to list them. Almost all of the time, ads should be ignored because they are paid promotions for commercial products or services rather than scholarly contributions. Many other Google entries, such as most of those shown in Exhibit 2.4 (a YouTube video, commercial reading booklets), are also of little or no value for your purposes. When using Google, you must always separate legitimate scholarly contributions from the “background noise” that your search will inevitably pull up. Using Google Scholar or Google Print rather than regular Google will address this problem but will also reduce the number of results considerably.

Exhibit 2.4 Google Search Results for “Early Childhood Reading”

58

 

 

Remember the following warnings when you conduct searches on the Web:

Clarify your Goals

Before you begin the search, jot down the terms that you think you need to search for as well as a statement of what you want to accomplish with your search. This will help to ensure that you have a sense of what to look for and what to ignore.

Quality is not Guaranteed

Anyone can post almost anything, so the accuracy and adequacy of the information you find are always suspect. There’s no journal editor or librarian to evaluate quality and relevance.

Anticipate Change

Websites that are not maintained by stable organizations can come and go very quickly. Any search will result in attempts to link to some URLs that no longer exist.

One Size does not Fit All

Different search engines use different procedures for indexing websites. Some attempt to be all-inclusive, whereas others aim to be selective. As a result, you can get different results from different search engines (such as Google or Yahoo) even though you are searching for the same terms.

Be Concerned about Generalizability

You might be tempted to characterize district reading policies by summarizing the documents you find at school district websites. But how many school districts are there? How many have posted their policies on the Web? Are these policies representative of all school districts? To answer all these questions, you would have to conduct a research project just on the websites themselves.

Evaluate the Sites

There is a lot of information out there, so how do you know what’s good? Some websites contain excellent advice and pointers on how to differentiate the good from the bad.

Avoid Web Addiction

59

 

 

Another danger of the enormous amount of information available on the Web is that one search will lead to another and to another and so on. There are always more possibilities to explore and one more interesting source to check. Establish boundaries of time and effort to avoid the risk of losing all sense of proportion.

Cite your Sources

Using text or images from Web sources without attribution is plagiarism. It is the same as copying someone else’s work from a book or article and pretending that it is your own. Record the Web address (URL), the name of the information provider, and the date on which you obtain material from the site. Include this information in a footnote to the material that you use in a paper.

60

 

 

Reviewing Research

Effective review of the prior research you find is an essential step in building the foundation for new research. You must assess carefully the quality of each research study, consider the implications of each article for your own plans, and expand your thinking about your research question to take account of new perspectives and alternative arguments. It is through reviewing the literature and using it to extend and sharpen your own ideas and methods that you become a part of the educational science community. Instead of being just one individual studying an issue that interests you, you are making your own small contribution to an ever- growing body of knowledge that is being constructed by the entire community of scholars.

The research information you find on various websites comes in a wide range of formats and represents a variety of sources. Caveat emptor (buyer beware) is the watchword when you search the Web; following review guidelines such as those we have listed will minimize, but not eliminate, the risk of being led astray. By contrast, the published scholarly journal literature that you find in databases such as ERIC and Psychological Abstracts follows a much more standard format and has been subject to a careful review process. There is some variability in the contents of these databases—some journals publish book reviews, comments on prior articles, dissertation abstracts, book reviews, and conference papers. However, most literature you will find on a research topic in these databases represents peer-reviewed articles reporting analyses of data collected in a research project. These are the sources on which you should focus. This section concentrates on the procedures you should use for reviewing these articles. These procedures also can be applied to reviews of research monographs—books that provide much more information from a research project than that which can be contained in a journal article.

Reviewing the literature is really a two-stage process. In the first stage, you must assess each article separately. This assessment should follow a standard format such as that represented by the “Questions to Ask About a Research Article” in Appendix A. However, you should keep in mind that you can’t adequately understand a research study if you just treat it as a series of discrete steps, involving a marriage of convenience among separate techniques. Any research project is an integrated whole, so you must be concerned with how each component of the research design influenced the others—for example, how the measurement approach might have affected the causal validity of the researcher’s conclusions and how the sampling strategy might have altered the quality of measures.

The second stage of the review process is to assess the implications of the entire set of articles (and other materials) for the relevant aspects of your research question and procedures and then to write an integrated review that highlights these implications. Although you can find literature reviews that consist simply of assessments of one published article after another—that never get beyond stage one in the review process— your understanding of the literature and the quality of your own work will be much improved if you make the effort to write an integrated review.

In the next two sections, we will show how you might answer many of the questions in Appendix A as we review a research article about reading. We will then show how the review of a single article can be used

61

 

 

within an integrated review of the body of prior research on this research question. Because at this early point in the text you will not be familiar with all the terminology used in the article review, you might want to read through the more elaborate article review in Appendix B later in the course.

A Single-Article Review: Home Literacy Activities and Signs of Children’s Emerging Literacy

Christine Winquist Nord, Jean Lennon, Baiming Liu, Westat, and Kathryn Chandler (1999) of the National Center for Education Statistics (NCES) analyzed data from 1993 and 1999 concerning family literacy practices in the home and their effects on emerging literacy in young children. In this section, we will examine the article that resulted from that analysis, which was published by the Office of Educational Research and Improvement of the U.S. Department of Education. This type of report is very common in education, where large-scale survey data sets gathered by state or federal agencies produce mountains of statistics that require analysis integrating the survey outcomes with important questions in the field and with previous research.

The Research Question

The data used were from the 1993 and 1999 administrations of the National Household Education Survey for 3-, 4-, and 5-year-olds not enrolled in kindergarten, in which “the parent most knowledgeable about the child, usually the child’s mother” was asked a standard set of questions about literacy practices in the home and “school readiness skills” (Nord et al., 1999, p. 2). Literacy practices surveyed included reading to the child; telling a story; teaching letters, words, or numbers; teaching songs or music; doing arts and crafts; and visiting a library in the past month. School readiness skills surveyed included “recognizes all letters,” “counts to 20 or higher,” “writes name,” and “reads or pretends to read storybook” (p. 7).

There is no explicit discussion of ethical guidelines in the article, although reference is made to a more complete unpublished report. Clearly, important ethical issues had to be considered, given the focus on what is going on in the private space of the home across a wide range of income, racial, and ethnic groups, but the adherence to standard survey procedures suggests attention to these issues.

The Research Design

Asking the same set of questions of statistically similar groups (e.g., mothers with 3-, 4-, and 5-year-olds) at periodic intervals (in this case, 6 years) is a standard way of measuring change over time. The survey design was careful to match the 1993 and 1999 groups of mothers in such important areas as race/ethnicity, home language, education level, employment status, family type (one, two, or no parents present in home), and poverty status. The analysis acknowledged some unavoidable limitations in the design, including the fact that “parents may overestimate both their involvement in home literacy activities and their children’s skills because they recognize that such activities and skills are socially desirable” as a source of non-sampling error (Nord et al., 1999, p. 2).

The article’s discussion embedded the survey findings in a theoretical framework drawn from other research studies on emerging literacy that have shown, for instance, that “children begin the process of learning to read

62

 

 

long before they enter formal schooling” and “families, and parents in particular, play an important role in this

process” (Nord et al., 1999, p. 1). The choice of literacy activities surveyed comes in part from research showing that “reading to children, telling them stories, and singing with them” helps them to learn (p. 1).

The Research Findings and Conclusion

The article’s report on findings was framed in the context of Goal One of the National Education Goals —“the importance of family-child engagement in literacy activities to children’s learning and readiness for school” (Nord et al., 1999, p. 1). Overall, the analysts felt that survey data showed that “families have gotten the message about the importance of reading to their young children. Eighty-two percent of children ages 3 to 5 years in 1999 who were not yet enrolled in kindergarten were read to three or more times in the last week by a family member” (p. 1). Many families also engaged in other literacy activities. Of course, readers of this analysis who are skeptical about this high figure may wonder about parents’ acknowledged tendency to overestimate their involvement. The report also outlines differences in families’ engagement by the children’s race and ethnicity but found “no statistically significant differences” in results for “black, non-Hispanic children and white, non-Hispanic children Hispanic children, though, are significantly less likely than either white, non-Hispanic or black, non-Hispanic children” to have done the specified literacy activities with their families (pp. 2, 5).

Overall, this NCES study represents an important contribution to understanding the changing dynamics of families and literacy. Although there are certainly limitations to what can be discovered through such a survey, the national scope of the data and the fact that it was repeated at a regular interval under controlled sampling conditions give it a high level of usefulness. It is not the last word, by any means, but forms an important contextual background to more close-up, detailed studies by other researchers of individual classrooms, students, and families. It is not hard to understand why such studies continue to stimulate further research and ongoing policy discussions.

An Integrated Literature Review: Home and School Factors in Reading

The goal of the second stage of the literature review process is to integrate the results of your separate article reviews and develop an overall assessment of the implications of prior research. The integrated literature review should accomplish three goals:

1. Summarize prior research. 2. Critique prior research. 3. Present pertinent conclusions. (Hart, 1998, pp. 186–187)

We’ll discuss each of these goals in turn.

Summarize Prior Research

Your summary of prior research must focus on the particular research questions that you will address, but you also may need to provide some more general background. Jeanne R. Paratore (2002) begins her research review of home and school effects on early readers with a section labeled “Early Beliefs About Parents and

63

 

 

Children’s Literacy.” In this section, she first describes the evolution of, then questions, the belief that what

parents do or don’t do has a large role in whether their children succeed in school. She then reviews different theories and supporting research studies on parent and school roles that have influenced classroom instruction, school district policies, and construction of basal reader textbook series.

Ask yourself three questions about your summary of the literature:

1. Have you been selective? If there have been more than a few prior investigations of your research question, you will need to narrow your focus to the most relevant and highest quality studies. Don’t cite a large number of prior articles “just because they are there.”

2. Is the research up-to-date? Be sure to include the most recent research, not just the “classic” studies. 3. Have you used direct quotes sparingly? To focus your literature review, you need to express the key points

from prior research in your own words. Use direct quotes only when they are essential for making an important point (Pyrczak, 2005, pp. 51–59).

Critique Prior Research

Evaluate the strengths and weaknesses of the prior research. In addition to all the points you develop as you answer the “article review questions” in Appendix A, you should also select articles for review that reflect work published in peer-reviewed journals and were written by credible authors who have been funded by reputable sources. Consider the following questions as you decide how much weight to give each article:

1. How was the report reviewed prior to its publication or release? Articles published in academic journals go through a rigorous review process, usually involving careful criticism and revision. Top “refereed” journals may accept only 10% of submitted articles, so they can be very selective. Dissertations go through a lengthy process of criticism and revision by a few members of the dissertation writer’s home institution. A report released directly by a research organization is likely to have had only a limited review, although some research organizations maintain a rigorous internal review process. Papers presented at professional meetings may have had little prior review. Needless to say, more confidence can be placed in research results that have been subject to a more rigorous review.

2. What is the author’s reputation? Reports by an author or team of authors who have published other work on the research question should be given somewhat greater credibility at the outset.

3. Who funded and sponsored the research? Major federal funding agencies and private foundations fund only research proposals that have been evaluated carefully and ranked highly by a panel of experts. They also often monitor closely the progress of the research. This does not guarantee that every such project report is good, but it goes a long way toward ensuring some worthwhile products. On the other hand, research that is funded by organizations that have a preference for a particular outcome should be given particularly close scrutiny (Locke, Silverman, & Spirduso, 1998, pp. 37–44).

Present Pertinent Conclusions

Don’t leave the reader guessing about the implications of the prior research for your own investigation. Present the conclusions you draw from the research you have reviewed. As you do so, follow several simple

64

 

 

guidelines (Pyrczak, 2005, pp. 53–56). Distinguish clearly your own opinion of prior research from conclusions of the authors of the articles you have reviewed (Fink, 2005, pp. 190–192).

Make it clear when your own approach is based on the theoretical framework you are using rather than on the results of prior research. Acknowledge the potential limitations of any empirical research project. Don’t emphasize problems in prior research that you cannot avoid either. Explain how the unanswered questions raised by prior research or the limitations of methods used in prior research make it important for you to conduct your own investigation.

A good example of how to conclude an integrated literature review is provided by Paratore (2002), who begins her closing section by posing and then addressing the provocative question, “So, what have we learned about parents and their role in children’s literacy learning that we did not know or understand 10 or 20 years ago?” (p. 56). This approach gives her a chance to look back at the studies she has reviewed, then forward to new studies, concluding with, “When all is said and done, the teacher makes the difference” (p. 65).

65

 

 

The Role of Educational Theory

With a research question formulated and a review of the pertinent literature taking shape, we are ready to consider the process of conducting our research.

When we conduct educational research, we are attempting to connect theory with empirical data—the evidence we obtain from the educational world. Researchers may make this connection by starting with an educational theory and then testing some of its implications with data. This is the process of deductive research; it is most often the strategy used in quantitative methods. Alternatively, researchers may develop a connection between educational theory and data by first collecting the data and then developing a theory that explains patterns in the data (see Exhibit 2.5). This inductive research process is more often the strategy used in qualitative methods. As you’ll see, a research project can draw on both deductive and inductive strategies.

We have already pointed out that educational theory can be a source of research questions and that it plays an important role in literature reviews. What deserves more attention at this point is the larger role of educational theory in research. Theories are logically interrelated sets of propositions that help us make sense of many interrelated phenomena and predict behavior or attitudes that are likely to occur when certain conditions are met; they help educational scientists to know what to look for in a study and to specify the implications of their findings for other research. Building and evaluating theory is therefore one of the most important objectives of educational research.

Exhibit 2.5 The Links between Theory and Data

Most educational research is guided by some theory, although the theory may be only partially developed in a particular study or may even be unrecognized by the researcher. Educational theories do not provide the answers to the questions we pose as topics for research. Instead, educational theories suggest the areas on which we should focus and the propositions that we should consider for a test. Educational theory makes us much more sensitive to the possible directions for research and so helps us to design better research and draw

66

 

 

out the implications of our results. Before, during, and after a research investigation, we need to keep thinking theoretically.

In Chapter 1, we discussed and showed examples of three types of research: explanatory, exploratory, and descriptive. Exhibit 1.4 in that chapter showed relationships between these types of research and their goals and methods. We will now look at a different relationship, the one between the three types of research and theoretical approaches—specifically, deductive and inductive methods of looking at a research question.

67

 

 

Explanatory Research

The process of conducting research designed to test explanations for educational phenomena involves moving from theory to data and then back to theory. This process can be characterized with a research circle (Exhibit 2.6).

68

 

 

Deductive Research

As Exhibit 2.6 shows, in deductive research, a specific expectation is deduced from a general theoretical premise and then tested with data that have been collected for this purpose. We call the specific expectation deduced from the more general theory a hypothesis. It is the hypothesis that researchers actually test, not the complete theory itself. A hypothesis proposes a relationship between two or more variables—characteristics or properties that can vary. Variation in one variable is proposed to predict, influence, or cause variation in the other variable. The proposed influence is the independent variable; its effect or consequence is the dependent variable. After the researchers formulate one or more hypotheses and develop research procedures, they collect data with which to test the hypothesis.

Exhibit 2.6 The Research Circle

Hypotheses can be worded in several different ways, and identifying the independent and dependent variables is sometimes difficult. When in doubt, try to rephrase the hypothesis as an “if-then” statement: “If the independent variable increases (or decreases), then the dependent variable increases (or decreases).”

Both explanatory and evaluative studies are types of deductive research. Rachel Brown, Michael Pressley, Peggy Van Meter, and Ted Schuder (1996) were aware of the “seminal discovery” in earlier, descriptive research that “American students received little instruction about how to comprehend text” (p. 19). They designed a study to test the hypothesis that formal instruction in comprehension would improve the reading of “low-achieving” second graders more than traditional approaches that lacked comprehension instruction.

Hypothesis: A tentative statement about empirical reality, involving a relationship between two or more variables.

Example of a hypothesis: Students who receive instruction in specific reading comprehension strategies will improve their reading more than students who do not receive such instruction.

Variable: A characteristic or property that can vary (take on different values or attributes).

Example of a variable: grade level of students.

69

 

 

Independent variable: A variable that is hypothesized to cause, or lead to, variation in another variable.

Example of an independent variable: type of reading comprehension strategy.

Dependent variable: A variable that is hypothesized to vary depending on, or under the influence of, another variable.

Example of a dependent variable: amount of reading improvement.

They recruited five teachers willing to use a specially designed “transactional strategies” instructional package, called SAIL, and five “comparison” teachers who would use their regular instructional methods (p. 19). The study lasted the entire instructional year. The researchers collected four types of data: a “strategies interview” with students, a set of “retelling questions” to assess change in retelling and sequencing skills, a “think-aloud task,” and standardized measures of comprehension and word skills (p. 20). Overall, the researchers found that the impact of SAIL was positive in both the short and long term. In the short term, SAIL students were better than control group students in acquiring information from what they read and having a deeper understanding. In the long term, SAIL students knew more reading strategies and had higher scores on standardized tests at the end of the year.

In both explanatory and evaluative research, the statement of expectations for the findings and the design of the research to test these expectations strengthen the confidence we can place in the test. The deductive researcher shows her hand or states her expectations in advance and then designs a fair test of those expectations. Then, “the chips fall where they may”—in other words, the researcher accepts the resulting data as a more or less objective picture of reality.

The relationship between hypothesis, independent variable, and dependent variables in the Brown et al. (1996) study is shown in Exhibit 2.7. The patterns Brown and her colleagues found in their data, or their empirical generalizations, were consistent with the hypothesis that the researchers deduced from their theoretical assumptions and from previous research showing a lack of comprehension instruction (Exhibit 2.7). The theory thus received support from the experiment.

70

 

 

Inductive Research

In contrast to deductive research, inductive research begins with specific data, which are then used to develop (induce) a general explanation (a theory) to account for the data. One way to think of this process is in terms of the research circle: Rather than starting at the top of the circle with a theory, the inductive researcher starts at the bottom of the circle with data and then develops the theory. Another way to think of this process is represented in Exhibit 2.8, using social context theory—the theory that a child’s language development is heavily influenced by the social context in which the child grows up. Specific predictions about language development are deduced from general theories about development. In deductive research, reasoning from specific premises results in a conclusion that a theory is supported, while in inductive research, the identification of similar empirical patterns results in a generalization about some educational process.

An Inductive Approach to Changes in Reading Instruction

In 2000, James F. Baumann, James V. Hoffman, Ann M. Duffy-Hester, and Jennifer Moon Ro published a modern-day replication of a classic reading study from the 1960s (Austin & Morrison, 1963). Data from the 2000 study showed many similarities in reading instruction to the 1960s, including significant amounts of time devoted to reading at the elementary level, explicit instruction in phonics, the administration of standardized tests, and difficulties in meeting the challenges posed by underachieving readers. The new survey also showed significant differences: a balanced and eclectic approach among teachers rather than strict skills instruction, more whole-class instruction, and much greater use of trade books than in the 1960s. Many teachers had adopted an emergent literacy perspective, there were more and better classroom libraries in use, and schools were much more likely to make changes in their reading programs than they were in the 1960s.

Many of the major changes described have roots in theoretical approaches such as social context theory,

71

 

 

emergent literacy, whole language, and balanced phonics instruction. The Baumann team’s research thus offers some evidence of the extent to which such reading theories are actually being adopted in schools.

72

 

 

Exploratory Research

Qualitative research is often exploratory and, hence, inductive from beginning to end. The researchers begin by observing educational interaction or interviewing educational actors in depth and then developing an explanation for what has been found. The researchers often ask questions such as, “What is going on here?” “How do people interpret these experiences?” or “Why do people do what they do?” Rather than testing a hypothesis, the researchers are trying to make sense of some educational phenomenon. They may even put off formulating a research question until after they begin to collect data—the idea is to let the question emerge from the situation itself (Brewer & Hunter, 1989, pp. 54–58).

Of course, the research questions that serve as starting points for qualitative data analyses do not simply emerge from the setting studied but are shaped by the investigator. As Harry Wolcott (1995) explains,

[The research question] is not embedded within the lives of those whom we study, demurely waiting to be discovered. Quite the opposite: We instigate the problems we investigate. There is no point in simply sitting by, passively waiting to see what a setting is going to “tell” us or hoping a problem will “emerge.” (p. 156)

Our focus on the importance of the research question as a tool for guiding qualitative data analyses should not obscure the creative nature of the analytic process. The research question can change, narrow, expand, or multiply throughout the processes of data collection and analysis.

Explanations derived from qualitative research will be richer and more finely textured than they often are in quantitative research, but they are likely to be based on fewer cases from a limited area. We cannot assume that the people studied in this setting are like others or that other researchers will develop explanations similar to ours to make sense of what was observed or heard. Because we do not initially set up a test of a hypothesis according to some specific rules, another researcher cannot come along and conduct the same test.

73

 

 

Descriptive Research

You learned in Chapter 1 that some educational research is purely descriptive. The study on emergent literacy discussed earlier in this chapter is an example. Such research does not involve connecting theory and data, but it is still a part of the research circle—it begins with data and proceeds only to the stage of making empirical generalizations based on those data (see Exhibit 2.6).

Valid description is important in its own right—in fact, it is a necessary component of all investigations. Much important research for the government and public and private organizations is primarily descriptive: What percentage of parents surveyed read to their children at least three times per week? Were there more or fewer home literacy practices in 1999 than there were in 1993? Do single-parent homes report as many literacy practices as two-parent homes? Simply put, good description of data is the cornerstone of the scientific research process and an essential component for understanding the educational world.

Good descriptive research can also stimulate more ambitious deductive and inductive research. Descriptive research showing performance differences on achievement tests based on race has fueled a great many “achievement gap” studies in recent years. This research has helped to establish priorities for both public policy and additional research.

74

 

 

Educational Research Goals

Educational researchers strive to fulfill three goals: validity, authenticity, and practical significance.

75

 

 

Validity

Measurement validity: Exists when a measure measures what we think it measures.

Generalizability: Exists when a conclusion holds true for the population, group, setting, or event that we say it does, given the conditions that we specify.

Causal validity (internal validity): Exists when a conclusion that A results in B is correct.

We have reached the goal of validity when our conclusions about empirical reality are correct. In the chapters that follow, we will be concerned with three aspects of validity: measurement validity, generalizability, and causal validity (also known as internal validity). We will learn that invalid measures, invalid generalizations, or invalid causal inferences result in invalid conclusions.

Measurement Validity

Measurement validity is our first concern in establishing the validity of research results, because without having measured what we think we measured, we really don’t know what we’re talking about. Measurement validity is discussed further in Chapter 4.

Problems with measurement validity can result for many reasons. In studies of TV viewing and learning such as those discussed in Chapter 1, relying on parents’ reports of TV viewing by their children results in underestimates. Instead, researchers have used timing devices connected to the family TV(s). It is also difficult to obtain valid measures of the amount of time parents or other caregivers spend interacting with children. Some researchers question people repeatedly over a period of time to avoid inaccuracies that can occur when respondents are asked to recall what they did at some time in the past.

We must be very careful in designing our measures and in subsequently evaluating how well they have performed. We cannot just assume that measures are valid.

Generalizability

The generalizability of a study is the extent to which it can be used to inform us about persons, places, or events that were not studied. We rarely have the resources to study the entire population that is of interest to us, so we have to select a sample of cases so that our findings can be generalized to the population of interest. When many studies using different cases or in different settings produce similar results, conclusions can have a high level of generalizability. Generalizability is a key concern in research design and is the focus of Chapter 5.

Causal Validity

Causal validity, also known as internal validity, refers to the truthfulness of an assertion that A causes B. Educational researchers frequently must be concerned with causal validity. As we have seen, much educational research focuses on causal questions such as, “What influence does television watching have on aggressive

76

 

 

behavior?” and “What impact does direct teaching of reading comprehension strategies have on children’s reading development?”

You’ll learn more about these different aspects of validity in Chapters 4 and 5. The main take-away message about validity for now is that the goal of educational research is not to come up with conclusions that people will like or conclusions that suit our own personal preferences but to figure out how and why some aspect of the educational world operates as it does.

77

 

 

Authenticity

You already know that the educational world is complex. You may have wondered whether investigations can really be objective, because no matter how you look at educators and educational practices, you are doing so through your own eyes and developing interpretations that invariably will draw on your own life and prior experiences. Many educational researchers share your skepticism about the possibility of achieving a valid understanding of the educational world—of finding out how the educational world really operates. These researchers do not accept validity, as it is commonly defined, as the goal for research.

For researchers who feel that attempts to understand the educational world are inevitably subjective and cannot give us much confidence that we have learned the reality of that world, authenticity is an alternative research goal. An authentic understanding of an educational process or educational setting is one that reflects fairly the various perspectives of participants in that setting (Gubrium & Holstein, 1997). In fact, authenticity is a worthy goal for all educational research endeavors.

Authenticity: When the understanding of an educational process or educational setting is one that reflects fairly the various perspectives of participants in that setting.

Authenticity is often the goal for qualitative research investigations, and it reflects the belief that those who study the educational world can hope to understand only how others view that educational world. From this perspective, every observer sees the educational world from his or her own vantage point; there is no basis for determining that one perspective is the “valid” one. “The conception of knowledge as a mirror of reality is replaced by knowledge as a linguistic and social construction of reality” (Kvale, 2002, p. 306).

Practical significance: When research answers the “So what?” question and affects the everyday practice of teachers, administrators, and policy makers.

78

 

 

Practical Significance

Researchers have an obligation to ask questions and produce conclusions that are in some way useful to others, particularly to moving forward the collective enterprise of teaching and learning. The goal of practical significance must be addressed in both the design and conclusions of good educational research.

In recent years, two developments have raised awareness of practical significance as a goal for researchers. First, the federal No Child Left Behind legislation has placed pressure on the states to raise student test scores and to find and use only “research-proven” methods to make schools better. Second, participatory research methods that are school based and classroom based have become increasingly popular. Using these methods, teachers are investigating their own practice and using the results to modify instruction and curriculum in their own classrooms and schools.

79

 

 

Educational Research Proposals, Part I

Be grateful for those people or groups who require you to write a formal research proposal (as hard as that seems), and be even more grateful for those who give you constructive feedback. Whether your proposal is written for a professor, a thesis committee, an organization seeking practical advice, or a government agency that funds basic research, the proposal will force you to set out a problem statement and a research plan. So even in circumstances when a proposal is not required, you should prepare one and present it to others for feedback. Just writing your ideas down will help you to see how they can be improved, and almost any feedback will help you to refine your plans.

Each chapter in this book includes “Developing a Research Proposal” exercises that will guide you through the process of proposal writing. This section presents an overview of the process of proposal writing that also serves as an introduction to these special end-of-chapter exercises. The last chapter in the text (Chapter 15) contains a wrap-up discussion of the entire proposal preparation process.

Every research proposal should have at least five sections. The following list is adapted from Locke, Spirduso, and Silverman (2000, pp. 8–34):

1. An introductory statement of the research problem, in which you clarify what it is that you are interested in studying

2. A literature review, in which you explain how your problem and plans build on what has already been reported in the literature on this topic

3. A methodological plan, detailing just how you will respond to the particular mix of opportunities and constraints you face

4. An ethics statement, identifying human subjects issues in the research and how you will respond to them in an ethical fashion

5. A statement of limitations, reviewing weaknesses of the proposed research and presenting plans for minimizing their consequences

You will also need to include a budget and project timeline, unless you are working within the framework of a class project.

If your research proposal will be reviewed competitively, it must present a compelling rationale for funding. It is not possible to overstate the importance of the research problem that you propose to study (see the first section of this chapter). If you propose to test a hypothesis, be sure that it is one for which there are plausible alternatives. You want to avoid focusing on a “boring hypothesis”—one that has no credible alternatives, even though it is likely to be correct (Dawes, 1995, p. 93).

A research proposal also can be strengthened considerably by presenting results from a pilot study of the research question. This might have involved administering the proposed questionnaire to a small sample, conducting a preliminary version of the proposed experiment with a group of students, or making observations over a limited period of time in a setting such as that proposed for a qualitative study. Careful presentation of

80

 

 

the methods used in the pilot study and the problems that were encountered will impress anyone who reviews the proposal.

Don’t neglect procedures for the protection of human subjects. You will learn much more about this in our next chapter, “Ethics in Research.” But even before you begin to develop your proposal, you should find out what procedure your university’s institutional review board (IRB) requires for the review of student research proposals. Follow those procedures carefully, even if they require that you submit your proposal for an IRB review. No matter what your university’s specific requirements are, if your research involves human subjects, you will need to include in your proposal a detailed statement that describes how you will adhere to these requirements.

You have learned in this chapter how to formulate a research question, review relevant literature, consider ethical issues, and identify some possible research limitations, so you are now ready to begin proposing new research. If you plan to do so, you can use the proposal exercises at the end of each subsequent chapter to incorporate more systematically the research elements discussed in those chapters. By the book’s end, in Chapter 15, you will have attained a much firmer grasp of the various research components. At that point, we will return to the process of proposal writing.

81

 

 

Conclusions

We began this chapter with the question, “How does a child learn to read?” In the course of the chapter, you saw widely different approaches researchers have taken to studying this question. These included a review of research on fluency in reading (Kuhn & Stahl, 2003), ERIC and Google searches on topics in early childhood reading, a review of research on home and school effects on early readers (Paratore, 2002), a deductive research study on the effects of instruction in reading comprehension (Brown et al., 1996), and a replication, 40 years later, of a classic study of the types of reading instruction used in American elementary schools (Baumann et al., 2000).

Selecting a worthy research question does not guarantee a worthwhile research project. The simplicity of the research circle presented in this chapter belies the complexity of the educational research process. Of course, our answers to research questions will never be complete or entirely certain. Thus, when we complete a research project, we should point out how the research could be extended and evaluate the confidence we have in our conclusions. The elaboration of knowledge about complex research topics requires recognizing research difficulties, carefully weighing evidence, and identifying unanswered questions.

Ethical issues, one of the main topics of Chapter 3, also must be considered when evaluating research proposals and completed research studies. Ethical issues in educational research are no less complex than the other issues that researchers confront. It is inexcusable to jump into research on people without any attention to ethical considerations.

We hope that you will return often to this chapter as you read the subsequent chapters, when you criticize the research literature, and when you design your own research projects. To be conscientious, thoughtful, and responsible—this is the mandate of every educational researcher. If you formulate a feasible research problem, ask the right questions in advance, adhere to the research guidelines, and steer clear of the most common difficulties, you will be well along the road to fulfilling this mandate.

82

 

 

Key Terms

Authenticity 38 Deductive research 34 Dependent variable 35 Empirical generalization 36 Generalizability 38 Hypothesis 35 Independent variable 35 Inductive research 36 Institutional review board (IRB) 41 Practical significance 38 Replication 37 Research circle 34 Theory 36 Validity 38 Variable 35

83

 

 

Highlights

 

“Looking for a Similar Assignment? Get Expert Help at an Amazing Discount!”

The post Research Methods in Education appeared first on nursing writers.

 

"Is this question part of your assignment? We Can Help!"

Essay Writing Service