Kinds of Design

p {margin: 0; padding: 0;} .ft00{font-size:21px;font-family:LatoLatin;color:#202122;} .ft01{font-size:60px;font-family:LatoLatin;color:#202122;} .ft02{font-size:28px;font-family:LatoLatin;color:#202122;} .ft03{font-size:24px;font-family:Arial;color:#000000;} .ft04{font-size:24px;font-family:ArialMT;color:#000000;} .ft05{font-size:24px;font-family:Arial;color:#660000;} .ft06{font-size:24px;font-family:ArialMT;color:#0000ff;} .ft07{font-size:24px;font-family:ArialMT;color:#551a8b;} .ft08{font-size:24px;font-family:TimesNewRomanPS;color:#000000;} .ft09{font-size:24px;font-family:TimesNewRomanPSMT;color:#000000;} .ft010{font-size:37px;font-family:LatoLatin;color:#202122;} .ft011{font-size:24px;font-family:LatoLatin;color:#ffffff;} .ft012{font-size:21px;font-family:LatoLatin;color:#202122;} .ft013{font-size:28px;font-family:LatoLatin;color:#202122;} .ft014{font-size:21px;font-family:LatoLatin;color:#006fbf;} .ft015{font-size:28px;font-family:LatoLatin;color:#202122;} .ft016{font-size:10px;font-family:.SFUI;color:#808080;} .ft017{font-size:10px;font-family:.SFUI;color:#808080;} .ft018{font-size:60px;line-height:73px;font-family:LatoLatin;color:#202122;} .ft019{font-size:24px;line-height:30px;font-family:ArialMT;color:#000000;} .ft020{font-size:24px;line-height:29px;font-family:ArialMT;color:#000000;} Reflect in ePortfolio Download Print Langston, Research Methods, Notes 6 -- Kinds ofDesign (Ch. 2 to "Propositional Logic") You have viewed this topic Last Visited Sep 27, 2023 9:37 AM I. Goals A. Continuum of Designs B. Important Definitions C. Wrap-Up II. Continuum of Designs Figure 6.1 contains the continuum of designs. This allows us to see how the designs relate to one another. The letters indicate a design type that isdiscussed below. Figure 6.1. Continuum of designs. The labels on the ends of the continuum are related. On the left end we exert less control because we know less about the phenomenon being studied. Thisallows us to find relationships, but not to make causal conclusions. These designs are usually more natural because we don't want to accidentally controlvariables that might be important. On the right end we know more so we can exert more control and make causal conclusions. These designs are morelikely to be artificial to help gain control. The plan for the data collection part of the course is to discuss observation designs, then correlation/survey research, then experiments. After we haveworked out the details for how experiments allow causal conclusions, we will come back to quasi-experiments. For now, we will just discuss each designtype briefly. A. Observation designs Go to the world and carefully observe what happens. The researcher usually wants to exert less control to let the data tell the story. B. Correlation/Survey designs In a correlation design you have a little more control. You now know the variables you think matter and you are focusing on just those. A survey is usually acorrelation design, and you have even more control because you are asking questions. Note, however, that you are only measuring, you are not causing thebehaviors that you are studying. Therefore, the only conclusion you can make is that two variables appear to be related. You cannot make a causalconclusion. One of the most important things we will discuss in this class is that correlation does not imply causation. Why not? The problem is interpretive ambiguity. To make a causal conclusion you have to have a direct link between the cause and the effect. With a correlation, thereare three possible causal interpretations: 1. A causes B. 2. B causes A. 3. Some third factor causes both to change even though they are not related. For example, suppose I find a correlation between ghost experiences and belief in ghosts. There are three possible causal relationships. Maybeexperiencing a ghost causes you to believe in them. But, it is just as likely that believing in ghosts causes you to have a ghost experience. Or, a variable likeschizotypy might cause you to experience ghosts (because it is associated with seeing patterns) and cause you to believe in ghosts (because it isassociated with looser thinking). You may have a preferred interpretation, but all a correlation allows you to say is that there is a relationship. In a way, determining causality is the goal of research (e.g. if we know what causes depression we'll know how to treat it). Why do people do correlationresearch if correlation does not imply causation? There are three good reasons: 1. Ethics. If I wanted to investigate the relationship between self-esteem and body image, I could randomly assign one group to be low in self-esteem, butthat would require me to lower their self-esteem. This would violate the ethical principles we discussed in Notes 4. Similarly, it would not be possible tomanipulate childhood abuse to investigate its effect on adult psychological well-being. To investigate these questions, we would have to do correlationalresearch. 2. Generalizability . A good experiment requires careful control by the experimenter. Achieving this control can reduce the naturalness of the task. If an experimenter wants the conditions of the research to closely match those in the real world, a correlation design may be chosen. 3. Feasibility. Experiments can be difficult and expensive to conduct. The corresponding correlation research might be simpler because participantpopulations are readily available. In those cases, researchers do correlation research to fully explore the relationship between two variables. Once therelationship is understood, it is then possible to do an experiment to determine causality. Later in the semester we'll look at some research on therelationship between judged control and depression that used this approach. C. Experimental designs Simple description: Take two groups of people, give one group a treatment, do nothing to the other group, look to see what effect the treatment had relativeto the no treatment group. You can make causal conclusions after an experiment. The properties of an experiment are: 1. Random assignment . Everyone in the experiment has an equal chance of being in any group in the experiment. This makes the groups unbiased at the start. 2. Manipulate something. This is the key difference between correlation designs and experiments. You go beyond measuring to actual manipulation. 3. Measure something. Measurement is part of any design, but the variable measured in an experiment is related to the manipulation (and should bechanged by it). For example, if I think ghost experience causes ghost belief, I can measure people's ghost belief and find that it is low, randomly assign them to two groups,give one group a ghost experience but not the other, and measure their belief again. If belief goes up in the group that had the experience but not in theother group, then I can conclude that experience caused an increase in belief. D. Quasi-experimental designs Quasi-experimental designs offer more control than correlation/survey designs, but less control than experiments (they lack random assignment). The goalwith quasi-experiments is to see if we can apply what we learned about experiments to at least narrow the possible causal interpretations of the results. Wewon't be able to make causal conclusions, but we can put cause in a box (narrow the possibilities enough to act, even though we can't do an experiment). Try comprehension check 6.1 to see if you can tell the kinds of designs apart. Comprehension check 6.1 Try It for Yourself Identifying the design used by a research project is vitally important. For this assignment, go to PsycInfo and find an article (on the front page of the librarysite they have "Databases A-Z" in the "Quick Links" section; click that and then "P" and scroll down to PsycInfo; visit the library page here: https://library.mtsu.edu/home ). Look at enough of your article to figure out what its design is. When you have it figured out, post your analysis in Notes 6 Try It for Yourself comprehension check . E. Terms 1. Observation designs2. Correlation/survey designs3. Generalizability4. Experimental designs5. Random assignment6. Quasi-experimental designs Comprehension check 6.1 evaluation: The answers to the items in the comprehension check are mixed in below. a. I have a group of Type A people and a group of Type B people take an exam with the answers on the back and instruct them not to look at the answers. Iobserve that more Type A people do look at the answers, so I conclude that being Type A causes people to cheat. This is a correlation design because youhave identified two variables (personality and cheating) and measured them both. You cannot draw a causal conclusion.b. I randomly assign one group of middle-aged adults to exercise regularly and a second group not to exercise. I measure the incidence of Alzheimer’sdisease in the two groups and find less in the exercise group. I conclude that exercising causes you to not get Alzheimer’s. This is an experiment becauseyou have random assignment so you can draw a causal conclusion.c. I find that there’s a strong, positive relationship between a child’s self-esteem (measured on a Likert scale survey) and juvenile delinquency. I concludethat having high self-esteem causes delinquency. This is a correlation (survey). Correlation does not imply causation.d. I randomly assign one group of people to drive down Maple Street (2 lanes) and another group to drive down Memorial (4 lanes). The people on Memorialexceed the speed limit more than the people on Maple, so I conclude that driving on a wider street causes speeding. This has random assignment so it is anexperiment. You can make a causal conclusion.e. I observe people driving down Main Street and record their speed. I also record the type of car they drive. I find that people in sports cars speed morethan people in sedans. I conclude that driving a sports car causes speeding. This is an observation. You cannot make a causal conclusion without randomassignment and manipulation. III. Important Definitions Going forward, it will be easier if we have a shared vocabulary. It is in your best interest to memorize these terms. A. Types of variables 1. Independent variable (IV) : What you manipulate. a. Levels : IV's are made up of levels. Each level is an amount of whatever the variable is. For example, if you're varying the amount of study time, then the levels are the various amounts of time. Levels are arranged along a single dimension (the IV). You'll always have at least two levels. b. Treatment group : In a simple (two-group) experiment, one group gets some manipulation or treatment. This group is called the treatment group. c. Control group : The other group in the simple experiment gets no treatment or a placebo (a treatment that mimics the procedures used in the real treatment group, but should produce no effect). This is the control group. Let's consider a ghost experience experiment as an example. To provide a ghost experience, we will take participants on a ghost hunt and use an EMFmeter to carry out a conversation with a ghost. This meter is supposed to measure changes in electromagnetic radiation and is used to ask yes/no questionsand get responses. For example, you say "light it once for yes and twice for no." You can watch for responses on the meter. To see it an action, I've includeda video of me and Dale Swain from the Shadow Chasers of Middle Tennessee playing hide and seek with a ghost in the Old South Pittsburgh hospital. Video clip 6.1. Hide and seek with a ghost. For the treatment group we will use an EMF pump (it can be used to make the meter respond) to make responses to a standard script (so there will appearto be a conversation with a ghost). For the control group, we will do the same thing but there won't be any responses. Table 6.1 has the experiment so far. Group IV (ghost experience) Treatment Yes (ghost conversation) Control No (no ghost conversation) Table 6.1. The design of our ghost experience experiment with the IV. 2. Dependent variable (DV) : What you measure. Every variable in observation and correlation designs is a DV. In our experiment example, the DV will be ghost belief. How will we measure it? We know we want something that is reliable and valid. We will use a ghost belief scale from the literature that hasthese properties. Table 6.2 has the updated design. Group IV (ghost experience) DV Treatment Yes (ghost conversation) Ghost belief Control No (no ghost conversation) Ghost belief Table 6.2. The updated design of our ghost experience experiment with the DV. 3. Participant variable : Things that masquerade as IV's, but are actually only characteristics of the participants, and not variables that you (the experimenter) manipulate. For instance if you ask participants to report their gender and compare them on the basis of that, gender could look like an IV (ithas levels), but it's not manipulated, so it's technically the same as a correlation in terms of interpretation, and correlation does not imply causation. Because these look like IVs they can be very tricky. If you're trying to decide whether or not a causal conclusion is allowed, look for random assignment andlet that help you decide. 4. Controlled variable : These are sources of variation that can potentially affect your experiment; you control them to make them constant. For example, I might worry about prior ghost experience in the study we're developing. I'm trying to see if having an experience causes belief, so I'm going to limitparticipation to people who have not encountered a ghost in the past. That makes ghost experience a constant (they don't have any) and prevents it frombecoming a confound (see the next variable). 5. Confounding variable : These variables have the potential to undermine your experiment. They're things you should have controlled, but overlooked. At the end of the experiment, you don't know if changes were due to the IV or to the confound. These have two properties: a. Covary with the variable of interest (IV). As the IV changes the confound changes with it. b. Could reasonably be expected to have caused the change in the DV. Because the confound moves with the IV and could produce the change in the DV, you've introduced interpretive ambiguity and you can't make a causalconclusion. Was the change in belief due to the IV or the confound? B. Types of validity 1. Internal validity . This type of validity is related to the quality of the study. Did you have a confound? Did you control the right variables? What type of conclusion can you draw? 2. External validity (generalizability). How well will the results of your experiment generalize to the real world or the population you think you're studying? Alternatively, do the results of my experiment mean anything outside the context of my experiment? Note that internal and external validity compete. If you look at the continuum, you will note that experiments are on the end with a lot of control. Controlnecessarily makes it less natural. The video above is a real ghost investigation, and we can use that to make our experiment as real as possible, but we arescripting the questions and the answers. The more control we take, the less natural it gets and the weaker our external validity. As a consumer ofinformation, you should be aware of both types of validity and place the value of each where you think it best fits your purpose in looking at the research. C. Control of the experiment 1. Random sampling : When every member of the population of interest has an equal chance of being included in your experiment. This has an impact on generalizability. To the extent that your sample is representative of the population, your results hold for that population. 2. Random assignment : Every participant in your experiment has an equal chance to be in any condition in your experiment. This has an impact on internal validity. Random sampling gets participants into the study, random assignment puts them in their groups. You rarely see both. If random sampling is used, it usuallygoes with observation or correlation/survey research with a high value on generalizability. Experiments are usually more concerned with internal validity andworry less about external validity. D. Terms 1. Independent variable (IV)2. Levels3. Treatment group4. Control group5. Dependent variable6. Participant variable7. Controlled variable8. Confounding variable9. Internal validity10. External validity11. Random sampling12. Random assignment IV. Wrap-Up A. Review the steps in design This is a good time to check in on the progress we're making on the steps laid out in Notes 1. 1. Ask a question. Done. 2. Make a hypothesis. Done. 3. Collect observations. Next. 4. Analyze statistically. Mixed in with collecting observations. 5. Conclude. Mixed in with collecting observations. B. Five big things As we go through the semester there is going to be a list of five big things that I especially want you to know. When the class is over, these are the things Iwant you to take away. We now have our first of the five big things. 1. Correlation does not imply causation. Every time you hear someone talk about a correlation they are going to have a causal interpretation in mind andthey are usually going to make recommendations about correlational data implying a causal relationship. Do not be tricked. Langston Research Methods Notes 6 Research Methods - PSY-3070… TH TaLaija Hill Course Home Content Examity Zoom Videos My Evaluations ePortfolio Assessments Communica ! on Help Table of Contents Langston, Research Methods, Notes 6 -- Kinds of Design Langston, Research Methods, Notes 6 -- Kinds of Design (Ch. 2 to "Propositional Logic") Activity Details 10/3/23, 3 : 00 PM Page 1 of 1