Causal Inference and Experimental Design

Reply Responses

1) What are the three criteria for inferring causality? Just want to make sure you all have this down because it’s so important to know in order to understand everything else in this chapter.

As stated in Rubin & Babbie (2017, p. 245), the three most commonly cited criteria for inferring causality are:

1) Time sequence (the cause must precede the effect)

2) Correlation (changes in the cause must go along with changes in the effect)

3) Alternative explanations are ruled out

2) Pick three of the primary threats to internal validity discussed in this chapter and make up new examples to illustrate each.

According to Rubin & Babbie (2017, p. 249), history is a threat to internal validity. They define history as extraneous events that coincide in time with the manipulation of the independent variable in the study. An example of history would be a physician prescribes a patient weight loss medication and asks the patient to come back in a month for a recheck. During that month, the patient joins a gym and follows a regular exercise schedule. At the recheck, the patient has lost 10 lbs. The physician cannot be sure that the weight loss occurred because of the medication, the exercise, or a combination of both.

Another threat to internal validity is maturation. Maturation is defined by Rubin & Babbie (2017, p. 249) as the effects of the passage of time. An example of maturation is a counselor studying the impact of a depression intervention on children of divorced parents. An 8-year-old child asked about their parent’s divorce will likely have a very different outlook than a 15-year-old.

One more threat to internal validity is selection biases. Rubin & Babbie (2017, p. 251) note that selection biases occur when groups being compared are not truly comparable. An example of this is using two groups to study a drug intervention therapy program. One group is court-ordered to participate in the intervention, while the other does so voluntarily. There will likely be a higher success rate in the voluntary group as they are more inclined to participate fully.

3) BRIEFLY describe a hypothetical experimental design for testing a new intervention in a social work agency with which you are familiar and how your design controls for threats to internal validity.

A hospice organization implements a pet therapy program to help patients in nursing home facilities overcome loneliness and depression. A study is performed to determine if patients who receive the visits show fewer symptoms of loneliness and depression. A pre-test/posttest control group design with one group receiving visits (experimental group), and one group not receiving visits (control groups) is used. The exact same test is administered to each individual in both the control and experimental groups before visits, and one month after pet therapy visits begin.

Participants for each group will be chosen randomly to avoid selection bias in the groups. The researcher will also need to take into account history when looking at results to determine if there were any external events during the month of the study that may have skewed the outcome. For instance, did a participant receive an unplanned visit from a family member? This would account for a positive change in levels of loneliness and depression that may not be attributed to pet therapy intervention.

4) What potential threats to the validity of the findings can you detect in the following hypothetical design? In a residential treatment center containing four cottages, the clinical director develops a new intervention to alleviate behavior problems among the children residing in the four cottages. The clinical director selects two cottages to receive the new intervention. The other two will receive the routine treatment. To measure outcomes, the clinical director assigns a social work student whose field placement is at the center to spend an equal amount of time at each cottage observing and recording the number of antisocial behaviors each child exhibits and the number of antisocial statements each makes.

Rubin & Babbie (2017, p. 262) point out that measurement bias is a threat to the validity of findings in that when using research staff to supply ratings whether, through direct observation or interviews, the individual should not know the experimental status of the participants they are rating as results can be biased. In the example above, the field intern may unknowingly look for favorable results for the intervention being studied as they would want to make a positive impression on the organizational leaders during field placement.

Another threat to the validity of findings is the research reactivity (Rubin & Babbie, 2017, p. 263) and the way that the field intern conduct and obtrusive or unobtrusive observation. The children may behave differently depending on the method used.

Reference

Rubin, A., & Babbie, E. R. (2017). Research methods for social work (9th ed.). Boston: Cengage Learning.

1) What are the three criteria for inferring causality? Just want to make sure you all have this down because it’s so important to know in order to understand everything else in this chapter.

In our text (Rubin and Babbie, 2017, p..245)., the three criteria for inferring causality is described as the following:

• Time Sequence- the time order linking two variables must be clear and the cause must precede the affect
• Correlation- changes in the cause must lead to changes in the effect
• Alternative must be ruled out- any alternative explanations to the outcome can question the intervention’s causality

It is also important to note that the three addition criteria are also important the in strength of causality in a study. Those criteria are strength of correlation, consistency in replication, and plausibility and coherence.

2) Pick three of the primary threats to internal validity discussed in this chapter and make up new examples to illustrate each.

• History- A study was conducted to determine the quality of meals provided by a homeless day facility. The individuals were asked to rate the quality, portions, appropriate temperature, and overall taste of the food on a 1 to 5 scale. The homeless participants also receive mail at the facility’s address and while the study was being conducted the participants also received their stimulus checks in the mail. Receiving $1200 at a homeless day facility may skew their opinion of the food quality to be more positive than if they had not received any money during the study.
• Passage of Time- participants who receive counseling as a result of a road rage incident are asked to complete a questionnaire about the effectiveness of counseling, before, halfway, and after counseling sessions have ended. The results may show an improvement in the effectiveness because the effects of a road rage incident are most intense immediately following the event and the strong feelings associated with the event naturally diminish over time. Therefore, the rage may have diminished without any intervention.
• Instrumentation Changes- the use of a questionnaire, scaled 1-5 to addresses work morale is implemented first, and then unobtrusive observation of staff conversations or body language during staff meetings is used after the intervention to determine if the intervention had an effect on staff morale. The two forms of instrumentation pose a threat to internal validity because two different instruments are used.

3) BRIEFLY describe a hypothetical experimental design for testing a new intervention in a social work agency with which you are familiar and how your design controls for threats to internal validity.

Experimental Design- A group of clients who receive food from a social service agency’s food bank is studied to determine if a monthly phone call correlates with attendance at the food bank. The phone call will only provide days and hours of operation for the food bank. A group will receive the call and the attendance of that group will be compared to the group who does not receive the call.

Randomization- used to control selection bias to ensure the comparison group is similar to the control group. The individuals used are all participants who receive food from the agency instead of comparing them to people who did not receive food from the food bank.

Instrumentation- consistent use of measurement is needed. Attendance will be taken when the food is disbursed to determine how many people attend to receive food. On the second observation, after the intervention is implemented, attendance will be taken again and the outcomes will be compared. If the researcher takes attendance as the first observation, and uses a survey as the second instrument of measurement, the validity is threatened because the form of instrumentation is not consistent.

History- Outside effects such as, access to transportation to the agency, or approval for food stamps may affect the need for the food bank and therefore affect attendance. These are outside influences that can affect the need to come to the agency for food. Also, whether or not the person called spoke with someone, receive a voicemail, or actually had a working phone at the time of the call can also affect the internal validity of the procedure.

4) What potential threats to the validity of the findings can you detect in the following hypothetical design? In a residential treatment center containing four cottages, the clinical director develops a new intervention to alleviate behavior problems among the children residing in the four cottages. The clinical director selects two cottages to receive the new intervention. The other two will receive the routine treatment. To measure outcomes, the clinical director assigns a social work student whose field placement is at the center to spend an equal amount of time at each cottage observing and recording the number of antisocial behaviors each child exhibits and the number of antisocial statements each makes.

History- the change in a participant’s behavior can be attributed to having a visitation with family prior to the observation, having a fight with another resident prior to the observation, or taking mediation that has side effects such as drowsiness that may affect the participant’s behavior.

Selection bias- instead of picking which cottage receives the intervention, use randomization to determine which individuals receive intervention, therefore there will be members within each cottage that has the intervention.

Obtrusive observation- the presence of the student may affect the participants’ behavior

Measurement Bias- if the social work student is aware of the hypothesis or if the student knows which cottage received the intervention, the student may be inclined to record behaviors that are aligned with the predictions of the hypothesis

References

Rubin, A., & Babbie, E. R. (2017). Research methods for social work. Cengage Learning.