The methods section of a research article is where the researcher tells about the study participants, the treatment they received, how they measured the results, and what statistical methods were used to measure the results. There are a number of things to consider here.

How Do The Children in the Study Compare to Your Child?

First, it’s important to consider the study participants.  If you are considering a specific study to see if it is the right choice for your child, it’s important to notice if the study participants are the same age as your child.  If the ages of the participants make up a big range (for example, 6 – 18 years), you should expect to find a table that breaks the numbers down into smaller age-ranges.  It’s important to consider how many of the study participants were the same age as your child.

Next, it’s important to consider if the study participants are similar to your child in their characteristics.

• What specific problems did the participants have and how does this align with your child?

• Does the author clearly define the problem?

•  Was the problem identified defined the same for each participant in the study?  In other words, were all the (1)participants diagnosed by the same people using the same criteria and methods or (2) were they diagnosed by multiple sources using multiple criteria and methods?  or (3) is this unclear?  , Beware if either 2 or 3 are true because the author may be comparing apples to oranges or kids with ADHD only to kids with ADHD, anxiety, and OCD.

• Look at the exclusionary criteria: were there participants who were excluded from the study? How do these participants compare to your child?

  In other words, would your child have been excluded from the study?

How Do the Results Compare to What You Want for Your Child?

When dealing with ADHD and related executive functions, it is important to ask yourself if the characteristics of the participants align with what you want your child is needing, and even more importantly, do the characteristics of the participants after treatment align with what you want for your child and what the seller of goods is claiming.

• For example, if the seller of a service or product is claiming that this service or product will help your ADHD child to not have to take medication (and this is important to you), did any or all of the participants in the study take medication before, during, or before the study?  If most or all did, the seller of the product really cannot make the claim that the product or service being sold will enable your child to not have to take medication.

 

How Does What Was Measured Help Your Child?

It is important to consider if what the author was measuring aligns with what you want for your child. In the procedures portion of the methods section, the author should give details about the types of measures used before and after the treatment.  If the author was measuring “fine motor coordination” and you are interested in improving your child’s sustained attention, working memory, or another executive functions, the results will not tell you if the treatment actually made the improvements you are seeking.

How Does the Study Support the Treatment You’re Getting?

Also in the procedures portion, they author will provide details about the treatment provided.  Two things are important to consider:

1. Is the treatment plan the seller of services in-line with the treatment provided in the research?  In other words, if the treatment plan being proposed is 20 sessions of neurofeedback and the study measured the impact of neurofeedback and direct and explicit training in reading comprehension, written expression, and math problem solving, you are strongly urged to ask for information about the research that supports the 20 sessions of neurofeedback only.

2. It is also important to consider what others (who are not trying to sell you a product or service) think about the treatment methods in the research article.

• You can get good quality information about strong research on ADHD from the National Resource Center on ADHD at http://www.help4adhd.org/en/about/wwk.  Look at the “What We Know” articles about proven treatments, alternative, and complimentary treatments.

• The University of Toronto and the Children’s Hospital in Toronto has strong research on meeting the educational needs of children with ADHD and related executive functions: http://www.teachadhd.ca/Pages/default.aspx.

• You can get good information about research based educational methods at the Institute of Educational Sciences What Works Clearing House: http://ies.ed.gov/ncee/wwc/.

• The National Institutes Of Mental Health has articles about numerous treatment methods: http://www.nimh.nih.gov/index.shtml.  Type “Executive Functions” in the search box at the top of the page to see the latest information.

How Do the Stats Add Up?

It is important to compare the statistical methods used in the methods section to the author’s discussion section. The statistical analyses section of the method section is where the author details the type of statistics used to analyze the data.  For example, look to see if the author describes using “correlation” data in the method section but then suggests some cause/effect in the discussion section.

Beware of the word “link” as this may make it sound as if there is a cause/effect relationship when this relationship may not truly exist.

For example, “correlations” can tell you if two (or more) factors studied are related in any way, but correlations cannot tell you if there is a cause/effect “link” between the two.  In other words if increased TV watching is highly correlated to a diagnosis of ADHD, watching more TV might have caused ADHD (not true), ADHD might have caused more TV watching, because kids with ADHD often can hyper-focus on things like TV, parents may use TV as a way to keep their ADHD children occupied and out of trouble, parents of ADHD kids may watch a lot of TV while ADHD kids are still awake, or it may just be some other coincidence that they are related.

If the author used “psycho-babble” and you’re not sure what they did for analyzing that stats, proceed with caution.

How Do You Know If the Treatment Was Effective?

The use of a controlled study is one way of establishing the effectiveness of  treatment.  In a controlled study, the sample of participants is split in two, with both groups being comparable in almost every way. The two groups then receive different treatments, and the outcomes of each group are assessed.

In the methods section, the author should talk about how he or she compared the participants getting the treatment to participants not getting the treatment in question.  This latter group is called the “control” group: beware a recent study does not detail how participants were assigned to a “treatment” group and a “control” group.  In this day and age, a study without a control group that is either “randomly assigned” or in some way matched as a way to control possible problems would be considered non-scientific in nature.

There are also more sophisticated statistical methods that can help researchers control certain aspects of their studies in order to better identify cause/effect.  For example, there may be issues that cannot be easily controlled.  To deal with these “confounding issues”, researchers can collect more information on each study participant . Then, they include this information into their statistical analyses and figure out if their findings still hold in light of these possibly complicating factors. In these types of studies, researchers will tell you which factors they controlled for. If they didn’t control for certain factors or  if  you can think of factors that might have had an impact that the researcher failed to control for, you should interpret the study findings with caution.

Need Additional Help?

If you need additional assistance with analyzing this aspect of research, please call 817.421.8780 to set up a consultation.

—————————————————

(c) 2013, Monte W. Davenport, Ph.D.