Attention Research Update

September 2013

"Helping parents, professionals and educators stay informed about new research on ADHD"

David Rabiner, Ph.D.  Research Professor, Duke University


                           FDA approves new brain wave test to help diagnose ADHD

Like all psychiatric disorders, ADHD is diagnosed based on the presence of behavioral symptoms that impair an individual's functioning, and not the results of a specific test.  In fact, evaluation guidelines from the American Academy of Pediatrics (AAP) state that no particular diagnostic test should be routinely used when evaluating a child for ADHD.

However, although no single test should be used in isolation to diagnose ADHD, the availability of an accurate objective test would be useful for several reasons.

First, many children do not receive a comprehensive assessment for ADHD and are diagnosed based on cursory evaluation procedures that are likely to result in frequent misdiagnoses.

Second, many parents would like an objective procedure be to part of their child's evaluation.  In fact, many parents doubt their child's diagnosis, and don't follow thru on treatment recommendations, because of the lack of objectivity they perceive in their child's evaluation.  You can find a review this issue at http://www.helpforadd.com/2006/january.htm

Finally, an objective test where negative findings highlight the need for further assessment prior to making an ADHD diagnosis could significantly reduce the number of children inappropriately placed on medication.

For these reasons, an accurate and objective diagnostic test for ADHD would be of value in many clinical situations. Two conditions would need to be met for such a test to be useful.

First, it would have to be highly sensitive to the presence of ADHD, i.e., individuals who have ADHD as determined by a comprehensive evaluation should score positive for ADHD on the test.  If the test were 100% sensitive, every individual who truly meets ADHD diagnostic criteria would score positive on the test.  As the sensitivity of a test drops, the number of "false negatives", i.e., normal test results in individuals who have the disorder, increases and its utility goes down.

Second, individuals who don't have ADHD should never score positive on the test, i.e., a positive result should occur only for individuals with ADHD and no one else.  When a diagnostic test has high specificity, individuals without the condition rarely score positive on the test.  When a test's specificity is low, many individuals without the condition will score positive and may be incorrectly diagnosed as a result. These are called "false positives".


Prior studies

Prior issues of Attention Research Update have summarized research examining the accuracy quantitative EEG (QEEG) as a diagnostic aide for ADHD was examined. These studies have found that many individuals with ADHD show a pattern of brain electrical activity referred to as "cortical slowing"; this is characterized by an elevation of low frequency theta waves and a reduction of higher frequency beta waves in the prefrontal cortex.  Theta wave activity is associated with an unfocused, inattentive state while beta activity is associated with more focused attention.  Thus, an elevated theta/beta ratio reflects a less alert and more unfocused state.

In a QEEG testing, EEG data is collected from a child or adult in a non-invasive procedure that requires 20-30 minutes. EEG data is digitized and computer scored so that an individual's theta/beta ratio can be computed and then compared to what is typical for individuals of similar age.  When this ratio is sufficiently elevated - the cut-off typically used is 1.5 standard deviations above average which corresponds to the highest 7% of the population - the individual is considered to have the EEG marker for ADHD.

You can find a summary of this work at http://www.helpforadd.com/2013/april.htm   Results from these studies suggest that QEEG results may improve the accuracy of ADHD diagnoses in children and adolescents.


Study that resulted in FDA approval

The study submitted for FDA review had a strong but somewhat complex design and understanding the logic of the design is essential for appreciating the potential value of the QEEG procedure.  (Note - The device tested is called the Neuropsychiatric EEG-Based Assessment Aid NEBA System, i.e., NEBA.  Throughout this article, I refer to it as QEEG).  Below I summarize the methods and findings as clearly as I can, omitting some details for the sake of clarity.  However, you can view the full report at http://www.accessdata.fda.gov/cdrh_docs/reviews/K112711.pdf

Here is an overview of what was done.

1. Subjects were children and adolescents who presented with attention and/or behavioral concerns to 13 geographically distinct clinics (5 Pediatric, 3 Psychological, and 5 Psychiatric) in the US. Of 364 subjects recruited, completed data was obtained on 275. All 275 subjects were included in the analyses.

2. At each site, clinicians used a comprehensive evaluation protocol to inform their diagnoses. This included semi-structured interviews, behavior rating scales, academic testing, a physical exam including vision and hearing screens, developmental history, and any further testing deemed necessary by the clinician.  Using these data, the clinicians determined ADHD was an appropriate primary diagnosis for each participant.

3. A separate team collected EEG data on participants to determine whether each participant showed the EEG marker for ADHD, i.e., an elevated theta/beta ratio.  EEG data was collected by individuals who were fully blinded to the clinical data and the  clinician's diagnosis.

4. The clinical data was reviewed by a multidisciplinary team that included a child and adolescent psychiatrist, a clinical psychologist, and a developmental pediatrician.  This team determined a consensus best estimate diagnosis, i.e., whether or not ADHD was a definite or likely primary diagnosis for each participant.

Because multiple specialists reviewed the data on each participant until a consensus was reached, this was considered the 'gold standard', i.e., the most accurate diagnosis for the child.

With these data in hand, the investigators then asked 2 basic questions.  First, how well do the diagnoses made by individual clinicians match the 'best estimate diagnosis' of the multidisciplinary team?  And, does integrating QEEG findings with clinicians' diagnosis significantly improve the match rate?  An affirmative answer to this question would support the value of collecting QEEG data when evaluating children and adolescents for ADHD.


Results

Results were examined separately for children (ages 6 to 11.99 years; n=201) and adolescents (12 to 17.99 years; n=74).  Among the children, 102 (51%) were judged to have ADHD by the expert team and 99 were judged to have another condition. Thus, the team ruled out ADHD as the primary explanation of the child's difficulties about half the time. 

Of the 102 children placed in the ADHD group by the team, clinicians diagnosed 93 with ADHD, an agreement rate of about 90%  When QEEG findings were integrated with the clinician diagnosis, the agreement rate declined to 80% because 19 of the 102 children did not show the QEEG marker for ADHD.  These 19 cases would be considered 'false negatives', i.e., the team assigns an ADHD diagnosis but the test misses it.

Of the 99 children for whom ADHD was ruled out by the team, 68 were diagnosed with ADHD by individual clinicians.  Because the team's diagnosis is considered the 'gold standard', this reflects a large percentage of children who would be considered incorrectly diagnosed. 

Among these 68 children incorrectly diagnosed by individual clinicians, QEEG results were inconsistent with ADHD in 65 cases.  Because a negative QEEG result signals the need to consider conditions other than ADHD as the reason for a child's presenting problems, these data suggest that it's use could dramatically reduce the number of children inappropriately diagnosed.


Findings for adolescents

Similar results were obtained for adolescents. Of the 74 12-18 year-old's in the study, only 28 were assigned a primary diagnosis of ADHD by the team (38%). Individual clinicians agreed on 24 of the 28 (86%) and integrating QEEG findings did not change this.

Among the 46 adolescents where ADHD was ruled out as a primary diagnosis by the team, clinicians diagnosed ADHD for 25; QEEG results were inconsistent with ADHD in 20 of these cases.  As was found for children, therefore, combining QEEG with clinician diagnoses could significantly reduce the number of adolescents who are inappropriately diagnosed with ADHD.


Overall accuracy findings

The researchers also computed an overall accuracy score for clinicians' diagnosis vs. team diagnosis both with and without the QEEG data.  Without incorporating QEEG findings, the overall match rate between clinicians and the expert team was 61%.  Thus, individual clinicians disagreed with the consensus diagnosis nearly 40% of the time. 

When QEEG results were included, the overall accuracy rate improved to 88%, indicating that integrating QEEG findings with clinician assessment would enable clinicians to more consistently match the assessment of a multidisciplinary expert team.


Summary and Implications

It would be ideal if ADHD evaluations for children and adolescents routinely involved the collection of comprehensive clinical data that was carefully reviewed by an interdisciplinary expert team.  In reality, however, time and resource limitations preclude this and many diagnostic decisions are made based on limited and insufficient information.

Results from this study indicate that in actual clinical practice, clinicians' assessment of whether ADHD is an appropriate diagnosis for children/teens presenting with attention and behavior problems will frequently diverge from what an expert team would conclude; in this sample, it occurred about 40% of the time, generally because clinicians 'saw' ADHD where the expert team did not.  This could result in many children and adolescents being inappropriately placed on medication that is unlikely to be helpful while more likely 'causes' for their difficulties go unaddressed.

The findings also indicate that routinely incorporating QEEG into diagnostic evaluations - as noted above, it is NOT intended to be used as a stand alone diagnostic test - could significantly reduce the rate of misdiagnosis and improve the care that children receive.  Here is how this might occur:

1. If the clinician believes his/her findings findings are consistent with ADHD, and the QEEG results support this, the diagnosis of ADHD is affirmed.  Clinicians could have greater confidence in their diagnosis and many parents would be reassured about the inclusion of an objective procedure in their child's evaluation.  Presumably, parents with greater confidence in the diagnosis would be more willing to pursue treatment recommendations that are offered.

2. If the clinician believes his/her findings are consistent with ADHD, but the QEEG results do not support this, it would encourage the clinician to carefully consider alternative explanations for the child's presenting problems; this could include the collection of additional data.  In this circumstance, some primary care physicians might choose to refer the child to a child mental health specialist.

As summarized above, the QEEG results came back negative in the vast majority of cases where clinicians diagnosed ADHD and the expert team did not, i.e., 65 of 68.  Thus, simply refraining from diagnosing ADHD without further assessment when QEEG results are negative could prevent many children and adolescents from being incorrectly diagnosed.  Note that a negative QEEG result would not eliminate the possibility of ADHD being an appropriate diagnosis, but only highlight the need to proceed cautiously.

3. If the clinician believes his/her findings findings are inconsistent with ADHD, but the QEEG results are positive, this by itself would not lead to the diagnosis being made.  This is because ADHD should only be diagnosed based on DSM criteria. However, it could lead clinicians to carefully review clinical findings before fully ruling out the diagnosis.


Study limitations

Although the information available on the FDA site suggests that this was a strongly designed study, there are limitations that are important to be aware of.

First, while not really a limitation, the study assumes that the team's consensus diagnosis is the 'correct' one', and that when a clinician's diagnosis did not match this, the clinician was in error.  The integrity of the study hinges on the validity of this assumption.  While it strikes me as reasonable, some might disagree given that clinicians interacted directly with patients while the team did not. 

Second, although clinicians from 13 sites participated in the study, this is not a nationally representative sample of professionals who evaluate children and adolescents for ADHD.  Thus, the extent to which the findings obtained here would be similar for different groups of clinicians cannot be known with certainty.  Assuming that the team's diagnosis is the 'best estimate', the ability of clinicians to match this would depend on their training and experience; information on this important factor was not provided in the FDA information, however.


Other concerns about the routine use of QEEG in ADHD evaluations

One criticism of using this QEEG diagnostic aide in ADHD evaluations is that it is unnecessary because the diagnosis should be made based on behavioral criteria and developmental history.  This is true and the FDA press release carefully specifies that this brain wave test should only be used in conjunction with a clinician's evaluation.  Thus, this does not strike me as a meaningful criticism since when used in this way the clinician's diagnostic decisions are likely to better match those that would be made by an expert team. 

Another concern is that the procedure will increase the costs associated with ADHD evaluations.  The strength of this concern depends on the cost of the procedure and I am unable to speak to that. However, any increase in costs would need to be considered in conjunction with the benefits of more accurate diagnostic evaluations that could accrue if the test were used properly. 

If fewer children and teens are inappropriately diagnosed and treated for ADHD, important benefits to these individuals and their families should result.  And, if more parents pursue treatment recommendations because the QEEG procedure provides greater confidence in their child's diagnosis, this would be an important benefit as well.

The value of the QEEG test will also depend on the training that clinicians and their staff receive in collecting the required data. Proper training is an issue in the use of any medical device or procedure and one hopes that adequate training protocols will be carefully implemented.

Two other concerns should be mentioned.  First, the QEEG test results were inconsistent with the team's diagnosis about 12% of the time.  This was usually when the team believed ADHD was an appropriate diagnosis and the QEEG results were negative.  In fact, this was more likely to occur when QEEG results were integrated with the clinicians' diagnosis than when diagnostic decisions were based on clincians' evaluation alone. This could result in cases where ADHD is inappropriately ruled out or a diagnosis is delayed pending additional assessment.  At the population level, I believe this would be more than offset by the reduction in children incorrectly diagnosed.   However, some individuals could be 'harmed' when QEEG introduces 'error' into their evaluation.

Finally, it should be noted that some recently published research on the utility of QEEG for ADHD evaluations presented results that conflict strongly with those obtained in the current study.  While not invalidating the findings submitted to and reviewed by FDA, these inconsistent reports are perplexing.


Overall Summary

While not minimizing the issues discussed above, FDA approval of an objective biologically-based test to aide in the assessment of ADHD represents a significant advance for the field.  If widely adopted by clinicians who are well trained in it's proper use, the potential is there to significantly enhance the care that children and teens presenting for ADHD evaluations receive.  This could occur in several important ways.

First, when clinical evaluation and QEEG results are consistent with ADHD, incorporating this objective procedure may provide many parents with greater confidence in their child's diagnosis.  Research suggests that this will result in many becoming more accepting of the diagnosis and more willing to pursue appropriate treatment, including medical treatment (see http://www.helpforadd.com/2006/january.htm ).

Second, because negative QEEG results would signal a need for caution and additional assessment before making an ADHD diagnosis, routine use of QEEG could significantly reduce the number of children/teens incorrectly diagnosed with ADHD and inappropriately treated for it. A more careful review of other factors contributing to the child's presenting problems could also contribute to more appropriate diagnoses and treatment recommendations for many children.

It will be very interesting to see what unfolds with the use of QEEG over the next several years and the extent to which the apparently promising aspects of this advance are realized.

Note - You can view the full report posted on the FDA site atyou can view the full report at http://www.accessdata.fda.gov/cdrh_docs/reviews/K112711.pdf

(c) 2013 David Rabiner, Ph.D.

Information presented in Attention Research Update is for informational purposes only, and is not a substitute for professional medical advice.  Although newsletter sponsors offer products and services that I believe will be of interest to subscribers, sponsorship of Attention Research Update does not constitute a specific endorsement or guarantee of any company's product or services.