Threatening Impairment: Assessing the Effects of Diagnosis Threat on Concussion Screening Tests

Abstract

Expectation has been found to be a major etiological factor in Post-concussion Syndrome (PCS), but research has mostly relied on selfreport. This study examines the role of diagnosis threat on actual cognitive performance on concussion screening tests. Negative performance expectations were induced in some participants by presenting research showing cognitive impairment in contact sport athletes. Other participants read neutral research findings showing no impairment in athletes. Overall, athletes in the “negative expectation” condition performed slower but more accurately than those in the “neutral expectation” condition. Negative expectations might have caused participants to hesitate before responding, allowing them to double-check their responses. However, due to a small, homogenous sample, these promising but preliminary findings should be further explored with further research.

Keywords: post-concussion syndrome, diagnosis threat, concussion screening, cognitive impairment, assessment

1. Introduction

Post-concussion Syndrome (PCS) refers to a cluster of self-reported symptoms occurring after a mild traumatic braininjury (mTBI). Symptoms range from emotional, somatic, to cognitive in nature and include headache, dizziness, nausea, blurred or double vision, fatigue, irritability, anxiety, impaired processing speed, slowed reaction time, concentration difficulty, and memory deficits (Panayiotou, Crowe, & Jackson, 2011). However, the etiology of PCS has become increasingly controversial. While it would seem that these symptoms are related to neurological damage as a result of physical injury to the brain, considerable research has shown that the symptoms of PCS have a psychological component that exaggerates the effects of the organic injury (Mittenberg et al., 1992). When Mittenberg et al. (1992) asked healthy participants to imagine the symptoms they would expect to experience following head trauma, their imagined postconcussion symptoms were almost identical to the symptoms reported by actual head trauma patients. Further, patients with head injuries underestimated the prevalence of PCS symptoms prior to their injury (Mittenberg et al., 1992). In a subsequent study by Ferguson et al. (1999), the researchers similarly found that head-injured athletes reported symptom rates that did not significantly differ from those reported by uninjured athletes. This study further replicated the results of Mittenberg et al., finding that head-injured subjects underestimated the prevalence of PCS symptoms prior to the injury. Taken together, these studies suggest that expectation can cause the healthy individuals to resemble head-injured patients in their endorsement of PCS symptomology. Further, head-injured patients may attribute unrelated and pre- existing symptoms to their head injury, showing that expectations play a major role in the subjective experience and attribution of PCS symptoms.

To date, research on the role of expectancy in PCS has mostly relied on participants’ self-report of their experience of symptoms. Yet, cognitive tests are widely used in the diagnosis and management of concussions and other head injuries in conjunction with medical examination and self-report of symptoms. Therefore, examining the role of expectancy on actual performance is important to validate these cognitive tests. Considerable research has been conducted on the effect of subjective expectancy on cognitive performance. Meece, Wingfiled, and Eccles (1990) found a significant relationship between performance expectancies and subsequent grades on mathematics tests. Students with a negative perception of their math abilities had lower expectations for their performance and, accordingly, performed worse on tests, even when the researchers controlled for actual ability. Reinhard and Dickhäuser (2009) examined how performance expectancies affect subsequent performance by manipulating subjects’ perception of cognitive task difficulty. They found that the perception of high task difficulty led to lower performance expectations, which in turn led to poorer performance.

It would follow that expectations of cognitive impairment might negatively affect performance on post-concussion diagnostic tests of cognitive performance, a phenomenon Suhr and Gunstad termed “diagnosis threat” (2002). Ozen and Fernandes (2011) tested diagnosis threat by manipulating the expectations of individuals with and without a history of mild head injury. Among participants who had recently experienced a mild head injury, those in the “diagnosis threat” condition were told they were being tested for cognitive impairment as a result of head injury, while those in the “neutral” condition were only told their cognitive functioning was being tested. Participants in the “diagnosis threat” condition selfreported more cognitive problems but did not score markedly differently on neuropsychological measures of cognitive function compared to participants in the “neutral condition,” leading the researchers to conclude that “diagnosis threat” might not affect actual cognitive performance.

Suhr and Gunstad (2002) selected participants who reported an extensive history of head injury. Those in the “diagnosis threat” condition read statements calling attention to possible cognitive effects of their previous head injuries and showed poorer performance on some cognitive measures compared with those who read more neutral statements. Suhr and Gunstad (2005) replicated these results in a follow-up study, finding that participants whose attention had been drawn to their prior head injuries performed significantly worse on tests of attention, working memory, psychomotor speed, and memory compared to those who read neutral statements. In these studies, the researchers concluded that diagnosis threat caused poorer cognitive performance.

This study was designed to extend the examination of diagnosis threat to an increasingly used set of computerized cognitivetests: the Axon Sports Computerized Cognitive Assessment Tool (CCAT). By investigating whether performance on these tests is affected by expectations of impaired cognitive performance associated with participation in a collision or contact sport, this study will add to the rather sparse literature on diagnosis threat and cognitive performance. Moreover, it will do so for a set of tests that are increasingly used in concussion screening and management, with important practical implications for the diagnosticity and utility of those tests. We sought to influence athletes’ expectations and induce diagnosis threat by drawing their attention to the research showing cognitive impairment in collision or contact sport athletes. Based on the original and replicated findings of Suhr and Bunstad (2002; 2005), we expected athletes in this “negative expectation” condition to perform worse on the CCAT in both speed and accuracy compared to athletes in the “neutral expectation” condition, who were exposed to research findings revealing no cognitive impairment in contact or collision sport athletes.

Method

Participants

The participants were National Collegiate Athletic Association (NCAA) Division III varsity athletes at a small liberal arts university in the American South. Eleven played soccer (8 female, 3 male), and 15 played field hockey (all female). Data from three participants were excluded due to failure to complete a CCAT practice test. The remaining sample consisted of 7 female soccer players, 2 male soccer players, and 14 female field hockey players. The remaining sample consisted of 4 freshmen, 7 sophomores, 6 juniors, and 6 seniors; 22 were white, and one was African-American. Participants’ years of participation in their respective sports ranged from 5 to 18 years (M=11, SD=3.814). No participants reported that they were currently experiencing a concussion. There was no considerable demographic difference between the two conditions.

Measures

All participants completed Axon Sports’ CCAT, which consists of four tasks using computerized French playing cards. The first is the Detection Task, a test of processing speed, in which participants are asked to indicate whether or not each individually flipped card has turned over. Second, the Identification Task, a test of visual attention, asks participants to indicate whether or not each individually flipped card is red. Third, the One Card Learning Task, a test of visual learning and memory, requires participants to indicate whether or not they have seen each individually flipped card before in the task. Fourth, the One Back Task, a test of working memory and attention, requires

participants to indicate whether or not each individually flipped card is the same as the card that appeared before it. For each task, participants are instructed to go as fast as possible without making any mistakes. Participants first completed a practice test that consisted of shortened versions of each task and then completed the full version of the CCAT. The CCAT scores both the accuracy and speed of participants’ responses (“Testing,” 2012).

All participants completed a version of the Positive and Negative Affect Schedule (PANAS) as a mood inventory (Watson, Clark, & Tellegen, 1988). This 20-item self-report consists of emotions classified as either positive or negative affect. On a 5-point likert scale, participants rate the extent to which they either experience or have experienced each emotion. The directions of the scale can be modified to assess mood in the present, in general, or during a particular time period. In this case, participants were instructed to retroactively rate the extent to which they experienced each emotion immediately after reading the article excerpt used as the manipulation.

All participants completed the short form of the Spielberger State-Trait Anxiety Inventory (STAI-6) as an anxiety inventory (Marteau & Bekker, 1992). This six-item questionnaire requires participants to rate the degree to which they feel calm, tense, upset, relaxed, content, and worried on a 4-point likert scale. The STAI-6 was modified slightly for the purpose of the study, and participants were asked to retroactively rate the extent to which they felt each emotion immediately after reading the article excerpt used as the manipulation.

Procedure

The study was conducted in a computer laboratory located in the on-campus library. Each of the three teams (women’s soccer, men’s soccer, women’s field hockey) was tested separately, with one 40-minute testing session devoted to each team. All teams were tested on a weeknight within a week after the completion of their season.

Before each session, a researcher laid instruction packets in front of each individual computer terminal. Half of the packets contained the “negative expectation” manipulation, and half contained the “neutral expectation” manipulation. While distributing the packets at each terminal, the researcher alternated between each condition. As the participants entered the computer lab, they were asked to fill in every seat, starting with the front row. This way, the researcher obtained quasi-randomization of conditions and almost the same number of participants in each condition without alerting participants to the existence of two different conditions.

Participants were thanked for their participation and asked to read over and sign an informed consent form if they wished to participate. After all informed consent forms were signed and collected, the researcher instructed them flip past the blank cover sheet of their instruction packet and to carefully follow all written instructions. After this point, the researcher provided no further verbal instruction and allowed the participants to complete the remainder of the testing by following their written instruction packets.

In each packet, the participants were first asked to read an excerpt from a news article reporting on a recent research study. The excerpt was introduced to all participants with the following instructions: “As an athlete, we thought you might be interested in the following information from a recent study by researchers at Dartmouth Medical School on cognitive performance in collegiate athletes. The study itself was published in the journal. The following excerpt comes from MedPage Today, a news service for physicians.” However, the subsequent excerpts and instructions participants in each condition read were different. Both excerpts were taken from the same article (Neale, 2012), but each excerpt was selectively abridged by the researchers to induce either negative or neutral expectations in the participants.

Participants in the “negative expectation” condition read the following excerpt and instructions:

“Most head impacts during sports do not result in concussion or mild traumatic brain injury, but concerns have been raised about the cumulative effects of repetitive impacts....McAllister and colleagues evaluated the cognitive effects of repetitive head impacts over a single season...at three National Collegiate Athletic Association (NCAA) programs....On one of the tests included in the neuropsychological battery, the contact athletes were more likely to fall short of the expected performance after the season. In addition, poor performance on two of the postseason cognitive measures...was associated with greater exposure to various head impact metrics.” The cognitive tests you are about to take are similar to those used in the aforementioned study that found poorer post-season performance on some cognitive tasks.

Participants in the “neutral expectation” condition read the following excerpt and instructions:

“Most head impacts during sports do not result in concussion or mild traumatic brain injury....McAllister and colleagues evaluated the cognitive effects of repetitive head impacts over a single season...at three Division I National Collegiate Athletic Association (NCAA) programs....There were few cognitive differences between the contact and noncontact athletes in the preseason and postseason assessments, ‘suggesting that accumulated impacts over multiple previous seasons (i.e., prior to the index season) are not associated with reduced cognitive performance...,’ according to the authors.” The cognitive tests you are about to take are similar to those used in the aforementioned study that overall found no reductions in cognitive performance over multiple seasons.

After the excerpts, the instruction packet directed all participants to follow a URL that led to the CCAT on the Axon Sport website. Participants put on headphones provided for them, completed a practice test of the CCAT, and then completed the full version of the CCAT. After completing the CCAT, participants entered and followed a URL to an electronic post-test questionnaire powered by Survey Monkey. The post-test questionnaire included a manipulation check assessing whether participants understood the excerpt and the degree to which they perceived the excerpt to be personally relevant and true. The manipulation check also asked participants to retroactively rate how well they expected to perform on the cognitive tests and how likely they believed it to be that they were experiencing cognitive impairment as a result of their participation in a contact or collision sport. The post-test questionnaire included the PANAS, STAI-6, and questions regarding participants’ alcohol consumption, caffeine consumption, and amount of sleep in the preceding 24 hours.

Further questions also asked about depression and anxiety over the past month, history of concussion and head injury, diagnosis of any psychological or intellectual disorders, and any psychoactive medications they were taking at the time. The post-test also contained a contamination check asking if participants had received any unauthorized information about the study before participating. Finally, it asked for basic demographic information and the number of years athletes had been playing their respective sports.

Upon their completion of the post-test, participants were asked to sit quietly until all other participants were finished.

After all participants had completed the post-test, they were verbally debriefed by the researcher, thanked for their participation, and dismissed.

Results

Manipulation Checks

Ten of the 11 participants in the “neutral expectation” condition and 11 of the 12 participants in the “negative expectation condition” correctly answered the question assessing their comprehension excerpt of the manipulation. A multivariate ANOVA test was run with condition as a fixed factor and responses to the manipulation check items assessing perceived personal relevance and truth of the manipulation excerpt, performance expectations, and perceived likelihood cognitive impairment as dependent variables. A statistically significant relationship was found between condition and ratings of performance expectations at an alpha level of<.05. Participants in the “neutral expectation” condition rated their expected performance significantly higher than participants in the “negative expectation” condition (F=4.549,=.048). A summary of manipulation check responses and the ANOVA tests are presented in Table 1.

Table 1 - Responses to manipulation check items and condition
See Full Size >

Condition and CCAT Task Speed

For each task, each participant’s mean response time underwent a log10 transformation when it was recorded in the Axon Sports system to obtain a speed score appropriate for statistical analysis. (Note: a higher log10 transformed speed score indicates a slower response time). Each participant’s scores can be back-transformed to obtain approximate mean response time in milliseconds. A multivariate ANOVA test was performed with the transformed speed scores for each task as dependent variables and the condition, whether or not alcohol had been consumed in the preceding 24 hours, and the number of caffeinated beverages consumed in the preceding 24 hours as a fixed factors. For every task, participants in the “negative expectation” condition responded slower than participants in the “neutral expectation” condition, and this relationship was statistically significant at an alpha level of<.05 for the Identification task (F=5.555,=0.40), One Card Learning task (F=8.040,=.018), and One Back task (F=6.368,=.030). A summary of the transformed speed scores and the ANOVA test are presented in Table 2. The relationship between condition and approximate mean response time is depicted graphically in Figure 1.

Table 2 - Transformed speed scores for each CCAT task and condition
See Full Size >
Figure 1: Approximate mean response time (ms) for each CCAT task and condition
Approximate mean response time (ms) for each CCAT task and condition
See Full Size >

Condition and CCAT Task Accuracy

For each task, each participant’s proportion of correct to incorrect responses underwent an arcsine transformation when it was recorded in the Axon Sports system to obtain an accuracy score appropriate for statistical analysis. (Note: a higher transformed accuracy score indicates better accuracy). The score can be back-transformed to obtain each participant’s percentage of responses correct for each task. A multivariate ANOVA test was performed with the transformed accuracy

scores for each task as dependent variables and the condition, whether or not alcohol had been consumed in the preceding 24 hours, and the number of caffeinated beverages consumed in the preceding 24 hours as a fixed factors. In every task, participants in the “negative expectation” condition responded more accurately than participants in the “neutral expectation” condition, and this relationship was statistically significant at an alpha level of<.05 for the Detection task (F=5.974,=0.035) and One Back task (F=6.561,=.028). A summary of the transformedaccuracy scores and the ANOVA test are presented in Table 3. The relationship between condition and mean percentage of responses correct is depicted graphically in Figure 2.

Table 3 - Transformed accuracy scores for each CCAT task and condition
See Full Size >
Figure 2: Mean percentage of responses correct for each CCAT task and condition
Mean percentage of responses correct for each CCAT task and condition
See Full Size >

Correlation between Task Speed and Accuracy

Pearson Correlation Coefficients were calculated between participants’ transformed speed and accuracy scores for each CCAT task. For every task, transformed speed and accuracy scores were positively correlated. In other words, response time and accuracy were positively correlated. However, these correlations were not statistically significant at an alpha level of<.05. The correlation approached significance for the Detection task (r=.402,=.057). The correlations are summarized in Table 6.

Table 4 - Correlations between participants’ transformed speed and accuracy scores for each CCAT task
See Full Size >

Clinical Significance

No participant in either condition displayed a pattern of responses indicative of a concussion or trauma-

induced cognitive impairment. All participants performed in the normative range.

Discussion

This study yielded several interesting findings, especially in light of its mild manipulation and small sample size. As predicted, the participants in the “negative expectation” condition performed with slower response time compared to participants in the neutral “expectation condition” on every task. Interestingly, contrary to the researchers’ hypothesis, participants in the “negative expectation” condition performed more accurately than participants in the “neutral expectation” condition on every task.

The slower speed and greater accuracy of the “negative expectation” condition might be explained by response hesitancy resulting from diagnosis threat. Because they expected to perform worse and were faced with the possibility of cognitive impairment, these participants might have hesitated and second-guessed their responses on the CCAT tasks.

However, taking more time allowed them to verify their responses, leading to superior accuracy compared to participants with neutral expectations who responded without hesitation. In line with this explanation, response time was positively correlated with accuracy on every task; these correlations might have been significant with a larger sample size.

The disparity in significance between the speed and accuracy of some tasks raise several interesting discussions. Across conditions, there was no significant difference between accuracy scores on the Identification and One Back tasks, which might reflect a ceiling effect. For both these tasks, on average, participants answered over 97% of items correctly; the ease of these tasks might have made them less sensitive to subtle between-group differences. In contrast, a significant between-condition difference was found on the One Card Learning task; on average participants answered

approximately 80% of items correctly. This task is marked more difficult and less susceptible to ceiling effect.

The significant between-condition difference of Detection task accuracy seems inconsistent with the ceiling effect explanation. The Detection task, in which participants averaged over 99% accuracy, is the easiest task on the CCAT. It asks participants to indicate whether or not a card has been flipped over. However, the nature of the task aligns the between-condition accuracy difference with the response hesitation hypothesis discussed above. From personal experience taking the CCAT, the researchers observe that the most common error on the Detection task occurs by responding prematurely, before the card has been flipped. If facing diagnosis threat indeed caused participants to hesitate with their response, significantly higher accuracy would be expected because they would be less likely to commit this common error of premature response. Also in line with this explanation, the strongest correlation was found between response time and accuracy for the Detection task, and it very nearly reached significance. It seems that, more so than in any other task, hesitating before responding gave participants greater accuracy in the Detection task.

According to the post-test questionnaire, all but two participants correctly comprehended the manipulation excerpts.

The excerpts from each condition were equivalent in believability and perceived personal relevance. However, participants in the “negative expectation” condition reported significantly lower performance expectations compared to participants in the “neutral expectation” condition, indicating that the manipulation affected performance expectations in the proper direction. Even so, on average participants in the “negative expectation” condition still expected their performance to be above average. Further, there was no significant between-condition difference in the likelihood participants believed they might be experiencing cognitive impairment. On average, participants in both conditions believed it was at least “somewhat unlikely” that they were experiencing impairment.

Overall, it seems that the “negative expectation” condition did not actually induce strong negative expectations in participants according to their retrospective self-report. However, as previously discussed, the manipulation had many significant effects on actual performance. It is possible the negative expectations operate at a subconscious level, causing the participants to hesitate even if they do not consciously expect cognitive impairment. Further, the manipulation check questions were retrospective, occurring after participants completed the CCAT. Because they were retroactively asked to report their mental state directly after reading the excerpt, participants’ memories might have been inaccurate and altered by exposure to the test. Indeed, the CCAT consists of fairly simple tasks that should seem “easy” to any college student; the act of taking the CCAT might have inflated the participants’ retrospective ratings of performance expectation. The manipulation was also indirect in its inducement of diagnosis threat by presenting research and allowing participants to assess its implacability to their own situation. Participants in the “negative expectation” condition could have easily discounted the negative consequences of contact or collision sport participation as not personally relevant or not true as a self-protecting mechanism against anxiety.

This study has a few weaknesses that should be addressed. For ethical reasons, we could not directly lead participants to believe they were experiencing a concussion and cognitive impairment. Rather, the manipulation was indirect and perhaps weaker as a result. Future research might use settings in which athletes’ expectations of impairment arise from natural circumstances and do not need to be experimentally manipulated. Because the study was only interested in contact or collision sport athletes that had recently ended their season at the time of data collection, the population of interest was quite narrow, and the sample size was small as a result. The sample was also fairly homogenous, composed almost exclusively of Caucasian females, and the findings may not be applicable to more diverse populations. Future research

should also extend to Division I collegiate athletes and even professional athletes, whose athletic participation is more long-standing and rigorous compared to the participants in this study.

In spite of its limitations, this study found several interesting results and has important clinical implications. The CCAT and similar tools are used every day in the diagnosis and management of concussions, and the act of taking a concussion screening test following head trauma might induce expectations of cognitive impairment. This study shows that expectations alone can affect performance, even in healthy participants. It also shows that response speed and accuracy scores must be analysed separately, and impairment in one might not be accompanied by impairment in the other. Therefore, subjective expectancy and diagnosis threat are essential considerations in the administration of these tests and the interpretation of their results. This is a promising pilot study in an area in which the research has been relatively sparse and opens the possibility for extensive future investigation.

References

  • Berman, M. G., Jonides, J., & Kaphlan, S. (2008). The cognitive benefits of interacting with nature. Psychological science, 19(12), 1207-1212.

  • Ferguson, R. J., Mittenberg, W., Barone, D. F., & Schneider, B. (1999). Postconcussion syndrome following sports-related injury: Expectation as etiology. Neuropsychology, 13(4), 582-589.

  • Geh, C. L. M., Beauchamp, M. R., Crocker, P. R. E., & Carpenter, M. G. (2011). Assessed and distressed: White-coat effects on clinical balance performance. Journal of psychosomatic research, 70, 45-51.

  • Meece, J. L., Wigfield, A., & Eccles, J. (1990). Predictors of math anxiety and its influence on young adolescents' course enrollment intentions and performance in mathematics. Journal of educational psychology, 82(1), 60-70.

  • Mittenberg, W., DiGiulio, D. V., Perrin, S., & Bass, A. E. (1992). Symptoms following minor head injury: Expectation as aetiology. Journal of neurology, neurosurgery, and psychiatry, 41, 611-616.

  • Neale, T. (2012, May 16). Season of ‘head hits’ may dull athletes’ thinking. MedPage Today. Retrieved from http:www.medpagetoday.com/Neurology/HeadTrauma/32726

  • Ozen, L. J., & Fernandes, M. A. (2011). Effects of 'diagnosis threat' on cognitive and affective functioning long after mild head injury. Journal Of The International Neuropsychological Society, 17(2), 219-229.

  • Panayiotou, A., Crowe, S., & Jackson, M. (2011). An analogue study of the psychological and psychosocial processes associated with post-concussion symptoms. Australian Psychologist, 46(4).

  • Reinhard, M., & Dickhäuser, O. (2009). Need for cognition, task difficulty ,and the formation of performance expectancies. Journal of personality and social psychology, 96(5), 1062-1076.

  • Suhr, J.A., & Gunstad, J. (2002). “Diagnosis Threat”: The effect of negative expectations on cognitive performance in head injury. Journal of Clinical and Experimental Neuropsychology, 24, 448-457.

  • Suhr, J.A., & Gunstad, J. (2005). Further exploration of the effect of “diagnosis threat” on cognitive performance in individuals with mild head injury. Journal of the International Neuropsychological Society, 11, 23-29.

  • Testing. (2012). Axon Sports. Retrieved from http://www.axonsports.com/index.cfm?pid=3&pageTitle=Testing

Copyright information

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

About this article

Publication Date

14 May 2014

eBook ISBN

978-1-80296-000-6

Publisher

Future Academy

Volume

1

Print ISBN (optional)

-

Edition Number

1st Edition

Pages

1-236

Subjects

Social psychology, collective psychology, cognitive psychology, psychotherapy

Cite this article as:

Hagler, M., & Yu, K. (2014). Threatening Impairment: Assessing the Effects of Diagnosis Threat on Concussion Screening Tests. In Z. Bekirogullari, & M. Y. Minas (Eds.), Cognitive - Social, and Behavioural Sciences – icCSBs 2014, vol 1. European Proceedings of Social and Behavioural Sciences (pp. 111-122). Future Academy. https://doi.org/10.15405/epsbs.2014.05.14