A clinical decision support system for the diagnosis of probable migraine and probable tension-type headache based on case-based reasoning
- Ziming Yin†1,
- Zhao Dong†2,
- Xudong Lu1Email author,
- Shengyuan Yu2Email author,
- Xiaoyan Chen2 and
- Huilong Duan1
© Yin et al.; licensee Springer. 2015
Received: 12 January 2015
Accepted: 15 March 2015
Published: 1 April 2015
The overlap between probable migraine (PM) and probable tension-type headache (PTTH) often confuses physicians in clinical practice. Although clinical decision support systems (CDSSs) have been proven to be helpful in the diagnosis of primary headaches, the existing guideline-based headache disorder CDSSs do not perform adequately due to this overlapping issue. Thus, in this study, a CDSS based on case-based reasoning (CBR) was developed in order to solve this problem.
First, a case library consisting of 676 clinical cases, 56.95% of which had been diagnosed with PM and 43.05% of which had been diagnosed with PTTH, was constructed, screened by a three-member panel, and weighted by engineers. Next, the resulting case library was used to diagnose current cases based on their similarities to the previous cases. The test dataset was composed of an additional 222 historical cases, 76.1% of which had been diagnosed with PM and 23.9% of which had been diagnosed with PTTH. The cases that comprised the case library as well as the test dataset were actual clinical cases obtained from the International Headache Center in Chinese PLA General Hospital.
The results indicated that the PM and PTTH recall rates were equal to 97.02% and 77.78%, which were 34.31% and 16.91% higher than that of the guideline-based CDSS, respectively. Furthermore, the PM and PTTH precision rates were equal to 93.14% and 89.36%, which were7.09% and 15.68% higher than that of the guideline-based CDSS, respectively. Comparing CBR CDSS and guideline-based CDSS, the p-value of PM diagnoses was equal to 0.019, while that of PTTH diagnoses was equal to 0.002, which indicated that there was a significant difference between the two approaches.
The experimental results indicated that the CBR CDSS developed in this study diagnosed PM and PTTH with a high degree of accuracy and performed better than the guideline-based CDSS. This system could be used as a diagnostic tool to assist general practitioners in distinguishing PM from PTTH.
The International Classification of Headache Disorders (ICHD) published by the International Headache Society (IHS) has been proven to be effective and has been widely applied to clinical practice worldwide [1,2]. However, practitioners are often confronted with patients whose symptoms meet some but not all diagnostic criteria. In some studies, researchers have found that some specific types of headaches share multiple similarities [3-6]. For example, many migraine attacks are accompanied by tension headache-like symptoms, such as neck pain. In addition, tension-type headaches are often accompanied by migraine headache-like symptoms, such as photophobia, phonophobia, and aggravation by routine physical activity. Among these overlaps, the overlap between probable migraine (PM) and probable tension-type headache (PTTH) is most common due to their high individual incidence rates [7-10]. Since PM and PTTH have different preventative therapies, the differential diagnosis of these two types of headaches is necessary.
The development of clinical decision support systems (CDSSs) for the diagnosis of primary headaches has long been a major research topic. Most existing headache CDSSs are based on ICHD criteria [11-14]. However, research has shown that none of these CDSSs are capable of differentiating among primary headaches with overlapping features. In our previous studies [15,16], the guideline-based CDSS did not perform adequately when faced with PM and PTTH. Thus, in this study, a feasible computer-aided diagnosis method for differentiating between these two types of headaches was proposed.
In clinical practice, headache experts diagnose these headaches based on their clinical experience by recalling key indicators and previously solved cases. In an attempt to emulate this expertise and reasoning, a CDSS based on case matching and recommendations, or case-based reasoning (CBR), was developed. CBR, an artificial intelligence technique, is the process of solving new problems based on the solutions of similar, previously solved problems, and is considered to be one of the most effective methods of managing implicit knowledge, such as intuition and experiences. In CBR, in order to solve a new problem, cases with features that are most similar to the new problem are retrieved from a case library, and their solutions are considered for reuse. This method is very suitable for the diagnosis of PM and PTTH. CBR has been applied to many other medical areas as well. For example, Koton  developed a case-based system named CASEY for the diagnosis of heart complications, Guessoum et al.  presented a decision-based support system for the diagnosis of chronic obstructive pulmonary disease, and Sharaf-El-Deen et al.  proposed a hybrid case-based reasoning approach for the diagnosis of breast cancer and thyroid diseases. All of these systems have been successful in local settings. In this study, a CBR CDSS was developed in order to assist general practitioners in differentiating between PM and PTTH.
The protocol used in this study was approved by the Chinese PLA General Hospital ethics committee in Beijing, China.
Clinical data acquisition
The data sheet using in the clinical interview
Date of headache onset
Duration of pain episodes
Attack with fixed period
The number of attacks
Aggravation by or causing avoidance of routine physical activity
Persistent headache, daily from its onset
Years of smoking
Years of drinking
How many cups of tea per day
How many cups of coffee per day
Family medical history
Location of pain*
Stay in dark room
The options of some items in Table 1
Location of pain
Tempus | Crown | Forehead | Pars orbitalis | Face | Ear | Occiput | Neck
Sleeplessness | Mood swings | Food | Activities | Weather change | Menstruation | Hard light | Smell | Noise
Lay Down | Stay in Dark Room | Massage | Hot Compress | Cold Compress | Fast Walk | Exercise | Stand | Pregnancy
Nausea | Vomit | Photophoby | Phonophobia
Visual | Sensory | Speech and/or language | Motor | Brainstem | Retinal
Loquacity | Depression | Irritability | Dysesthesia | Stiff neck | Thirstiness | Yawn | Drowsiness | Fidget | Poor appetite | Photophobia | Phonophobia | Constipation | Attention Disorder | Diarrhea | Diuresis | Dysphasia | Feel weak | Feel dizzy | Feel cold
Case library construction
A three-member panel preprocessed the cases used in the headache database. Only cases with a PM or PTTH diagnosis were retained; any cases that contained incomplete or inaccurate information were excluded. Furthermore, the remaining cases were processed in order to ensure that the diagnosis was correct. Next, engineers calculated the weight of each feature, or symptom, in each case. The resulting filtered headache case library consisted of 676 cases, 56.95% of which had been diagnosed with PM and 43.05% of which had been diagnosed with PTTH.
Patient feature selection
Sample headache case library
The weight of each feature was used to represent the importance of that feature in measuring similarity. The higher the weight of an attribute, the more relevant it was considered to be. Thus, establishing accurate weights was of high importance. In this study, the weights of the attributes were calculated using an evolutionary approach: the Genetic Algorithm (GA). The GA is a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to generate practical solutions to optimization and searching problems using techniques inspired by natural evolution, such as inheritance, mutation, selection, and crossover. First, an equal weight was assigned to each feature. Then, the GA searched the solution space and determined the optimized weight of each feature. The weights were stored in the case library and utilized as the case similarity measurement parameters.
where A is the problem case, B is an existing case in the case library, n is the number of attributes/features of each case, i is the individual features of each case from 0 to n, f is the similarity function of feature i in cases A and B, and w is the weight of each individual feature based on the number of matches, which is calculated in the last step.
Note that the recommended results obtained from the system cannot be automatically denoted the final diagnostic result. The results must be confirmed by a physician, who will determine whether the recommended K cases with the highest similarities meet the clinical needs of a patient.
The core functionality of this system was comprised of structured symptomatic input, computer-aided diagnosis, and automated report generation. Structured symptomatic input generalizes digitalized forms of cases and further analyzes cases. The CBR technique performs computer-aided diagnosis as the system ranks similar cases based on their similarity scores. Automated report generation reduces the time consumed by writing reports and increases work efficiency.
The proposed system, a web-based system, was deployed on a cloud platform in this study. It not only acted as a complete computer-aided diagnostic system, but also as a data center for primary headaches. A profile that could be updated by a physician at each successive follow-up appointment was created for each patient. A three-member panel checked each case in order to ensure accurate diagnoses and select representative cases for storage in the case library at regular intervals.
Test data set
The test data set was comprised of 222 previous cases that were collected by the International Headache Center in Chinese PLA General Hospital. These data were not included in the CBR CDSS case library. The diagnosis of each case was determined by a three-member panel as either PM or PTTH; 76.1% of the cases were PM.
Evaluation metrics and method
where TP, FP, FN, and TN refer to the number of true positives, false positives, false negatives, and true negatives, respectively, and N refers to the number of test data.
In addition to the above metrics, the Receiver Operator Characteristic (ROC) curve is another useful visual tool used to compare the accuracies of different algorithms. The ROC is a graphical plot of the sensitivity (the true positive rate) versus the false positive rate (one minus the true negative rate) of a binary classifier system as its discrimination threshold is varies. When the performance of a method is reflected by the area under its ROC curve (AUC), its effectiveness becomes apparent. As the area under the curve increases, the performance of the classifier also increases.
Experiment 1: the variation in performance as the value of K varies
In the KNN method, the value of K is most influential to the recommendation performance. An appropriate K value could improve the accuracy of the recommendation. In this experiment, the value of K was increased from 1 to 51, and the accuracy of each value of K was recorded.
Experiment 2: the comparison between the weighted CBR and unweighted CBR
Since there is no weighting operation in traditional CBR, each feature is assigned a weight of 1. Assigning different weights to the features of each case was expected to improve the recommendation accuracy. In order to validate this, the recommendation accuracies of the weighted CBR and unweighted CBR were compared using AUC and Pearson’s chi-square test. A p-value of 0.05 or less was used to indicate statistical significance. SPSS software for Windows (Version 18.0) was used for the statistical analyses.
Experiment 3: the comparison between the CBR CDSS and the guideline-based CDSS
The PM and PTTH diagnostic performances of the guideline-based CDSS and CBR CDSS were compared. A statistical analysis was also performed using Pearson’s chi-square test in order to determine whether a significant difference existed between these two methods. A p-value of 0.05 or less was used to indicate statistical significance.
The top 20 features and their weights
Date of headache onset
Aggravation by routine physical activity
Location of pain
Attack with fixed period
Duration of pain episodes
Family medical history
Menstruation related factor
The number of attacks
Attack frequency per month
During the weight calculations, most of the weights were determined to be consistent with the experience of the specialists’. For example, nausea is considered to be an important diagnostic feature for differentiating migraine and tension-type headaches . Thus, as shown in Table 4, nausea and vomiting were assigned higher weights. In addition, some features that are not included in the ICHD-3 beta criteria, such as family history, exercise, and menstruation-related factors, were also given higher weights.
The ROC curves of the weighted CBR and unweighted CBR were compared, and the AUC of the weighted CBR (0.983) was determined to be significantly larger than that of the unweighted CBR (0.799). This illustrated not only that the weighting operation was necessary for case matching, but also that the proposed approach was superior to the unweighted CBR approach. A statistical analysis between the weighted CBR and unweighted CBR was performed using the Pearson’s chi-square test. The PM chi-square value was equal to 9.571, and the p-value was 0.002 < 0.05. Likewise, the PTTH chi-square value was equal to 18.002, and the p-value was 0 < 0.05. The results indicated that there was a significant difference between the two approaches. Thus, the weighting operation was necessary for CBR.
The diagnostic performance for two systems
In this study, a CDSS for the diagnosis of headaches was developed using a CBR technique. The system demonstrated a higher diagnostic performance for PM and PTTH than that of a guideline-based CDSS. The proposed CDSS exhibited multiple advantages. Since CBR is a data-based reasoning approach, it is particularly effective in fields where summarizing explicit knowledge is difficult; thus, the proposed CDSS prevented the bottleneck of knowledge acquisition. In addition, unlike other machine learning methods, CBR does not require a model; instead, it calculates the similarities between the new and previous cases and, thereby, conserves time and effort. Furthermore, the proposed system gained intelligence as the number of cases increased due to the increased probability of a case with a high similarity.
The system was designed to assist with distinguishing PM from PTTH. The following example was used to illustrate the effectiveness of this system in practice. Patient A, a 24-year-old female, had headaches for four years that lasted approximately two days and were unilateral, pulsating, mild in intensity, and aggravated by routine physical activity with no nausea, vomiting, photophobia, or phonophobia. In addition, Patient A’s headache attacks were obviously related to her menstrual cycles, and her mother and sisters had experienced similar headaches. Determining the type of headache Patient A has is difficult since the diagnostic criteria of both PM and PTTH are met according to ICHD-3 beta. Based on this evidence, a similarity measurement of this case to previous clinical cases was performed. One case retrieved from the library, a 49-year-old female, had experienced recurrent attacks for more than 30 years; each attack lasted half a day or more. The patient’s headaches were unilateral, pulsating, severe in intensity, and aggravated by routine physical activity, with no nausea, vomiting, photophobia, or phonophobia. Furthermore, her attacks often occurred during menstruation. The similarity measurement of these two cases was equal to 0.965. Based on the information provided by the retrieved case, Patient A was determined to likely be experiencing migraine headaches.
However, the proposed CBR CDSS has some limitations. For example, the number of cases in the case library is inadequate due to the complexity of headaches. In addition, since some complicated diseases were not included in the case library, the diagnostic accuracy is not as high as it could be. Although this problem would be solved with the addition of new cases into the case library, the computing time would grow significantly as a result since CBR measures the distance between a new case and each case in the case library. The recent development of techniques used to manage large amounts of data could be a viable solution to this problem.
In this study, a weighted CBR method was applied to CDSSs in order to develop a CBR CDSS capable of differentiating between two types of probable primary headaches (PM and PTTH), which often confuse physicians in clinical practice. The results of the experiments indicated a high degree of accuracy in recognizing these two types of headaches and a dramatic improvement compared to guideline-based CDSSs. The accurate diagnosis of these types of headaches is imperative since their treatments involve different preventative therapies.
In future studies, the proposed CBR CDSS will be further evaluated in order to determine its validity. A prospective study and multicenter validation will be performed before it is recommended for routine clinical use. The final aim of this study is the development of a reliable CDSS for the diagnosis of different types of headaches using multiple techniques in order to assist physicians in primary care units and community hospitals and thereby improve the diagnosis and treatment of headaches.
Written informed consent was obtained from the patient for the publication of this report and any accompanying images.
This work was supported by the National High Technology Research and Development Program of China (863 Program) (Grant No. 2012AA02A602); the National Science and Technology Major Project of China (Grant No. 2013ZX03005012); the Clinical and Scientific Supporting Fund of Chinese PLA General Hospital (Grant No. 2012FC-TSYS-3041); the Capital Development Scientific Research (Grant No. 2011-5001-04 and Grant No. 2014-4-5013).
- Steiner TJ (2005) Lifting The Burden: the global campaign to reduce the burden of headache worldwide. J Headache Pain 6(5):373–377View ArticlePubMed CentralPubMedGoogle Scholar
- Stovner L, Hagen K, Jensen R, Katsarava Z, Lipton R, Scher A, Steiner T, Zwart JA (2007) The global burden of headache: a documentation of headache prevalence and disability worldwide. Cephalalgia 27(3):193–210View ArticlePubMedGoogle Scholar
- Messinger HB, Spielings EL, Vincent AJ (1991) Overlap of migraine and tension-type headache in the International Headache Society classification. Cephalalgia 11(5):233–237View ArticlePubMedGoogle Scholar
- Kaniecki RG (2002) Migraine and tension-type headache an assessment of challenges in diagnosis. Neurology 58(9 suppl 6):S15–S20View ArticlePubMedGoogle Scholar
- Applebee AM, Shapiro RE (2007) Cluster-migraine: does it exist? Curr Pain Headache Rep 11(2):154–157View ArticlePubMedGoogle Scholar
- Gelfand AA, Reider AC, Goadsby PJ (2013) Cranial autonomic symptoms in pediatric migraine are the rule, not the exception. Neurology 81(5):431–436View ArticlePubMed CentralPubMedGoogle Scholar
- Lucchetti G, Peres MF (2011) The prevalence of migraine and probable migraine in a brazilian favela: results of a community survey. Headache 51(6):971–979View ArticlePubMedGoogle Scholar
- Kim B-K, Chung YK, Kim J-M, Lee K-S, Chu MK (2013) Prevalence, Clinical Characteristics and Disability of Migraine and Probable Migraine: A Nationwide Population-Based Survey in Korea., Cephalalgia:0333102413484990Google Scholar
- Ho K, Ong B-C (2003) A community-based study of headache diagnosis and prevalence in Singapore. Cephalalgia 23(1):6–13View ArticlePubMedGoogle Scholar
- Vuković V, Plavec D, Pavelin S, Jančuljak D, Ivanković M, Demarin V (2010) Prevalence of migraine, probable migraine and tension-type headache in the Croatian population. Neuroepidemiology 35(1):59–65View ArticlePubMedGoogle Scholar
- Kopec D, Shagas G, Selman J, Reinharth D, Tamang S (2004) Development of an expert system for aiding migraine diagnosis. J Inform Technol Healthcare 200:355Google Scholar
- Simić S, Simić D, Slankamenac P, Simić-Ivkov M (2008) Rule-Based Fuzzy Logic System for Diagnosing Migraine. In: Artificial Intelligence: Theories, Models and Applications. Springer, pp 383–388Google Scholar
- Eslami V, Rouhani-Esfahani S, Hafezi-Nejad N, Refaeian F, Abdi S, Togha M (2013) A computerized expert system for diagnosing primary headache based on International Classification of Headache Disorder (ICHD-II). SpringerPlus 2(1):1–4, doi: 10.1186/2193-1801-2-199View ArticleGoogle Scholar
- Maizels M, Wolfe WJ (2008) An expert system for headache diagnosis: the Computerized Headache Assessment tool (CHAT). Headache 48(1):72–78PubMedGoogle Scholar
- Yin Z, Dong Z, Yu S, Lu X, Feng G, Duan H (2012) A guideline-based decision support system for headache diagnosis. Stud Health Technol Inform 192:1022–1022Google Scholar
- Dong Z, Yin Z, He M, Chen X, Lv X, Yu S (2014) Validation of a guideline-based decision support system for the diagnosis of primary headache disorders based on ICHD-3 beta. J Headache Pain 15(1):1–7View ArticleGoogle Scholar
- Koton PA (1988) Using Experience in Learning and Problem Solving. Massachusetts Institute of TechnologyGoogle Scholar
- Guessoum S, Laskri MT, Lieber J (2014) RespiDiag: a case-based reasoning system for the diagnosis of chronic obstructive pulmonary disease. Expert Syst Appl 41(2):267–273View ArticleGoogle Scholar
- Sharaf-El-Deen DA, Moawad IF, Khalifa M (2014) A new hybrid case-based reasoning approach for medical diagnosis systems. J Med Syst 38(2):1–11View ArticleGoogle Scholar
- Watson I (1999) Case-based reasoning is a methodology not a technology. Knowl-Based Syst 12(5):303–308View ArticleGoogle Scholar
- Saper JR (1996) Diagnosis and symptomatic treatment of migraine. Headache 37:S1–S14Google Scholar
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.