Medical professional rarely support Diabetic Alert Dogs and current research often concludes they are inaccurate and unreliable. To me it's critical that research is conducted in the field to prove the dogs have the physical ability to assist those with type 1 diabetes. But these studies can be hard to find and tedious to sift through. I will be writing blog posts breaking these studies down and analysing them from both a dog trainer's perspective as well as an academic's perspective. Below is the first post.
Discussion of:
Reliability of Trained Dogs to Alert to Hypoglycemia in Patients With Type 1 Diabetes
In 2017, researchers Los, Ramsey, Gottman-Bauman, and Ahmann published an exploratory study to attempt to evaluate the reliability of Diabetic Alert Dogs (DADs)1. One of the primary reasons for the study was that other authors’ previous research focused on perceived effectiveness of DADs, preventing medical professionals from providing information to patients regarding the physical benefits to those with diabetes and not just perceptions.
Los and colleagues’ study incorporated eight DADs and their handlers over one week. Each handler had type 1 diabetes and wore the Dexcom continuous glucose monitor (CGM) for one week, blinded to the readings. Every time the subject’s dog alerted, the subject recorded their capillary blood glucose value (blood sugar reading from a meter) and their current symptoms. After the week, the researchers downloaded and analyzed subjects’ CGM data and DAD alert records.
The authors concluded that DADs have a positive predictive value of only 12%, essentially only 12% of the dogs’ alerts actually occurred during hypoglycemia. This makes it appear that DADs are incredibly inaccurate and unreliable. However, the researchers failed to incorporate numerous critical aspects into their calculations, causing it to appear the dogs’ performances were very poor. The primary oversights are discussed below, demonstrating why this study was poorly designed and is not an accurate representation of the dogs’ reliabilities so additional research is needed.
While DADs are certainly far from perfect and this research asks an important question which needs to be answered, the study was not designed to properly analyze the reliability of Diabetic Alert Dogs. The authors failed to take into account numerous critical aspects that significantly altered the results of the DADs' reliabilities throughout the week of testing.
1.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5505410/
2. https://www.fda.gov/media/98983/download
Los and colleagues’ study incorporated eight DADs and their handlers over one week. Each handler had type 1 diabetes and wore the Dexcom continuous glucose monitor (CGM) for one week, blinded to the readings. Every time the subject’s dog alerted, the subject recorded their capillary blood glucose value (blood sugar reading from a meter) and their current symptoms. After the week, the researchers downloaded and analyzed subjects’ CGM data and DAD alert records.
The authors concluded that DADs have a positive predictive value of only 12%, essentially only 12% of the dogs’ alerts actually occurred during hypoglycemia. This makes it appear that DADs are incredibly inaccurate and unreliable. However, the researchers failed to incorporate numerous critical aspects into their calculations, causing it to appear the dogs’ performances were very poor. The primary oversights are discussed below, demonstrating why this study was poorly designed and is not an accurate representation of the dogs’ reliabilities so additional research is needed.
- Failure to account for hyperglycemia alerts. While the researchers’ goal was to focus on hypoglycemic events, they failed to consider that most DADs are trained to alert to hyperglycemic events as well. Every dog’s alert above 70mg/dL, even if during a hyperglycemic event, was considered a false alarm for the purpose of this study. The authors mention this was not anticipated when designing the study and should be studied further. However, this is not just something that “should be studied further”. This oversight caused an inaccurate number of false alarms because DADs which correctly alerted to hyperglycemic events throughout the week were considered to be alerting incorrectly, solely due to the authors’ failure to research DADs prior to designing the study. Furthermore, instead of reanalyzing the data according to the dogs’ training to alert to hyperglycemia, the authors acknowledged the limitation and released clearly inaccurate results.
- Inconsistent hypoglycemia threshold. This study considered a DAD’s alert to be valid only if the subject’s glucose level was 70mg/dL or below at the time of an alert. However, a DAD’s trained criteria for hypoglycemia alerts varies based on the organization and client; some alert to ≤80mg/dL and others even alert to ≤100mg/dL. Setting a universal hypoglycemia threshold for the study without taking into consideration each dog’s training provides a falsely excessive number of false alarms, as almost all DADs are trained to alert to a threshold above 70mg/dL. The researchers even acknowledged that there were frequent alerts between 70-99mg/dL. However they only responded to this observation by calculating the percent of instances where the handler’s blood sugar would drop below the threshold within 20 minutes if it continued at that rate of change. While interesting, it is not nearly as relevant as considering the dog’s trained alert threshold. To take this variation into account appropriately, the researchers could have asked each individual what their dog’s trained threshold was and evaluated each dog according to its individual training.
- Inconsistent meters. Another aspect to consider in this study is the various glucose monitors used. Glucose meter accuracy varies greatly and four subjects utilized meters which did not pass the Diabetes Technology Societies (DTS) accuracy standards, so the dogs were being compared to a possibly inaccurate measurement. While there is no perfectly accurate glucose meter, to consider this the researchers could have provided subjects with the same brand of meter which did pass the DTS accuracy standards.
- Inaccurate CGMs. In addition to comparing the dogs alerts to inaccurate glucose meters, the researchers were also comparing the dogs alerts to CGMs which are known to be incorrect at times, which is often why individuals turn to DADs. The study claims the dogs alerted to only 36% of the lows according to the CGM data. But, the CGM is known to provide false alarms. In the Dexcom G5 FDA briefing committee documents2, when the CGM has a low threshold set to 70mg/dL for a pediatric patient, the true alert rate (PPV) is only 68% with the missed alert rate of 25%. When analyzing DAD missed alerts, the researchers were basing the data solely on a piece of technology which has a false alert rate of 32% meaning that the hypoglycemic events that the DADs “missed”, rely upon the CGM not having a false alarm at the time. The Dexcom is one of the most accurate CGMs on the market, so there is not a CGM that would have been preferable. However, the researchers could have incorporated an alert system which allowed the subjects to be made aware of CGM hypoglycemia alerts and prompted them to check their glucose with a meter, providing a more accurate result and confirming whether the DAD missed the alert or the CGM was giving a false alarm.
While DADs are certainly far from perfect and this research asks an important question which needs to be answered, the study was not designed to properly analyze the reliability of Diabetic Alert Dogs. The authors failed to take into account numerous critical aspects that significantly altered the results of the DADs' reliabilities throughout the week of testing.
1.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5505410/
2. https://www.fda.gov/media/98983/download
