While in Chicago recently, I was asked how we validated our quality measures when we moved from chart abstraction to automated computation of PRQS, Meaningful Use, Pioneer ACO, and Alternative Quality Contract measures via the Massachusetts eHealth Collaborative Quality Data Center (QDC). This is an important question because Meaningful Use Stage 2 enables easy use of modular components outside the EHR such that data can be captured in the EHR and sent to a cloud based analytics engine via standards such as CCD/C32 for content and Direct for transport.
Initially we did spot checks to validate the integrity of the Continuity of Care Document data flows from electronic health records to the normalized QDC schema.
When Mitre Corporation offered to test their popHealth tool against 2 million BIDMC patient records to validate the Meaningful Use quality measures computed by our QDC, we jumped at the opportunity.
First, we ensured appropriate business associate agreements were in place to protect the privacy of patient data. Next, we required all work to be done on site in the Quality Data Center to protect the security and integrity of clinical summary data.
Mitre ran the tool against 2 million BIDMC Continuity of Care Documents and compared the results to the reports generated by the QDC.
The results were enlightening.
The computations aligned well for most quality measures, justifying our early manual validation.
However, Mitre discovered ambiguities in the CCD specification itself that led to some differences in the calculations. This was despite our use of this CCD implementation guide which provides even greater specificity than the HL7 standard.
For example, the CCD does not specify an allergy vocabulary. At BIDMC we use First Data Bank to codify medication allergies. PopHealth expects RxNorm, the vocabulary standard required for exchanging medication history. Even the Stage 2 NPRM does not specify an allergy vocabulary and we recognized the need to enhance the Stage 2 to include RxNorm for medication allergies (Penicillin VK), NDF-RT for categories of medication allergies (all Penicillins and Cephalosporins) and SNOMED-CT for non-medication allergies (food and environmental agents).
I'll post other pertinent findings from the Mitre analysis after the debrief meeting.
Mitre demonstrated their work at HIMSS in the interoperability showcase.
BIDMC and MAeHC were proud to participate in this event, which we hope provided lessons learned for other provider, payer, and government stakeholders wanting to compute quality measures in the cloud using popHealth.
John Halamka, MD, blogs regularly at Life as a Healthcare CIO.
