The big reveal happened last week when NHSX launched its Digital Technology Assessment Criteria, complete with an NHS styled catchy acronym “DTAC”. After almost two painful years of wrangling between various government health technology quangos, this was the result. I can forgive most people for trying to work out what’s changed and why this was launched? There’s always a possibility that when Government quangos or Arms Length Bodies get created that policy gets rebadged, rewritten or reissued.
If we go back a few years to a time long ago before NHSX existed, well perhaps not that long ago, the NHS Apps library was launched in 2017 by NHS England with NHS Digital as part of the citizen-facing initiative. It was part of the Digital Transformation Portfolio to very clearly create a national library of apps, approved once across England for use by citizens and patients. The route of approval was through the Digital Assessment Questions, which was managed via NHS Digital. In the run-up to the 70th birthday of the NHS, the Apps library was heralded a huge success.
In the last two years something clearly changed, not only was NHSX created but there seemed to be confusion as to what should be included in the NHS Apps library. The library began to see a few apps appearing that weren’t patient or citizen-facing but were in fact clinician facing. It’s still unclear how that deviation came about but it certainly added confusion to the developer community - more technology companies wanted their apps listed - and yet were finding they weren’t able to, often without clear reasoning.
The end of the NHS Apps library
DTAC has come in response to developers and commissioners of digital health technology, apparently wanting simplified criteria. That assessment criteria now only covers clinical safety, data protection, technical assurance, interoperability (pass/fail) and – perhaps as an afterthought - usability & accessibility (scored).
The criteria itself is also interesting, large parts of it are unassessed so it’s unclear how the information will be used and what impact it’ll have on decision making. The big change from previous criteria is that this assessment now applies to any product, not just citizen-facing apps but pretty much anything that can be used in health and social care, for example, workforce apps or clinical decision apps. It appears that there will be a ‘transition away from the NHS Apps library’ and certain procurement frameworks (HSSF) will also need to incorporate the criteria, especially those that use using national funding.
Burden of proof
What still seems to be missing though is any assessment of the evidence of how well a product works and to what extent? This is notoriously difficult, but increasingly more important as we enter the world of digital therapeutics and combined technologies where medical technology, apps and medicines will all begin to blend.
There isn’t any sense of how the global evidence base was applied to even creating these new criteria, there are other countries that have taken a different approach including Belgium and Germany. Belgium has introduced different levels so that the decision to ‘approve’ a product is less binary, but more depending on what the intended outcome is, whereas Germany has set a high bar but that links the approval decision to reimbursement on patient-facing therapeutic applications.
A mixed estate of DTAC products
NHSX says that those buying digital health technologies ‘should also ensure that they consider efficacy and the impact and evidence’ and that it’s working with NICE to build on the Evidence Standards Framework for digital health technologies, but it isn’t part of the criteria currently. This has the potential to create a mixed estate of those that have been proven to meet the DTAC now versus those who might meet a later iteration that includes evidence. What’s more DTAC will only apply to new products and contracts within the NHS, and won’t assess solutions retrospectively – resulting in a mix of products that will and won’t have met the criteria being used from one department or organisation to the next. The fact that there doesn’t appear to be any regulatory change that mandates any of this criteria and that it hasn’t been issued as a formal standard that would require adoption, is also likely to create a lack of the precise consistency it was meant to bring.
A digital postcode lottery
It’s inevitable that not all the criteria can be applied uniformly, for example, the expectations of clinical decision support systems used in healthcare settings will need a different type of assessment to applications used by citizens to access health information or need signposting. With organisations being left to their own 'devices’, this could result in different hospitals and commissioners interpreting the criteria in different ways. We’ve already seen this when it comes to other procurement decisions so it seems unlikely that this will be any different. Only time will tell if a post-code lottery emerges in digital health technology but the potential is certainly there. Could we end up with some digital health inequalities? Will there be a novel application of the infamous inverse care law? Right now this looks likely.
Prof Sam Shah is the founder and director of the Faculty of Future Health with Ulster University, CMO of Numan and advisor to Silver Buck.
