Should AI scribes used in clinical practice be regulated?

The Therapeutic Goods Administration (TGA) is working to determine what oversight should be required for the use of AI scribes used in clinical practice and plans to publish a summary of the outcomes in coming months.

The tools are increasingly used in clinical settings to capture conversations between patients and healthcare practitioners as well as having the ability to generate clinical notes, summaries or even referral letters.

Results from a study released last year found that clinical notes compiled by AI scribes were of a higher quality than those put together by GPs.

A TGA spokesperson told Medical Forum that while the regulator oversees the use of therapeutic goods, there are elements of AI scribes that fall outside of its remit.

It does not regulate clinical practice, privacy, informed consent by health practitioners, or the use of AI tools by health practitioners.

However, it is working with other agencies including the Office of the Australian Information Commissioner, the Australian Commission on Safety and Quality in Health Care, and Ahpra to support their oversight of the use of AI tools in clinical practice.

The review of digital scribes, which has been underway for almost a year, is looking at public information and materials provided by developers to confirm if individual digital scribe products should be considered medical devices.

User complaints and concerns submitted to the TGA, and expert views from health practitioners, medical software industry experts, consumer groups and advisory committees are also being considered.

In Australia, if an AI scribe is used for transcribing and summarising a conversation it is treated as a regular administrative tool and not regulated by the TGA.

In cases where the AI tool is used beyond that for analysing or interpretation, such as providing diagnoses or prognoses, it may be seen as a medical device under the Therapeutic Goods Act.

In such cases those devices would have to comply with regulatory frameworks, including being listed on the Australian Register of Therapeutic Goods (ARTG).

However, the spokesperson told Medical Forum there are currently no digital scribes included in ARTG.

The review is ongoing, prioritising those products more widely used in Australia. 

The AMA has said while the use of digital scribes can assist in enhancing the health and wellbeing of patients, it does come with risks and has stressed that use needs to be clinically led, ethical, safe, and patient-centred.

Safe adoption and urgent action

The focus on AI scribes comes at the same time that the Australian Alliance for Artificial Intelligence in Healthcare (AAAiH) released its new AI roadmap outlining priorities for safe adoption of the technology while also calling for urgent action.

The third National Policy Roadmap for AI in Healthcare acknowledged AI was opening new pathways to a smarter, more efficient, and more adaptive health system while also being a major disrupter, presenting both opportunities and threats.

“From new digital ‘front door’ virtual health services, the algorithmic interpretation of imaging and pathology data, triage, resource allocation, clinical documentation to the personalisation of therapy, AI today is being used both by consumers and clinicians and is touching many aspects of healthcare,” the roadmap stated.

It pointed out that in the past two years the pace of industry growth far exceeded our current capabilities to ensure the technology was safe and effective.

“For Australia’s healthcare system to remain sustainable and resilient, it will need to shift gears, and quickly, from catching up with overseas industry, to moving ahead and defining our own pathway.

“If there is to be wholesale change to healthcare, then it must be on our terms. There is a clear and increasingly urgent role for Australian governments and agencies to set the conditions for safe and effective adoption of AI services in healthcare.”

Earlier this month Prime Minister Anthony Albanese announced his government would introduce a "world-leading" artificial intelligence framework to ensure Australia could capture the opportunity, share the benefits and keep Australians safe.

The government is expected to outline its whole-of-government AI consumer safety priorities in coming weeks.