An Exploratory Case Study of Vancomycin Therapeutic Drug Monitoring (TDM-AID)

The Problem

Hospital administrators are increasingly asked to approve the deployment of artificial intelligence tools, including large language models (LLMs), in high-risk clinical workflows where errors can cause direct patient harm. Existing governance frameworks provide high-level ethical principles but offer limited operational guidance for evaluating and deploying these tools safely.

The Solution

A hybrid clinical decision support system, TDM-AID, was developed and evaluated as a proof-of-concept. The system integrates three modules:

1. A deterministic pharmacokinetic engine for rule-based calculations.
2. An LLM (GPT-4o) for clinical interpretation and generating recommendations.
3. A retrieval-augmented generation (RAG) component using the Ministry of Health Malaysia's clinical handbook to ground outputs in local practice standards.

The study used vancomycin therapeutic drug monitoring (TDM), a high-risk workflow, as its test case.

Implementation

The system was retrospectively evaluated against 30 adult vancomycin TDM cases from Hospital Tengku Ampuan Rahimah, a Malaysian tertiary hospital. Two independent expert pharmacists scored the system's outputs against the original pharmacist consultations using a purpose-built, six-domain weighted rubric. The study team then performed a structured qualitative analysis of the system's failure modes to derive candidate safeguards for safer deployment.

Outcomes and Impact

The deterministic calculation module achieved 100% accuracy. However, the LLM-dependent components showed variable performance. Clinical judgement and core recommendations scored 'Good' (83% median score), but prospective predictions 'Needed Improvement' (58%), and timing recommendations for resampling failed in all cases (0%). The system also generated dose recommendations that exceeded a pre-defined safety screening threshold in 17% of cases.

From this analysis, the study derived four candidate safeguards:

1. Architectural separation: Calculable tasks should be handled by deterministic functions, not generated by an LLM.
2. Hard-coded safety boundaries: Known safety limits (e.g., maximum dose) should be enforced as system overrides, not just advisory flags.
3. Deterministic logistics: Time-critical scheduling and timing should be handled by rule-based engines.
4. Structured pre-deployment evaluation: Tools should be assessed against an expert-benchmarked rubric before deployment, with poor performance in any domain acting as a potential barrier.

The study received ethical approval from the Medical Research and Ethics Committee, Ministry of Health Malaysia. The paper discusses the local regulatory context under the Medical Device Authority (MDA) and the Medical Device Act 2012. It also references international frameworks, including the WHO's guidance on AI for health, the EU AI Act (Regulation 2024/1689), and the ISO/IEC 42001:2023 standard for AI management systems.