Technical Report: Fine-Tuning Qwen 2.5 0.5B for Dental Office Administration (v0.1)
This report documents the first iteration of Sephrain's dental clinic AI assistant pipeline. The objective is a compact, browser-deployable model capable of handling scheduling, intake triage, and billing queries for dental practices — running entirely on the patient's device with zero data leaving the browser.
1. Base Model
Qwen 2.5 0.5B-Instruct (494M parameters). Selected for its instruction-following baseline, multilingual tokenizer, and small footprint suitable for WebAssembly browser deployment. The Instruct variant ships with a ChatML chat template, eliminating the need for manual template configuration.
2. Dataset Curation (Module 1)
A mixed training dataset of ~8,000 ShareGPT-format conversations was assembled from four sources:
- Task-Oriented Dialogues (40%) — Schema-Guided Dialogue (SGD) scheduling and booking conversations, adapted to dental office context
- Medical Dialogue Style (20%) — HealthCareMagic/ChatDoctor patient interaction patterns for tone and empathy calibration
- Dental Domain Knowledge (15%) — CDT procedure codes (D0120–D9230), dental terminology, insurance/billing vocabulary
- Hand-Crafted Examples (25%) — Gold-standard dental office conversations covering scheduling, triage, billing, and intake scenarios
3. Tokenizer Profiling (Module 2)
Analysis of Qwen 2.5's tokenizer on dental vocabulary confirmed reasonable handling of CDT codes, clinical terms, and drug names. Terms fragmenting into 3+ tokens were flagged for potential continued pretraining in future iterations.
4. Training Configuration (Module 4)
Fine-tuned using Unsloth with DoRA (Weight-Decomposed Low-Rank Adaptation) and QLoRA 4-bit quantization. Response-only masking was applied — the model learns only from assistant responses, not user or system tokens.
base: Qwen2.5-0.5B-Instruct | DoRA r=32 a=64 | dropout=0.05
lr=2e-4 | cosine schedule | warmup=5% | weight_decay=0.01
batch=16 effective (4×4 grad accum) | epochs=5
max_seq_length=1024 | train_on_responses_only=True
optimizer=adamw_8bit | val_split=5%
5. Export & Deployment
DoRA adapters were merged into the base model and exported to GGUF format in two quantizations: Q4_K_M (~250 MB) for browser deployment and Q8_0 (~507 MB) for higher-quality inference. The model is served in-browser via wllama (llama.cpp compiled to WebAssembly), enabling zero-server-cost inference with complete patient data privacy — no information leaves the device.
6. Roadmap (v0.2)
Formal evaluation harness for dental admin task completion rates (scheduling accuracy, CDT code recognition, triage escalation detection). Expanded hand-crafted examples based on partner clinic feedback. Evaluation of the 1.5B Qwen variant for quality-sensitive triage tasks. WebGPU acceleration via WebLLM for GPU-equipped devices.
Model weights available on HuggingFace. Full benchmark results with v0.2.