AI Model Suite
The Specialist Panel
AOB does not rely on a single model. Five distinct AI specialists — two vision models and three domain-tuned LLM adapters — collaborate in a structured pipeline. All reside simultaneously in the AMD MI300X's 192 GB HBM3 unified memory pool.
3 GBGigaPath
15 GBQwen-VL 7B
70 GBLlama 3.3 70B
16 GBLoRA suite ×3
30 GBKV cache
9 GBQdrant + overhead
143 GB / 192 GB used · 49 GB headroom
🔬
Prov-GigaPath
ViT-Giant · Vision Foundation Model
Agent 1 · Pathologist◌ Status unknown
Prov-GigaPath is a pre-trained ViT-Giant model trained on 1.3 billion pathology image tokens from Providence Health. It extracts rich morphological embeddings from 224×224 histology patches. We use it as a frozen encoder — no fine-tuning required. Monte Carlo dropout (N=20 passes) provides calibrated uncertainty estimates.
Parameters1.1 B
VRAM~3 GB (FP16)
Precisionfloat16
Sourceprov-gigapath/prov-gigapath
Sample Input
224×224 histology patch tensor [1, 3, 224, 224], FP16
Sample Output
{"tissue_classification": "lung_adenocarcinoma", "confidence": 0.94, "uncertainty_interval": "±4.2%", "morphological_features": ["glandular patterns", "nuclear atypia"]}
🧠
Qwen2.5-VL-7B-Instruct
Vision–Language Model · Second Opinion
Agent 1b · Second Opinion◌ Status unknown
Qwen2.5-VL provides a language-grounded second opinion on each histology patch. Unlike GigaPath (pure vision), Qwen-VL can reason in natural language about visual features and flag edge cases where GigaPath's confidence is low. Its 7B parameters fit alongside all other models in the MI300X's 192 GB VRAM.
Parameters7 B
VRAM~15 GB (BF16)
Precisionbfloat16
SourceQwen/Qwen2.5-VL-7B-Instruct
Sample Input
Histology patch image + prompt: 'Describe tissue morphology and flag any malignant features'
Sample Output
"The patch shows irregular glandular structures with pleomorphic nuclei and increased mitotic activity, consistent with adenocarcinoma. Nuclear-to-cytoplasmic ratio is elevated. EGFR mutant morphology pattern suspected."
🧬
TNM Staging LoRA
Llama-3.1-8B · LoRA Rank 16
Agent 2b · Staging Specialist● Loaded
A LoRA adapter fine-tuned on AJCC 8th-edition TNM staging cases. Base model is Llama-3.1-8B-Instruct; rank-16 adapter constrains the output to valid TNM notation (T0–T4, N0–N3, M0–M1). Outperforms the base 70B model on structured staging by +7.9 pp TNM accuracy.
Parameters8 B base + 4.2 M LoRA
VRAM~5 GB (FP16) shared with base
Precisionfloat16
SourceLlama-3.1-8B + custom adapter
Sample Input
{"tissue": "lung_adenocarcinoma", "size_cm": 3.2, "node_involvement": "ipsilateral", "distant_mets": false}
Sample Output
{"T": "T2a", "N": "N1", "M": "M0", "stage": "IIB", "nccn_category": "2A"}
🔭
Biomarker Specialist LoRA
Llama-3.1-8B · LoRA Rank 16
Agent 2c · Biomarker Panel● Loaded
Fine-tuned to identify actionable biomarkers (EGFR, ALK, ROS1, KRAS, BRAF, PD-L1, MSI-H) from morphological findings and tissue type. Generates structured test-ordering panels and gates treatment options behind pending molecular results — exactly as NCCN guidelines require.
Parameters8 B base + 4.2 M LoRA
VRAM~5 GB (FP16) shared with base
Precisionfloat16
SourceLlama-3.1-8B + custom adapter
Sample Input
{"tissue": "lung_adenocarcinoma", "morphology": ["glandular", "solid"], "smoking_history": "never"}
Sample Output
{"recommended_panel": ["EGFR", "ALK", "ROS1", "KRAS G12C"], "priority": "EGFR", "rationale": "Never-smoker adenocarcinoma — EGFR mutation rate ~50%"}
💊
Treatment Specialist LoRA
Llama-3.1-8B · LoRA Rank 16
Agent 2d · Treatment Planning● Loaded
Generates NCCN-aligned first- and second-line treatment options conditioned on TNM stage and biomarker results. Trained on NCCN Clinical Practice Guidelines (lung, colon, breast) and TCGA outcome studies. Always cites the specific NCCN category (1, 2A, 2B, 2B-preferring) for each recommendation.
Parameters8 B base + 4.2 M LoRA
VRAM~5 GB (FP16) shared with base
Precisionfloat16
SourceLlama-3.1-8B + custom adapter
Sample Input
{"stage": "IIIA", "tissue": "lung_adenocarcinoma", "egfr": "mutant (exon 19 del)", "pdl1_tps": 35}
Sample Output
"First-line: Osimertinib 80 mg/day (NCCN Category 1, FLAURA trial). Second-line: Platinum-doublet chemotherapy if progression on TKI."