Reviewer tooling for CDSCO regulatory submissions
सीडीएससीओ नियामक प्रस्तुतियों के लिए समीक्षक उपकरण
Six locked workflows behind a single response envelope. Deterministic rule baselines on every path; LLM polish opt-in and injection-gated. Every metric named in Annexure I §II.5 has a reproducible offline harness in make eval-all.
Submission Guidelines §3.I
Workflows
- PS 1
Anonymise
POST /v1/anonymiseHybrid regex + NLP. HMAC pseudonymisation; optional irreversible generalisation. India-relevant identifiers (Aadhaar, ABHA, PAN, UHID, IMEI).
KIE micro F1 0.978 - PS 2
Summarise
POST /v1/summariseThree CDSCO source types: SUGAM checklist, SAE narrative, DTAB / DCC meeting minutes. Deterministic extractor; optional LLM polish behind injection gate.
ROUGE-L 0.253 · BERTScore F1 0.821 - PS 3
Completeness
POST /v1/completenessLocked schema registry. Per-item missing flags + cross-field consistency rules (5 cx-rules: fatal+recovered, non-serious+ICU, etc.).
accuracy 1.00 · MCC 1.00 - PS 4
Compare
POST /v1/comparedifflib + similarity ratio + added/removed lines. Pipe-markdown and TSV tables parsed into row-level table_deltas.
accuracy 1.00 · MCC 1.00 - PS 5
Classify
POST /v1/classifySeverity tiers: death · disability · hospitalisation · other. Routes immediate / priority / standard. SHA-256 + simhash + minhash dedupe.
Macro-F1 1.00 · MCC 1.00 - PS 6
Inspection report
POST /v1/inspection-report4-section CDSCO template. Numbered observations with severity + recommended actions + due dates. Tesseract OCR for scans.
OCR CER 0.0058 (Tesseract live)
Annexure I §II.5 - Technical robustness
Live evaluation gates
Reproducible offline. Numbers regenerate on every make eval-all run.
| Metric | Result | Source |
|---|---|---|
| k-anonymity / l-diversity / t-closeness | 1.0 / 5.5 / 0.625 | last_eval.json |
| KIE strict micro F1 / macro F1 | 0.978 / 0.964 | last_eval_kie.json |
| ROUGE-1 / ROUGE-2 / ROUGE-L F1 | 0.294 / 0.161 / 0.253 | last_eval_summarise.json |
| BERTScore (live) | F1 0.821 | VAIDYAH_BERTSCORE=1 |
| Classification Macro-F1 / MCC | 1.00 / 1.00 | last_eval_classify.json |
| Compare + Completeness MCC | 1.00 / 1.00 | respective JSON |
| OCR CER - synthetic | 0.0046 | last_eval_ocr.json |
| OCR CER - Tesseract live | 0.0058 | VAIDYAH_OCR_BACKEND=pytesseract |
| Segmentation mIoU (synthetic) | 0.648 | last_eval_segmentation.json |
| Latency p95 (8-worker) | 8.7 – 12 ms | last_eval_load.json |
| Throughput per route | 875 – 1135 rps | last_eval_load.json |
| Adversarial robustness | 33 / 33 across 6 categories | last_injection_gate.json |
| PII leak gate | 0 leaks / 4 docs | last_leak_test.json |
GET /v1/workflows
Workflow registry
- Workspaceslive
/v1/workspaces/*
- Reviewer queuelive
/v1/queue
- Batch triagelive
classify (batch)
- Anonymise documentlive
/v1/anonymise
- Summarise documentlive
/v1/summarise
- Check completenesslive
/v1/completeness
- Compare versionslive
/v1/compare
- Classify + prioritiselive
/v1/classify
- Inspection reportlive
/v1/inspection-report