- Jupyter Notebook 79.6%
- Python 20%
- Shell 0.3%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
|
Some checks failed
Deployment / Build Image (watcher) (push) Has been cancelled
Deployment / Deploy Service (watcher) (push) Has been cancelled
Deployment / Deployment Summary (push) Has been cancelled
Functional testing / pytest (push) Has been cancelled
Lint / Lint (push) Has been cancelled
|
||
| .codex/environments | ||
| .github | ||
| .vscode | ||
| data | ||
| docs | ||
| hcms | ||
| kubernetes/worker-service | ||
| notebooks | ||
| script | ||
| tests | ||
| .dockerignore | ||
| .editorconfig | ||
| .flake8 | ||
| .gitignore | ||
| .jscpd.json | ||
| .markdownlint.yml | ||
| .mypy.ini | ||
| .pre-commit-config.yaml | ||
| .prettierignore | ||
| .prettierrc.yml | ||
| .pylintrc | ||
| AGENTS.md | ||
| compose.lint.yml | ||
| compose.yml | ||
| contributing.md | ||
| Dockerfile | ||
| env.example | ||
| evaluate_dataset.py | ||
| generate_dataset.py | ||
| license | ||
| poetry.lock | ||
| poetry.toml | ||
| pyproject.toml | ||
| readme.md | ||
| upload_to_gdrive.py | ||
| utils.sh | ||
HCMS
Hybrid Clinician-in-the-Loop Monitoring System (HCMS) for safety monitoring and evaluation of LLM conversational agents.
Requirements
- Python
3.13+ - Poetry
2.1.2 - Docker with Compose v2
pre-commit
Installation
-
Bootstrap local environment:
./script/bootstrap -
Install package dependencies:
./script/install
Usage
Paper dataset generation and evaluation instructions are documented in:
Lint and tests
-
Run linters:
./script/lint -
Run tests (excluding regression):
./script/test -
Run full tests (including regression):
./script/test -r
Local monitoring stack
Use these lifecycle scripts for local MongoDB + watcher:
-
Build stack:
./script/build -
Start stack:
./script/start -
Stop stack (keep data):
./script/stop -
Destroy stack (remove containers + volumes):
./script/destroy
Watcher runtime behavior:
- Startup backfill retries transient provider failures (default
10retries,2sbetween no-progress retries). - Change stream reconnect uses exponential backoff with jitter (default
1sbase,30smax,0.25sjitter). - Logging level defaults to
INFOand can be tuned withHCMS_LOG_LEVEL.
Main tuning variables (set in .env):
MONGODB_CHANGE_STREAM_MAX_AWAIT_MSMONGODB_BACKFILL_BATCH_SIZEMONGODB_BACKFILL_MAX_RETRIESMONGODB_BACKFILL_RETRY_SLEEP_SECONDSMONGODB_RECONNECT_BACKOFF_BASE_SECONDSMONGODB_RECONNECT_BACKOFF_MAX_SECONDSMONGODB_RECONNECT_BACKOFF_JITTER_SECONDSHCMS_LOG_LEVEL
Connect to local MongoDB from host tools (MongoDB Compass, mongosh, app clients):
mongodb://mongo:27017/?replicaSet=rs0
Note: the local replica set member is advertised as mongo:27017 inside Docker.
From the host machine, use directConnection=true to avoid hostname resolution
errors for mongo.
Documentation
- Architecture Decision Record (ADR) for compose split and local monitoring stack: Compose restructure
- ADR for Mongo schema alignment and realtime watcher: Schema and watcher
- Paper data workflow: paper-data-workflow.md
- CI/CD deployment: ci-cd-deployment.md
- Contribution guide: contributing.md
Related
- Readme template: minimal-readme
- Shell style: styleguide-sh