feat: update default persistence to MongoDB and enhance configuration options
Chore App Build, Test, and Push Docker Images / build-and-push (push) Failing after 2m19s

This commit is contained in:
2026-07-29 16:45:10 -04:00
parent da7ed41938
commit f3e9201afd
6 changed files with 13 additions and 13 deletions
+2 -2
View File
@@ -8,7 +8,7 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
- Activate venv: `source .venv/bin/activate`
- Dev server: `python -m flask run --host=0.0.0.0 --port=5000` (entry: `main.py`)
- Required env vars: `SECRET_KEY`, `REFRESH_TOKEN_EXPIRY_DAYS`, `DIGEST_TOKEN_SECRET`, `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY` — Flask raises `RuntimeError` on boot if any are missing
- Optional persistence switch: `USE_MONGODB` (`true` | `false`). When `true`, set `MONGO_URI` and optionally `MONGO_DB_NAME`. Default `false` keeps TinyDB.
- Optional persistence switch: `USE_MONGODB` (`true` | `false`). Defaults to `true`; set `MONGO_URI` (and optionally `MONGO_DB_NAME`). Set to `false` to use TinyDB instead.
- Optional: `DB_ENV` / `DATA_ENV` (`prod` | `test` | `e2e`) — picks `data/` vs `test_data/` dir (see `config/paths.py`). For MongoDB these also select the default database name (`chore_db`, `chore_db_test`, `chore_db_e2e`) unless `MONGO_DB_NAME` is set.
- Tests: `pytest tests/``conftest.py` forces `DB_ENV=test`, `USE_MONGODB=true`, `MONGO_URI=mongomock`, and sets dummy secrets. Single test: `pytest tests/test_routine_api.py::test_name`
- Python imports assume `backend/` is on `sys.path` (set by `conftest.py` / `flask run` cwd). Run pytest from `backend/`.
@@ -31,7 +31,7 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
### Models — strict 1:1 parity
- Python `@dataclass`es in `backend/models/`. TypeScript interfaces in `frontend/src/common/models.ts`. Any model change requires updating both.
- Persistence is TinyDB by default, or MongoDB when `USE_MONGODB=true`. Both are accessed through the `LockedTable` / `MongoLockedTable` wrappers in `backend/db/db.py`. Operate on model instances with `from_dict()` / `to_dict()` — never raw dicts.
- Persistence is MongoDB by default (`USE_MONGODB=true`), or TinyDB when `USE_MONGODB=false`. Both are accessed through the `LockedTable` / `MongoLockedTable` wrappers in `backend/db/db.py`. Operate on model instances with `from_dict()` / `to_dict()` — never raw dicts.
- MongoDB client initialization is lazy (`backend/db/mongo_client.py`). `backend/gunicorn.conf.py` provides the `post_fork` hook required for multi-worker Gunicorn deployments; `backend/Dockerfile` loads it with `-c gunicorn.conf.py`.
- Migration script: `cd backend && python -m scripts/migrate_to_mongodb [--dry-run]`. It reads TinyDB JSON files and writes them to MongoDB idempotently, backing up the originals to `<db_dir>/backups/<timestamp>/`.
+5 -5
View File
@@ -38,7 +38,7 @@ npm run dev
| `ACCOUNT_DELETION_THRESHOLD_HOURS` | Hours to wait before deleting marked accounts | 720 (30 days) |
| `DB_ENV` | Database environment (`prod` or `test`) | `prod` |
| `DATA_ENV` | Data directory environment (`prod` or `test`) | `prod` |
| `USE_MONGODB` | Use MongoDB instead of TinyDB (`true`/`false`) | `false` |
| `USE_MONGODB` | Use MongoDB (`true`/`false`) | `true` |
| `MONGO_URI` | MongoDB connection URI (required when `USE_MONGODB=true`) | — |
| `MONGO_DB_NAME` | MongoDB database name (optional) | Parsed from `MONGO_URI`, or `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV` |
@@ -46,8 +46,8 @@ npm run dev
The application supports two persistence backends:
- **TinyDB** (default): JSON-file storage in `backend/data/db/` (or `backend/test_data/db/` for `test`/`e2e`). No extra configuration needed.
- **MongoDB**: Set `USE_MONGODB=true` and provide `MONGO_URI`. Useful for production deployments and managed database hosting (e.g., MongoDB Atlas).
- **MongoDB** (default): Set `MONGO_URI` (and optionally `MONGO_DB_NAME`). This is the recommended backend for production and managed hosting (e.g., MongoDB Atlas).
- **TinyDB**: JSON-file storage in `backend/data/db/` (or `backend/test_data/db/` for `test`/`e2e`). Opt in by setting `USE_MONGODB=false`.
#### Migrating from TinyDB to MongoDB
@@ -62,9 +62,9 @@ python -m scripts.migrate_to_mongodb
The migration script reads the existing TinyDB JSON files and inserts each record into the matching MongoDB collection, skipping records that already exist. Original TinyDB files are backed up to `backend/data/db/backups/<timestamp>/`.
#### Rolling Back
#### Rolling Back to TinyDB
To revert to TinyDB, simply set `USE_MONGODB=false` (or unset it). The original JSON files remain in place.
Set `USE_MONGODB=false`. The original JSON files remain in place.
#### Gunicorn / Docker
+1 -1
View File
@@ -25,7 +25,7 @@ except ImportError: # pragma: no cover - pymongo is a required dependency
ASCENDING = 1
USE_MONGODB = os.environ.get('USE_MONGODB', 'false').lower() == 'true'
USE_MONGODB = os.environ.get('USE_MONGODB', 'true').lower() == 'true'
# Resolve the MongoDB database name once at module load so runtime changes to
# DB_ENV/DATA_ENV in tests do not switch databases mid-process.
_mongo_db_name = get_mongo_db_name() if USE_MONGODB else None
+1 -1
View File
@@ -184,7 +184,7 @@ def main():
)
args = parser.parse_args()
if os.environ.get('USE_MONGODB', 'false').lower() != 'true':
if os.environ.get('USE_MONGODB', 'true').lower() != 'true':
print('Set USE_MONGODB=true to run the migration.', file=sys.stderr)
sys.exit(1)
+1 -1
View File
@@ -15,7 +15,7 @@ from db.mongo_client import get_mongo_client, get_mongo_db_name
# All tests in this module require the mongomock-backed MongoDB adapter.
pytestmark = pytest.mark.skipif(
os.environ.get('USE_MONGODB', 'false').lower() != 'true',
os.environ.get('USE_MONGODB', 'true').lower() != 'true',
reason='MongoDB adapter tests require USE_MONGODB=true',
)
+3 -3
View File
@@ -69,7 +69,7 @@
### 3. `LockedTable` MongoDB adapter
- Refactor `backend/db/db.py` so that each exported `*_db` object continues to expose the same methods: `search(...)`, `get(...)`, `all()`, `insert(...)`, `insert_multiple(...)`, `update(...)`, `remove(...)`, `truncate()`.
- Implement a new `MongoLockedTable` class that mirrors the existing `LockedTable` API but delegates to a MongoDB collection.
- Keep `LockedTable` (TinyDB) as a fallback selectable by `USE_MONGODB=true|false` (default `false`).
- Keep `LockedTable` (TinyDB) as a fallback selectable by `USE_MONGODB=true|false` (default `true`).
- Store the model `id` field as MongoDB `_id` on writes; do not keep a duplicate `id` field in storage.
- Restore the model `id` field from `_id` on reads and return TinyDB-compatible `Document` objects with `doc_id` set so existing callers that rely on `doc_id` continue to work.
- Support `update(fields, doc_ids=[...])` and callable updater functions (`update(lambda rec: ..., cond)`) to match TinyDB behavior.
@@ -98,9 +98,9 @@
```
### 6. Environment variables
- `MONGO_URI` — required when `USE_MONGODB=true`.
- `MONGO_URI` — required when `USE_MONGODB=true` (the default).
- `MONGO_DB_NAME` — optional database name; defaults parsed from `MONGO_URI` or falls back to `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV`/`DATA_ENV`.
- `USE_MONGODB` — `true` to use MongoDB backend; `false` to keep TinyDB.
- `USE_MONGODB` — `true` to use MongoDB backend (default); `false` to keep TinyDB.
### 7. Default data initialization
- `backend/db/default.py` (`initializeImages`, `createDefaultTasks`, `createDefaultRewards`) must work with the new MongoDB-backed `*_db` objects. No API changes.