feat: update default persistence to MongoDB and enhance configuration options
Chore App Build, Test, and Push Docker Images / build-and-push (push) Failing after 2m19s
Chore App Build, Test, and Push Docker Images / build-and-push (push) Failing after 2m19s
This commit is contained in:
@@ -8,7 +8,7 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
|
||||
- Activate venv: `source .venv/bin/activate`
|
||||
- Dev server: `python -m flask run --host=0.0.0.0 --port=5000` (entry: `main.py`)
|
||||
- Required env vars: `SECRET_KEY`, `REFRESH_TOKEN_EXPIRY_DAYS`, `DIGEST_TOKEN_SECRET`, `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY` — Flask raises `RuntimeError` on boot if any are missing
|
||||
- Optional persistence switch: `USE_MONGODB` (`true` | `false`). When `true`, set `MONGO_URI` and optionally `MONGO_DB_NAME`. Default `false` keeps TinyDB.
|
||||
- Optional persistence switch: `USE_MONGODB` (`true` | `false`). Defaults to `true`; set `MONGO_URI` (and optionally `MONGO_DB_NAME`). Set to `false` to use TinyDB instead.
|
||||
- Optional: `DB_ENV` / `DATA_ENV` (`prod` | `test` | `e2e`) — picks `data/` vs `test_data/` dir (see `config/paths.py`). For MongoDB these also select the default database name (`chore_db`, `chore_db_test`, `chore_db_e2e`) unless `MONGO_DB_NAME` is set.
|
||||
- Tests: `pytest tests/` — `conftest.py` forces `DB_ENV=test`, `USE_MONGODB=true`, `MONGO_URI=mongomock`, and sets dummy secrets. Single test: `pytest tests/test_routine_api.py::test_name`
|
||||
- Python imports assume `backend/` is on `sys.path` (set by `conftest.py` / `flask run` cwd). Run pytest from `backend/`.
|
||||
@@ -31,7 +31,7 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
|
||||
|
||||
### Models — strict 1:1 parity
|
||||
- Python `@dataclass`es in `backend/models/`. TypeScript interfaces in `frontend/src/common/models.ts`. Any model change requires updating both.
|
||||
- Persistence is TinyDB by default, or MongoDB when `USE_MONGODB=true`. Both are accessed through the `LockedTable` / `MongoLockedTable` wrappers in `backend/db/db.py`. Operate on model instances with `from_dict()` / `to_dict()` — never raw dicts.
|
||||
- Persistence is MongoDB by default (`USE_MONGODB=true`), or TinyDB when `USE_MONGODB=false`. Both are accessed through the `LockedTable` / `MongoLockedTable` wrappers in `backend/db/db.py`. Operate on model instances with `from_dict()` / `to_dict()` — never raw dicts.
|
||||
- MongoDB client initialization is lazy (`backend/db/mongo_client.py`). `backend/gunicorn.conf.py` provides the `post_fork` hook required for multi-worker Gunicorn deployments; `backend/Dockerfile` loads it with `-c gunicorn.conf.py`.
|
||||
- Migration script: `cd backend && python -m scripts/migrate_to_mongodb [--dry-run]`. It reads TinyDB JSON files and writes them to MongoDB idempotently, backing up the originals to `<db_dir>/backups/<timestamp>/`.
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ npm run dev
|
||||
| `ACCOUNT_DELETION_THRESHOLD_HOURS` | Hours to wait before deleting marked accounts | 720 (30 days) |
|
||||
| `DB_ENV` | Database environment (`prod` or `test`) | `prod` |
|
||||
| `DATA_ENV` | Data directory environment (`prod` or `test`) | `prod` |
|
||||
| `USE_MONGODB` | Use MongoDB instead of TinyDB (`true`/`false`) | `false` |
|
||||
| `USE_MONGODB` | Use MongoDB (`true`/`false`) | `true` |
|
||||
| `MONGO_URI` | MongoDB connection URI (required when `USE_MONGODB=true`) | — |
|
||||
| `MONGO_DB_NAME` | MongoDB database name (optional) | Parsed from `MONGO_URI`, or `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV` |
|
||||
|
||||
@@ -46,8 +46,8 @@ npm run dev
|
||||
|
||||
The application supports two persistence backends:
|
||||
|
||||
- **TinyDB** (default): JSON-file storage in `backend/data/db/` (or `backend/test_data/db/` for `test`/`e2e`). No extra configuration needed.
|
||||
- **MongoDB**: Set `USE_MONGODB=true` and provide `MONGO_URI`. Useful for production deployments and managed database hosting (e.g., MongoDB Atlas).
|
||||
- **MongoDB** (default): Set `MONGO_URI` (and optionally `MONGO_DB_NAME`). This is the recommended backend for production and managed hosting (e.g., MongoDB Atlas).
|
||||
- **TinyDB**: JSON-file storage in `backend/data/db/` (or `backend/test_data/db/` for `test`/`e2e`). Opt in by setting `USE_MONGODB=false`.
|
||||
|
||||
#### Migrating from TinyDB to MongoDB
|
||||
|
||||
@@ -62,9 +62,9 @@ python -m scripts.migrate_to_mongodb
|
||||
|
||||
The migration script reads the existing TinyDB JSON files and inserts each record into the matching MongoDB collection, skipping records that already exist. Original TinyDB files are backed up to `backend/data/db/backups/<timestamp>/`.
|
||||
|
||||
#### Rolling Back
|
||||
#### Rolling Back to TinyDB
|
||||
|
||||
To revert to TinyDB, simply set `USE_MONGODB=false` (or unset it). The original JSON files remain in place.
|
||||
Set `USE_MONGODB=false`. The original JSON files remain in place.
|
||||
|
||||
#### Gunicorn / Docker
|
||||
|
||||
|
||||
+1
-1
@@ -25,7 +25,7 @@ except ImportError: # pragma: no cover - pymongo is a required dependency
|
||||
ASCENDING = 1
|
||||
|
||||
|
||||
USE_MONGODB = os.environ.get('USE_MONGODB', 'false').lower() == 'true'
|
||||
USE_MONGODB = os.environ.get('USE_MONGODB', 'true').lower() == 'true'
|
||||
# Resolve the MongoDB database name once at module load so runtime changes to
|
||||
# DB_ENV/DATA_ENV in tests do not switch databases mid-process.
|
||||
_mongo_db_name = get_mongo_db_name() if USE_MONGODB else None
|
||||
|
||||
@@ -184,7 +184,7 @@ def main():
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if os.environ.get('USE_MONGODB', 'false').lower() != 'true':
|
||||
if os.environ.get('USE_MONGODB', 'true').lower() != 'true':
|
||||
print('Set USE_MONGODB=true to run the migration.', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ from db.mongo_client import get_mongo_client, get_mongo_db_name
|
||||
|
||||
# All tests in this module require the mongomock-backed MongoDB adapter.
|
||||
pytestmark = pytest.mark.skipif(
|
||||
os.environ.get('USE_MONGODB', 'false').lower() != 'true',
|
||||
os.environ.get('USE_MONGODB', 'true').lower() != 'true',
|
||||
reason='MongoDB adapter tests require USE_MONGODB=true',
|
||||
)
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@
|
||||
### 3. `LockedTable` MongoDB adapter
|
||||
- Refactor `backend/db/db.py` so that each exported `*_db` object continues to expose the same methods: `search(...)`, `get(...)`, `all()`, `insert(...)`, `insert_multiple(...)`, `update(...)`, `remove(...)`, `truncate()`.
|
||||
- Implement a new `MongoLockedTable` class that mirrors the existing `LockedTable` API but delegates to a MongoDB collection.
|
||||
- Keep `LockedTable` (TinyDB) as a fallback selectable by `USE_MONGODB=true|false` (default `false`).
|
||||
- Keep `LockedTable` (TinyDB) as a fallback selectable by `USE_MONGODB=true|false` (default `true`).
|
||||
- Store the model `id` field as MongoDB `_id` on writes; do not keep a duplicate `id` field in storage.
|
||||
- Restore the model `id` field from `_id` on reads and return TinyDB-compatible `Document` objects with `doc_id` set so existing callers that rely on `doc_id` continue to work.
|
||||
- Support `update(fields, doc_ids=[...])` and callable updater functions (`update(lambda rec: ..., cond)`) to match TinyDB behavior.
|
||||
@@ -98,9 +98,9 @@
|
||||
```
|
||||
|
||||
### 6. Environment variables
|
||||
- `MONGO_URI` — required when `USE_MONGODB=true`.
|
||||
- `MONGO_URI` — required when `USE_MONGODB=true` (the default).
|
||||
- `MONGO_DB_NAME` — optional database name; defaults parsed from `MONGO_URI` or falls back to `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV`/`DATA_ENV`.
|
||||
- `USE_MONGODB` — `true` to use MongoDB backend; `false` to keep TinyDB.
|
||||
- `USE_MONGODB` — `true` to use MongoDB backend (default); `false` to keep TinyDB.
|
||||
|
||||
### 7. Default data initialization
|
||||
- `backend/db/default.py` (`initializeImages`, `createDefaultTasks`, `createDefaultRewards`) must work with the new MongoDB-backed `*_db` objects. No API changes.
|
||||
|
||||
Reference in New Issue
Block a user