feat: migrate backend persistence from TinyDB to MongoDB Atlas
- Implement lazy MongoDB client initialization in backend/db/mongo_client.py. - Create Gunicorn configuration to ensure MongoDB client is initialized per worker. - Refactor database access layer to support MongoDB with a new MongoLockedTable adapter. - Add migration script to transfer existing TinyDB data to MongoDB, preserving idempotency. - Update tracking event handling to ensure deterministic ordering with a monotonic sequence. - Modify tests to use mongomock for MongoDB integration and ensure existing tests pass. - Add integration test script to run tests against a local MongoDB Docker container. - Document environment variables and migration process in specs/feat-database-migration.md.
This commit is contained in:
@@ -4,7 +4,7 @@ A family-friendly application for managing chores, tasks, and rewards for childr
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
- **Backend**: Flask (Python) with TinyDB for data persistence
|
||||
- **Backend**: Flask (Python) with TinyDB or MongoDB for data persistence
|
||||
- **Frontend**: Vue 3 (TypeScript) with real-time SSE updates
|
||||
- **Deployment**: Docker with nginx reverse proxy
|
||||
|
||||
@@ -38,6 +38,37 @@ npm run dev
|
||||
| `ACCOUNT_DELETION_THRESHOLD_HOURS` | Hours to wait before deleting marked accounts | 720 (30 days) |
|
||||
| `DB_ENV` | Database environment (`prod` or `test`) | `prod` |
|
||||
| `DATA_ENV` | Data directory environment (`prod` or `test`) | `prod` |
|
||||
| `USE_MONGODB` | Use MongoDB instead of TinyDB (`true`/`false`) | `false` |
|
||||
| `MONGO_URI` | MongoDB connection URI (required when `USE_MONGODB=true`) | — |
|
||||
| `MONGO_DB_NAME` | MongoDB database name (optional) | Parsed from `MONGO_URI`, or `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV` |
|
||||
|
||||
### Database Backend
|
||||
|
||||
The application supports two persistence backends:
|
||||
|
||||
- **TinyDB** (default): JSON-file storage in `backend/data/db/` (or `backend/test_data/db/` for `test`/`e2e`). No extra configuration needed.
|
||||
- **MongoDB**: Set `USE_MONGODB=true` and provide `MONGO_URI`. Useful for production deployments and managed database hosting (e.g., MongoDB Atlas).
|
||||
|
||||
#### Migrating from TinyDB to MongoDB
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
# Dry run to preview what will be migrated
|
||||
python -m scripts.migrate_to_mongodb --dry-run
|
||||
|
||||
# Run the migration (backs up TinyDB files first)
|
||||
python -m scripts.migrate_to_mongodb
|
||||
```
|
||||
|
||||
The migration script reads the existing TinyDB JSON files and inserts each record into the matching MongoDB collection, skipping records that already exist. Original TinyDB files are backed up to `backend/data/db/backups/<timestamp>/`.
|
||||
|
||||
#### Rolling Back
|
||||
|
||||
To revert to TinyDB, simply set `USE_MONGODB=false` (or unset it). The original JSON files remain in place.
|
||||
|
||||
#### Gunicorn / Docker
|
||||
|
||||
When running multiple Gunicorn workers, each worker must create its own MongoDB client after forking. This is handled automatically by `backend/gunicorn.conf.py`, which is loaded by `backend/Dockerfile` via `-c gunicorn.conf.py`.
|
||||
|
||||
### Account Deletion Scheduler
|
||||
|
||||
@@ -145,7 +176,7 @@ npm run test
|
||||
├── backend/
|
||||
│ ├── api/ # REST API endpoints
|
||||
│ ├── config/ # Configuration files
|
||||
│ ├── db/ # TinyDB setup
|
||||
│ ├── db/ # TinyDB / MongoDB persistence layer
|
||||
│ ├── events/ # SSE event system
|
||||
│ ├── models/ # Data models
|
||||
│ ├── tests/ # Backend tests
|
||||
|
||||
Reference in New Issue
Block a user