feat: migrate backend persistence from TinyDB to MongoDB Atlas

- Implement lazy MongoDB client initialization in backend/db/mongo_client.py.
- Create Gunicorn configuration to ensure MongoDB client is initialized per worker.
- Refactor database access layer to support MongoDB with a new MongoLockedTable adapter.
- Add migration script to transfer existing TinyDB data to MongoDB, preserving idempotency.
- Update tracking event handling to ensure deterministic ordering with a monotonic sequence.
- Modify tests to use mongomock for MongoDB integration and ensure existing tests pass.
- Add integration test script to run tests against a local MongoDB Docker container.
- Document environment variables and migration process in specs/feat-database-migration.md.

AGENTS.md

@@ -8,8 +8,9 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
 - Activate venv: `source .venv/bin/activate`
 - Dev server: `python -m flask run --host=0.0.0.0 --port=5000` (entry: `main.py`)
 - Required env vars: `SECRET_KEY`, `REFRESH_TOKEN_EXPIRY_DAYS`, `DIGEST_TOKEN_SECRET`, `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY` — Flask raises `RuntimeError` on boot if any are missing
-- Optional: `DB_ENV` / `DATA_ENV` (`prod` | `test` | `e2e`) — picks `data/` vs `test_data/` dir (see `config/paths.py`)
+- Optional persistence switch: `USE_MONGODB` (`true` | `false`). When `true`, set `MONGO_URI` and optionally `MONGO_DB_NAME`. Default `false` keeps TinyDB.
-- Tests: `pytest tests/` — `conftest.py` forces `DB_ENV=test` and sets dummy secrets. Single test: `pytest tests/test_routine_api.py::test_name`
+- Optional: `DB_ENV` / `DATA_ENV` (`prod` | `test` | `e2e`) — picks `data/` vs `test_data/` dir (see `config/paths.py`). For MongoDB these also select the default database name (`chore_db`, `chore_db_test`, `chore_db_e2e`) unless `MONGO_DB_NAME` is set.
+- Tests: `pytest tests/` — `conftest.py` forces `DB_ENV=test`, `USE_MONGODB=true`, `MONGO_URI=mongomock`, and sets dummy secrets. Single test: `pytest tests/test_routine_api.py::test_name`
 - Python imports assume `backend/` is on `sys.path` (set by `conftest.py` / `flask run` cwd). Run pytest from `backend/`.
 - Create admin user: `python scripts/create_admin.py` (admin role cannot be set via signup)
 
@@ -18,7 +19,7 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
 - Lint: `npm run lint`
 - Type-check: `npm run type-check`
 - Unit tests: `npm run test:unit` (Vitest). Single: `npx vitest run path/to/file.spec.ts`
-- E2E: `npx playwright test` — config auto-starts both `npm run dev` and the Flask backend with `DB_ENV=e2e DATA_ENV=e2e`. Tests live in `e2e/`.
+- E2E: `npx playwright test` — config auto-starts both `npm run dev` and the Flask backend with `DB_ENV=e2e DATA_ENV=e2e USE_MONGODB=true MONGO_URI=mongomock`. Tests live in `e2e/`. `frontend/.env.test` contains the example MongoDB config.
 - E2E buckets are Playwright projects (see `playwright.config.ts`) targeting directories under `e2e/mode_parent/`
 
 ## Architecture
@@ -30,7 +31,9 @@ Family chore/reward manager. Flask + TinyDB backend (`backend/`), Vue 3 + TypeSc
 
 ### Models — strict 1:1 parity
 - Python `@dataclass`es in `backend/models/`. TypeScript interfaces in `frontend/src/common/models.ts`. Any model change requires updating both.
-- Persistence is TinyDB via the thread-safe `LockedTable` wrapper (`backend/db/db.py`). Operate on model instances with `from_dict()` / `to_dict()` — never raw dicts.
+- Persistence is TinyDB by default, or MongoDB when `USE_MONGODB=true`. Both are accessed through the `LockedTable` / `MongoLockedTable` wrappers in `backend/db/db.py`. Operate on model instances with `from_dict()` / `to_dict()` — never raw dicts.
+- MongoDB client initialization is lazy (`backend/db/mongo_client.py`). `backend/gunicorn.conf.py` provides the `post_fork` hook required for multi-worker Gunicorn deployments; `backend/Dockerfile` loads it with `-c gunicorn.conf.py`.
+- Migration script: `cd backend && python -m scripts/migrate_to_mongodb [--dry-run]`. It reads TinyDB JSON files and writes them to MongoDB idempotently, backing up the originals to `<db_dir>/backups/<timestamp>/`.
 
 ### SSE event bus — mandatory for every mutation
 - Every backend mutation (add/edit/delete/trigger) **must** call `send_event_for_current_user` from `api/utils.py`. Event types in `backend/events/types/` are mirrored in `frontend/src/common/backendEvents.ts`.

README.md

@@ -4,7 +4,7 @@ A family-friendly application for managing chores, tasks, and rewards for childr
 
 ## 🏗️ Architecture
 
-- **Backend**: Flask (Python) with TinyDB for data persistence
+- **Backend**: Flask (Python) with TinyDB or MongoDB for data persistence
 - **Frontend**: Vue 3 (TypeScript) with real-time SSE updates
 - **Deployment**: Docker with nginx reverse proxy
 
@@ -38,6 +38,37 @@ npm run dev
 | `ACCOUNT_DELETION_THRESHOLD_HOURS` | Hours to wait before deleting marked accounts | 720 (30 days) |
 | `DB_ENV`                           | Database environment (`prod` or `test`)       | `prod`        |
 | `DATA_ENV`                         | Data directory environment (`prod` or `test`) | `prod`        |
+| `USE_MONGODB`                      | Use MongoDB instead of TinyDB (`true`/`false`) | `false`       |
+| `MONGO_URI`                        | MongoDB connection URI (required when `USE_MONGODB=true`) | —             |
+| `MONGO_DB_NAME`                    | MongoDB database name (optional)              | Parsed from `MONGO_URI`, or `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV` |
+
+### Database Backend
+
+The application supports two persistence backends:
+
+- **TinyDB** (default): JSON-file storage in `backend/data/db/` (or `backend/test_data/db/` for `test`/`e2e`). No extra configuration needed.
+- **MongoDB**: Set `USE_MONGODB=true` and provide `MONGO_URI`. Useful for production deployments and managed database hosting (e.g., MongoDB Atlas).
+
+#### Migrating from TinyDB to MongoDB
+
+```bash
+cd backend
+# Dry run to preview what will be migrated
+python -m scripts.migrate_to_mongodb --dry-run
+
+# Run the migration (backs up TinyDB files first)
+python -m scripts.migrate_to_mongodb
+```
+
+The migration script reads the existing TinyDB JSON files and inserts each record into the matching MongoDB collection, skipping records that already exist. Original TinyDB files are backed up to `backend/data/db/backups/<timestamp>/`.
+
+#### Rolling Back
+
+To revert to TinyDB, simply set `USE_MONGODB=false` (or unset it). The original JSON files remain in place.
+
+#### Gunicorn / Docker
+
+When running multiple Gunicorn workers, each worker must create its own MongoDB client after forking. This is handled automatically by `backend/gunicorn.conf.py`, which is loaded by `backend/Dockerfile` via `-c gunicorn.conf.py`.
 
 ### Account Deletion Scheduler
 
@@ -145,7 +176,7 @@ npm run test
 ├── backend/
 │   ├── api/              # REST API endpoints
 │   ├── config/           # Configuration files
-│   ├── db/               # TinyDB setup
+│   ├── db/               # TinyDB / MongoDB persistence layer
 │   ├── events/           # SSE event system
 │   ├── models/           # Data models
 │   ├── tests/            # Backend tests

backend/Dockerfile

@@ -15,4 +15,4 @@ ENV PYTHONIOENCODING=utf-8
 VOLUME ["/app/data"]
 
 # Use Gunicorn instead of python main.py
-CMD ["gunicorn", "--bind", "0.0.0.0:5000", "-k", "gevent", "--workers", "1", "--timeout", "120", "--access-logfile", "-", "--error-logfile", "-", "--log-level", "info", "main:app"]
+CMD ["gunicorn", "--bind", "0.0.0.0:5000", "-k", "gevent", "--workers", "1", "--timeout", "120", "--access-logfile", "-", "--error-logfile", "-", "--log-level", "info", "-c", "gunicorn.conf.py", "main:app"]

backend/api/auth_api.py

@@ -29,6 +29,8 @@ from db.db import (
     users_db, refresh_tokens_db, child_db, task_db, reward_db, image_db,
     pending_reward_db, pending_confirmations_db, tracking_events_db,
     child_overrides_db, chore_schedules_db, task_extensions_db,
+    routine_db, routine_items_db, routine_schedules_db, routine_extensions_db,
+    push_subscriptions_db, digest_action_tokens_db,
 )
 from db.default import initializeImages, createDefaultTasks, createDefaultRewards
 from api.utils import normalize_email
@@ -623,6 +625,12 @@ def e2e_seed():
     chore_schedules_db.truncate()
     task_extensions_db.truncate()
     refresh_tokens_db.truncate()
+    routine_db.truncate()
+    routine_items_db.truncate()
+    routine_schedules_db.truncate()
+    routine_extensions_db.truncate()
+    push_subscriptions_db.truncate()
+    digest_action_tokens_db.truncate()
 
     # Recreate only baseline defaults for e2e runs.
     initializeImages()

backend/db/db.py

@@ -1,13 +1,42 @@
 # python
 import os
-from config.paths import get_database_dir
 import threading
+from config.paths import get_database_dir
 from tinydb import TinyDB
+from tinydb.queries import QueryInstance
 
+from db.mongo_client import get_mongo_client, get_mongo_db_name
+
+try:
+    from tinydb.table import Document
+except ImportError:  # pragma: no cover - tinydb version compatibility
+    from tinydb.database import Document
+
+
+def _stable_clause_key(clause: dict) -> str:
+    """Return a stable string key for sorting MongoDB filter clauses."""
+    import json
+    return json.dumps(clause, sort_keys=True, default=str)
+
+
+try:
+    from pymongo import ASCENDING
+except ImportError:  # pragma: no cover - pymongo is a required dependency
+    ASCENDING = 1
+
+
+USE_MONGODB = os.environ.get('USE_MONGODB', 'false').lower() == 'true'
+# Resolve the MongoDB database name once at module load so runtime changes to
+# DB_ENV/DATA_ENV in tests do not switch databases mid-process.
+_mongo_db_name = get_mongo_db_name() if USE_MONGODB else None
 base_dir = get_database_dir()
 os.makedirs(base_dir, exist_ok=True)
 
 
+# ---------------------------------------------------------------------------
+# TinyDB-backed table wrapper
+# ---------------------------------------------------------------------------
+
 class LockedTable:
     """
     Thread-safe wrapper around a TinyDB table. All callable attribute access
@@ -65,6 +94,361 @@ class LockedTable:
         with self._lock:
             return self._table.truncate()
 
+    def close(self):
+        with self._lock:
+            return self._table.close()
+
+
+# ---------------------------------------------------------------------------
+# TinyDB Query -> MongoDB filter translator
+# ---------------------------------------------------------------------------
+
+_MONGO_OP_MAP = {
+    '==': '$eq',
+    '!=': '$ne',
+    '<': '$lt',
+    '<=': '$lte',
+    '>': '$gt',
+    '>=': '$gte',
+}
+
+_NEGATED_OPS = {
+    '$eq': '$ne',
+    '$ne': '$eq',
+    '$lt': '$gte',
+    '$lte': '$gt',
+    '$gt': '$lte',
+    '$gte': '$lt',
+}
+
+
+def _field_to_mongo(field_path: tuple) -> str:
+    """Map a TinyDB field path to a MongoDB field name.
+
+    The model ``id`` field is stored as the MongoDB ``_id`` field, so queries
+    on ``id`` are translated to queries on ``_id``.
+    """
+    if len(field_path) == 1:
+        return '_id' if field_path[0] == 'id' else field_path[0]
+    return '.'.join('_id' if p == 'id' else p for p in field_path)
+
+
+def _negate_condition(cond: dict) -> dict | None:
+    """Return a MongoDB condition that negates a single-field condition."""
+    if len(cond) != 1:
+        return None
+    field, inner = next(iter(cond.items()))
+    if not isinstance(inner, dict) or len(inner) != 1:
+        return None
+    op, value = next(iter(inner.items()))
+    if op in _NEGATED_OPS:
+        return {field: {_NEGATED_OPS[op]: value}}
+    return None
+
+
+def _hash_to_mongo_filter(query_hash) -> dict | None:
+    """Translate a TinyDB query hash tuple to a MongoDB filter document.
+
+    Returns ``None`` when the query construct cannot be expressed as a native
+    MongoDB filter, signalling that the caller should fall back to in-memory
+    TinyDB evaluation.
+    """
+    if not isinstance(query_hash, tuple) or len(query_hash) == 0:
+        return None
+
+    op = query_hash[0]
+
+    if op in _MONGO_OP_MAP:
+        field = _field_to_mongo(query_hash[1])
+        value = query_hash[2]
+        return {field: {_MONGO_OP_MAP[op]: value}}
+
+    if op == 'exists':
+        field = _field_to_mongo(query_hash[1])
+        return {field: {'$exists': True}}
+
+    if op == 'one_of':
+        field = _field_to_mongo(query_hash[1])
+        return {field: {'$in': list(query_hash[2])}}
+
+    if op == 'any':
+        field = _field_to_mongo(query_hash[1])
+        return {field: {'$in': list(query_hash[2])}}
+
+    if op == 'all':
+        field = _field_to_mongo(query_hash[1])
+        return {field: {'$all': list(query_hash[2])}}
+
+    if op == 'matches':
+        field = _field_to_mongo(query_hash[1])
+        # TinyDB matches() anchors the regex at the start of the string.
+        return {field: {'$regex': f'^{query_hash[2]}'}}
+
+    if op == 'search':
+        field = _field_to_mongo(query_hash[1])
+        return {field: {'$regex': query_hash[2]}}
+
+    if op == 'and':
+        merged: dict = {}
+        for sub_hash in query_hash[1]:
+            sub = _hash_to_mongo_filter(sub_hash)
+            if sub is None:
+                return None
+            for field, inner in sub.items():
+                if field in merged:
+                    if isinstance(merged[field], dict) and isinstance(inner, dict):
+                        merged[field].update(inner)
+                    elif isinstance(merged[field], list) and isinstance(inner, list):
+                        merged[field].extend(inner)
+                    else:
+                        return None
+                else:
+                    merged[field] = (
+                        dict(inner) if isinstance(inner, dict)
+                        else list(inner) if isinstance(inner, list)
+                        else inner
+                    )
+        return merged
+
+    if op == 'or':
+        clauses = [_hash_to_mongo_filter(sub_hash) for sub_hash in query_hash[1]]
+        if any(c is None for c in clauses):
+            return None
+        return {'$or': sorted(clauses, key=_stable_clause_key)}
+
+    if op == 'not':
+        inner = _hash_to_mongo_filter(query_hash[1])
+        if inner is None:
+            return None
+        negated = _negate_condition(inner)
+        if negated is not None:
+            return negated
+        return None
+
+    # Unsupported operation (test, fragment, noop, etc.) -> fall back.
+    return None
+
+
+def _query_to_mongo_filter(query) -> dict | None:
+    """Translate a TinyDB QueryInstance to a MongoDB filter, if possible."""
+    if isinstance(query, dict):
+        return query
+    if isinstance(query, QueryInstance):
+        return _hash_to_mongo_filter(query._hash)
+    return None
+
+
+def _evaluate_in_memory(docs, query) -> list:
+    """Evaluate a TinyDB query against an in-memory list of documents."""
+    if callable(query):
+        return [doc for doc in docs if query(doc)]
+    return docs
+
+
+# ---------------------------------------------------------------------------
+# MongoDB-backed table wrapper
+# ---------------------------------------------------------------------------
+
+class MongoLockedTable:
+    """Drop-in replacement for ``LockedTable`` that delegates to MongoDB.
+
+    The adapter preserves the existing table API while mapping the model
+    ``id`` field to MongoDB's ``_id`` field on reads and writes.
+    """
+
+    def __init__(self, collection_name: str):
+        self.collection_name = collection_name
+
+    def _collection(self):
+        client = get_mongo_client()
+        return client[_mongo_db_name][self.collection_name]
+
+    @staticmethod
+    def _doc_to_mongo(doc: dict) -> dict:
+        """Store a copy of ``doc`` with ``id`` promoted to MongoDB ``_id``.
+
+        This avoids storing both ``_id`` and ``id`` with identical values.
+        The original ``id`` field is removed from the stored document.
+        """
+        if doc is None:
+            return None
+        d = dict(doc)
+        if 'id' in d:
+            d['_id'] = d.pop('id')
+        return d
+
+    @staticmethod
+    def _doc_from_mongo(doc: dict):
+        """Return a TinyDB-compatible Document with ``doc_id`` set to ``_id``.
+
+        Restores the model ``id`` field from MongoDB's ``_id`` and exposes
+        TinyDB's ``doc_id`` attribute so callers that rely on it continue to
+        work.
+        """
+        if doc is None:
+            return None
+        d = dict(doc)
+        doc_id = d.pop('_id', None)
+        if doc_id is not None:
+            d['id'] = doc_id
+        return Document(d, doc_id=doc_id)
+
+    def _mongo_filter(self, cond):
+        """Translate a TinyDB query or dict to a MongoDB filter."""
+        return _query_to_mongo_filter(cond)
+
+    def all(self):
+        return [self._doc_from_mongo(doc) for doc in self._collection().find({})]
+
+    def search(self, cond):
+        mongo_filter = self._mongo_filter(cond)
+        if mongo_filter is not None:
+            cursor = self._collection().find(mongo_filter)
+            return [self._doc_from_mongo(doc) for doc in cursor]
+
+        # Fallback: fetch all and evaluate the TinyDB query in Python.
+        docs = list(self._collection().find({}))
+        matched = _evaluate_in_memory(
+            [self._doc_from_mongo(doc) for doc in docs], cond
+        )
+        return matched
+
+    def get(self, cond):
+        mongo_filter = self._mongo_filter(cond)
+        if mongo_filter is not None:
+            doc = self._collection().find_one(mongo_filter)
+            return self._doc_from_mongo(doc)
+
+        docs = list(self._collection().find({}))
+        for doc in docs:
+            d = self._doc_from_mongo(doc)
+            if callable(cond) and cond(d):
+                return d
+        return None
+
+    def insert(self, document: dict):
+        doc = self._doc_to_mongo(document)
+        result = self._collection().insert_one(doc)
+        return str(result.inserted_id)
+
+    def insert_multiple(self, documents: list):
+        if not documents:
+            return []
+        docs = [self._doc_to_mongo(d) for d in documents]
+        result = self._collection().insert_many(docs)
+        return [str(iid) for iid in result.inserted_ids]
+
+    def update(self, fields, cond=None, doc_ids=None):
+        is_callable = callable(fields)
+
+        if doc_ids is not None:
+            mongo_filter = {'_id': {'$in': list(doc_ids)}}
+            target_ids = [str(did) for did in doc_ids]
+            if not target_ids:
+                return []
+
+            if is_callable:
+                # Fetch, apply callable in-memory, and replace each document.
+                updated_ids = []
+                for doc in self._collection().find(mongo_filter):
+                    d = self._doc_from_mongo(doc)
+                    fields(d)
+                    new_doc = self._doc_to_mongo(d)
+                    new_doc.pop('_id', None)
+                    self._collection().update_one(
+                        {'_id': doc['_id']}, {'$set': new_doc}
+                    )
+                    updated_ids.append(str(doc['_id']))
+                return updated_ids
+
+            update_doc = self._doc_to_mongo(fields) or {}
+            update_doc.pop('_id', None)
+            update_doc.pop('id', None)
+            if update_doc:
+                self._collection().update_many(mongo_filter, {'$set': update_doc})
+            return target_ids
+
+        mongo_filter = self._mongo_filter(cond)
+        if mongo_filter is not None and not is_callable:
+            update_doc = self._doc_to_mongo(fields) or {}
+            update_doc.pop('_id', None)
+            update_doc.pop('id', None)
+            target_ids = [
+                str(doc['_id'])
+                for doc in self._collection().find(mongo_filter, {'_id': 1})
+            ]
+            if target_ids and update_doc:
+                self._collection().update_many(
+                    mongo_filter, {'$set': update_doc}
+                )
+            return target_ids
+
+        # Fallback: evaluate the query in-memory and update one at a time.
+        docs = list(self._collection().find({}))
+        updated_ids = []
+        for doc in docs:
+            d = self._doc_from_mongo(doc)
+            match = cond(d) if callable(cond) else (mongo_filter is not None)
+            if not match:
+                continue
+            if is_callable:
+                fields(d)
+                new_doc = self._doc_to_mongo(d)
+                new_doc.pop('_id', None)
+                self._collection().update_one(
+                    {'_id': doc['_id']}, {'$set': new_doc}
+                )
+            else:
+                update_doc = self._doc_to_mongo(fields) or {}
+                update_doc.pop('_id', None)
+                update_doc.pop('id', None)
+                if update_doc:
+                    self._collection().update_one(
+                        {'_id': doc['_id']}, {'$set': update_doc}
+                    )
+            updated_ids.append(str(doc['_id']))
+        return updated_ids
+
+    def remove(self, cond):
+        mongo_filter = self._mongo_filter(cond)
+        if mongo_filter is not None:
+            target_ids = [
+                str(doc['_id'])
+                for doc in self._collection().find(mongo_filter, {'_id': 1})
+            ]
+            if target_ids:
+                self._collection().delete_many(mongo_filter)
+            return target_ids
+
+        # Fallback: evaluate the query in-memory and delete one at a time.
+        docs = list(self._collection().find({}))
+        removed_ids = []
+        for doc in docs:
+            d = self._doc_from_mongo(doc)
+            if callable(cond) and cond(d):
+                self._collection().delete_one({'_id': doc['_id']})
+                removed_ids.append(str(doc['_id']))
+        return removed_ids
+
+    def truncate(self):
+        self._collection().delete_many({})
+
+    def close(self):
+        # MongoDB clients are shared and long-lived; nothing to close here.
+        pass
+
+
+# ---------------------------------------------------------------------------
+# Collection factory
+# ---------------------------------------------------------------------------
+
+def _make_table(json_path: str, collection_name: str):
+    if USE_MONGODB:
+        return MongoLockedTable(collection_name)
+    db = TinyDB(json_path, indent=2)
+    return LockedTable(db)
+
+
 # Setup DB files next to this module
 
 child_path = os.path.join(base_dir, 'children.json')
@@ -86,46 +470,116 @@ refresh_tokens_path = os.path.join(base_dir, 'refresh_tokens.json')
 push_subscriptions_path = os.path.join(base_dir, 'push_subscriptions.json')
 digest_action_tokens_path = os.path.join(base_dir, 'digest_action_tokens.json')
 
-# Use separate TinyDB instances/files for each collection
+# Expose table objects backed by TinyDB or MongoDB based on USE_MONGODB
-_child_db = TinyDB(child_path, indent=2)
+child_db = _make_table(child_path, 'children')
-_task_db = TinyDB(task_path, indent=2)
+task_db = _make_table(task_path, 'tasks')
-_routine_db = TinyDB(routine_path, indent=2)
+routine_db = _make_table(routine_path, 'routines')
-_routine_items_db = TinyDB(routine_items_path, indent=2)
+routine_items_db = _make_table(routine_items_path, 'routine_items')
-_routine_schedules_db = TinyDB(routine_schedules_path, indent=2)
+routine_schedules_db = _make_table(routine_schedules_path, 'routine_schedules')
-_routine_extensions_db = TinyDB(routine_extensions_path, indent=2)
+routine_extensions_db = _make_table(routine_extensions_path, 'routine_extensions')
-_reward_db = TinyDB(reward_path, indent=2)
+reward_db = _make_table(reward_path, 'rewards')
-_image_db = TinyDB(image_path, indent=2)
+image_db = _make_table(image_path, 'images')
-_pending_rewards_db = TinyDB(pending_reward_path, indent=2)
+pending_reward_db = _make_table(pending_reward_path, 'pending_rewards')
-_pending_confirmations_db = TinyDB(pending_confirmations_path, indent=2)
+pending_confirmations_db = _make_table(pending_confirmations_path, 'pending_confirmations')
-_users_db = TinyDB(users_path, indent=2)
+users_db = _make_table(users_path, 'users')
-_tracking_events_db = TinyDB(tracking_events_path, indent=2)
+tracking_events_db = _make_table(tracking_events_path, 'tracking_events')
-_child_overrides_db = TinyDB(child_overrides_path, indent=2)
+child_overrides_db = _make_table(child_overrides_path, 'child_overrides')
-_chore_schedules_db = TinyDB(chore_schedules_path, indent=2)
+chore_schedules_db = _make_table(chore_schedules_path, 'chore_schedules')
-_task_extensions_db = TinyDB(task_extensions_path, indent=2)
+task_extensions_db = _make_table(task_extensions_path, 'task_extensions')
-_refresh_tokens_db = TinyDB(refresh_tokens_path, indent=2)
+refresh_tokens_db = _make_table(refresh_tokens_path, 'refresh_tokens')
-_push_subscriptions_db = TinyDB(push_subscriptions_path, indent=2)
+push_subscriptions_db = _make_table(push_subscriptions_path, 'push_subscriptions')
-_digest_action_tokens_db = TinyDB(digest_action_tokens_path, indent=2)
+digest_action_tokens_db = _make_table(digest_action_tokens_path, 'digest_action_tokens')
 
-# Expose table objects wrapped with locking
-child_db = LockedTable(_child_db)
-task_db = LockedTable(_task_db)
-routine_db = LockedTable(_routine_db)
-routine_items_db = LockedTable(_routine_items_db)
-routine_schedules_db = LockedTable(_routine_schedules_db)
-routine_extensions_db = LockedTable(_routine_extensions_db)
-reward_db = LockedTable(_reward_db)
-image_db = LockedTable(_image_db)
-pending_reward_db = LockedTable(_pending_rewards_db)
-pending_confirmations_db = LockedTable(_pending_confirmations_db)
-users_db = LockedTable(_users_db)
-tracking_events_db = LockedTable(_tracking_events_db)
-child_overrides_db = LockedTable(_child_overrides_db)
-chore_schedules_db = LockedTable(_chore_schedules_db)
-task_extensions_db = LockedTable(_task_extensions_db)
-refresh_tokens_db = LockedTable(_refresh_tokens_db)
-push_subscriptions_db = LockedTable(_push_subscriptions_db)
-digest_action_tokens_db = LockedTable(_digest_action_tokens_db)
 
+# ---------------------------------------------------------------------------
+# Index management
+# ---------------------------------------------------------------------------
+
+COLLECTION_INDEXES = {
+    # NOTE: The model ``id`` field is stored as MongoDB's primary key ``_id``,
+    # so no separate unique index on ``id`` is needed.  Only secondary indexes
+    # for frequently queried fields are defined here.
+    'children': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'tasks': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'routines': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'routine_items': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'routine_schedules': [],
+    'routine_extensions': [],
+    'rewards': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'images': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'pending_rewards': [
+        {'keys': [('child_id', ASCENDING)]},
+    ],
+    'pending_confirmations': [
+        {'keys': [('user_id', ASCENDING)]},
+        {'keys': [('child_id', ASCENDING)]},
+        {'keys': [('entity_id', ASCENDING), ('entity_type', ASCENDING)]},
+    ],
+    'users': [],
+    'tracking_events': [
+        {'keys': [('user_id', ASCENDING)]},
+        {'keys': [('child_id', ASCENDING)]},
+        {'keys': [('entity_id', ASCENDING), ('entity_type', ASCENDING)]},
+    ],
+    'child_overrides': [
+        {'keys': [('child_id', ASCENDING)]},
+        {'keys': [('entity_id', ASCENDING), ('entity_type', ASCENDING)]},
+    ],
+    'chore_schedules': [
+        {'keys': [('user_id', ASCENDING)]},
+        {'keys': [('child_id', ASCENDING)]},
+    ],
+    'task_extensions': [
+        {'keys': [('user_id', ASCENDING)]},
+        {'keys': [('child_id', ASCENDING)]},
+    ],
+    'refresh_tokens': [
+        {'keys': [('user_id', ASCENDING)]},
+        {'keys': [('token', ASCENDING)], 'unique': True, 'sparse': True},
+    ],
+    'push_subscriptions': [
+        {'keys': [('user_id', ASCENDING)]},
+    ],
+    'digest_action_tokens': [
+        {'keys': [('user_id', ASCENDING)]},
+        {'keys': [('token', ASCENDING)], 'unique': True, 'sparse': True},
+    ],
+}
+
+
+def ensure_mongodb_indexes(client=None, db_name=None):
+    """Create required indexes on all MongoDB collections.
+
+    Safe to call repeatedly: MongoDB treats index creation as idempotent.
+    """
+    if not USE_MONGODB:
+        return
+
+    client = client or get_mongo_client()
+    db_name = db_name or _mongo_db_name
+    db = client[db_name]
+
+    for collection_name, indexes in COLLECTION_INDEXES.items():
+        coll = db[collection_name]
+        for spec in indexes:
+            keys = spec['keys']
+            kwargs = {k: v for k, v in spec.items() if k != 'keys'}
+            coll.create_index(keys, **kwargs)
+
+
+# Clear test collections at import time so tests start with a clean slate.
 if os.environ.get('DB_ENV', 'prod') == 'test':
     child_db.truncate()
     task_db.truncate()
@@ -145,4 +599,3 @@ if os.environ.get('DB_ENV', 'prod') == 'test':
     refresh_tokens_db.truncate()
     push_subscriptions_db.truncate()
     digest_action_tokens_db.truncate()
-

backend/db/mongo_client.py

@@ -0,0 +1,113 @@
+# python
+"""Lazy MongoDB client factory and database-name helpers.
+
+The client is intentionally **not** created at module import.  Use
+``get_mongo_client()`` to obtain a cached singleton.  For Gunicorn multi-worker
+deployments call ``init_mongo_client()`` from a ``post_fork`` hook so each
+worker process owns its own connection pool rather than inheriting the parent
+process's client.
+"""
+import os
+import threading
+from urllib.parse import urlparse
+
+from pymongo import MongoClient
+from pymongo.uri_parser import parse_uri
+
+
+_mongo_client_lock = threading.Lock()
+_mongo_client = None
+
+
+def _create_mongo_client():
+    """Build a fail-fast MongoClient from environment variables."""
+    uri = os.environ.get('MONGO_URI')
+    if not uri:
+        raise RuntimeError(
+            'MONGO_URI environment variable is required when USE_MONGODB=true.'
+        )
+
+    # mongomock is used for unit/integration tests without a real server.
+    if uri.lower().startswith('mongomock') or uri.lower() == 'mongomock':
+        try:
+            import mongomock
+        except ImportError as exc:  # pragma: no cover - test dependency
+            raise RuntimeError(
+                'mongomock is required for test MongoDB mode. '
+                'Install it with: pip install mongomock'
+            ) from exc
+        return mongomock.MongoClient()
+
+    return MongoClient(
+        uri,
+        serverSelectionTimeoutMS=5000,
+        connectTimeoutMS=5000,
+        maxPoolSize=20,
+    )
+
+
+def init_mongo_client():
+    """Create a fresh MongoClient and store it as the process singleton.
+
+    Call this from a Gunicorn ``post_fork`` hook so each worker process gets
+    its own client after forking.  It can also be called in tests to reset the
+    shared client to a known state.
+    """
+    global _mongo_client
+    with _mongo_client_lock:
+        _mongo_client = _create_mongo_client()
+    return _mongo_client
+
+
+def get_mongo_client():
+    """Return the cached process-level MongoClient, creating it lazily once."""
+    global _mongo_client
+    if _mongo_client is None:
+        with _mongo_client_lock:
+            if _mongo_client is None:
+                _mongo_client = _create_mongo_client()
+    return _mongo_client
+
+
+def _db_name_from_uri(uri: str) -> str | None:
+    """Extract the database name from a MongoDB connection URI, if present."""
+    if not uri or uri.lower().startswith('mongomock'):
+        return None
+    try:
+        parsed = parse_uri(uri)
+        return parsed.get('database') or None
+    except Exception:
+        # Fallback to a simple path-based parse for non-standard URIs.
+        try:
+            path = urlparse(uri).path
+            return path.lstrip('/') or None
+        except Exception:
+            return None
+
+
+def get_mongo_db_name() -> str:
+    """Resolve the MongoDB database name from env vars or the connection URI.
+
+    Precedence:
+      1. ``MONGO_DB_NAME`` environment variable.
+      2. Database name parsed from ``MONGO_URI``.
+      3. Default based on ``DATA_ENV`` / ``DB_ENV``:
+           * prod -> ``chore_db``
+           * e2e  -> ``chore_db_e2e``
+           * test -> ``chore_db_test``
+    """
+    env_name = os.environ.get('MONGO_DB_NAME')
+    if env_name:
+        return env_name
+
+    uri = os.environ.get('MONGO_URI', '')
+    db_name = _db_name_from_uri(uri)
+    if db_name:
+        return db_name
+
+    env = (os.environ.get('DATA_ENV') or os.environ.get('DB_ENV', 'prod')).lower()
+    if env == 'prod':
+        return 'chore_db'
+    if env == 'e2e':
+        return 'chore_db_e2e'
+    return 'chore_db_test'

backend/db/tracking.py

@@ -1,4 +1,5 @@
 """Helper functions for tracking events database operations."""
+import itertools
 import logging
 from typing import Optional, List
 from tinydb import Query
@@ -8,6 +9,10 @@ from models.tracking_event import TrackingEvent, EntityType, ActionType
 
 logger = logging.getLogger(__name__)
 
+# Monotonic sequence used as a deterministic tiebreaker when tracking events
+# share the same ``occurred_at``/``created_at`` timestamps (common in tests).
+_tracking_event_seq = itertools.count()
+
 
 def insert_tracking_event(event: TrackingEvent) -> str:
     """
@@ -20,7 +25,9 @@ def insert_tracking_event(event: TrackingEvent) -> str:
         The event ID
     """
     try:
-        tracking_events_db.insert(event.to_dict())
+        event_dict = event.to_dict()
+        event_dict['_seq'] = next(_tracking_event_seq)
+        tracking_events_db.insert(event_dict)
         logger.info(f"Tracking event created: {event.action} {event.entity_type} {event.entity_id} for child {event.child_id}")
         return event.id
     except Exception as e:
@@ -61,8 +68,12 @@ def get_tracking_events_by_child(
     all_results = tracking_events_db.search(query_condition)
     total = len(all_results)
     
-    # Sort by occurred_at desc, then created_at desc
+    # Sort by occurred_at desc, then created_at desc, then _seq desc for
-    all_results.sort(key=lambda x: (x.get('occurred_at', ''), x.get('created_at', 0)), reverse=True)
+    # deterministic ordering when timestamps collide (common in fast tests).
+    all_results.sort(
+        key=lambda x: (x.get('occurred_at', ''), x.get('created_at', 0), x.get('_seq', 0)),
+        reverse=True,
+    )
 
     paginated = all_results[offset:offset + limit]
     events = [TrackingEvent.from_dict(r) for r in paginated]
@@ -99,7 +110,10 @@ def get_tracking_events_by_user(
     all_results = tracking_events_db.search(query_condition)
     total = len(all_results)
     
-    all_results.sort(key=lambda x: (x.get('occurred_at', ''), x.get('created_at', 0)), reverse=True)
+    all_results.sort(
+        key=lambda x: (x.get('occurred_at', ''), x.get('created_at', 0), x.get('_seq', 0)),
+        reverse=True,
+    )
 
     paginated = all_results[offset:offset + limit]
     events = [TrackingEvent.from_dict(r) for r in paginated]

backend/gunicorn.conf.py

@@ -0,0 +1,17 @@
+"""Gunicorn configuration for the chore/reward Flask backend.
+
+This file is automatically loaded by Gunicorn when it is started from the
+backend directory.  It ensures each worker process creates its own MongoDB
+client after forking, avoiding shared socket/file-descriptor issues.
+"""
+
+
+def post_fork(server, worker):
+    """Reinitialize the MongoDB client in each worker process after forking."""
+    try:
+        from db.mongo_client import init_mongo_client
+        init_mongo_client()
+    except Exception:
+        # If MongoDB is not configured (USE_MONGODB=false), there is no client
+        # to initialize; ignore the error silently.
+        pass

backend/main.py

@@ -4,7 +4,6 @@ import os
 
 from dotenv import load_dotenv
 from flask import Flask, request, jsonify
-from pymongo import MongoClient
 
 from api.admin_api import admin_api
 from api.auth_api import auth_api
@@ -28,6 +27,7 @@ from api.digest_action_api import digest_action_api
 from config.version import get_full_version
 
 from db.default import initializeImages, createDefaultTasks, createDefaultRewards
+from db.db import ensure_mongodb_indexes
 from events.broadcaster import Broadcaster
 from events.sse import sse_response_for_user, send_to_user
 from api.utils import get_current_user_id
@@ -39,6 +39,10 @@ from utils.state_expiry_scheduler import start_state_expiry_scheduler
 
 # Load environment variables
 load_dotenv()
+
+# Ensure MongoDB indexes exist when running against MongoDB.
+ensure_mongodb_indexes()
+
 # Configure logging once at application startup
 logging.basicConfig(
     level=logging.INFO,

backend/requirements.txt

@@ -26,6 +26,7 @@ iniconfig==2.3.0
 itsdangerous==2.2.0
 Jinja2==3.1.6
 MarkupSafe==3.0.3
+mongomock==4.3.0
 multidict==6.7.1
 packaging==25.0
 pillow==12.0.0

backend/scripts/migrate_to_mongodb.py

@@ -0,0 +1,203 @@
+# python
+"""
+Migrate existing TinyDB JSON files into MongoDB.
+
+Usage:
+    cd backend
+    python -m scripts.migrate_to_mongodb [--dry-run]
+
+The script reads files from ``data/db/`` (or ``test_data/db/`` when
+``DB_ENV=test``), maps each record's ``id`` field to MongoDB's ``_id`` field,
+and inserts the records idempotently.  TinyDB files are backed up to
+``<db_dir>/backups/<timestamp>/`` before the first migration run.
+"""
+import argparse
+import json
+import os
+import shutil
+import sys
+from datetime import datetime
+from pathlib import Path
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
+
+from config.paths import get_database_dir
+from db.db import COLLECTION_INDEXES, ensure_mongodb_indexes
+from db.mongo_client import get_mongo_client, get_mongo_db_name
+
+
+# Map TinyDB JSON filenames to MongoDB collection names.
+COLLECTION_FILE_MAP = {
+    'children.json': 'children',
+    'tasks.json': 'tasks',
+    'routines.json': 'routines',
+    'routine_items.json': 'routine_items',
+    'routine_schedules.json': 'routine_schedules',
+    'routine_extensions.json': 'routine_extensions',
+    'rewards.json': 'rewards',
+    'images.json': 'images',
+    'pending_rewards.json': 'pending_rewards',
+    'pending_confirmations.json': 'pending_confirmations',
+    'users.json': 'users',
+    'tracking_events.json': 'tracking_events',
+    'child_overrides.json': 'child_overrides',
+    'chore_schedules.json': 'chore_schedules',
+    'task_extensions.json': 'task_extensions',
+    'refresh_tokens.json': 'refresh_tokens',
+    'push_subscriptions.json': 'push_subscriptions',
+    'digest_action_tokens.json': 'digest_action_tokens',
+}
+
+
+def _load_tinydb_records(path: str) -> list[dict]:
+    """Load all records from a TinyDB JSON file."""
+    with open(path, 'r', encoding='utf-8') as f:
+        data = json.load(f)
+    default_table = data.get('_default', {})
+    return list(default_table.values())
+
+
+def _doc_to_mongo(doc: dict) -> dict:
+    """Map the model ``id`` field to MongoDB's ``_id`` field.
+
+    The original ``id`` field is removed so documents do not store both
+    ``_id`` and ``id`` with identical values.
+    """
+    mongo_doc = dict(doc)
+    if 'id' in mongo_doc:
+        mongo_doc['_id'] = mongo_doc.pop('id')
+    return mongo_doc
+
+
+def migrate(dry_run: bool = False) -> dict:
+    """Migrate TinyDB files to MongoDB and return a per-collection summary."""
+    db_dir = get_database_dir()
+    if not os.path.isdir(db_dir):
+        raise FileNotFoundError(f'Database directory does not exist: {db_dir}')
+
+    client = get_mongo_client()
+    db_name = get_mongo_db_name()
+    db = client[db_name]
+
+    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+    backup_dir = os.path.join(db_dir, 'backups', timestamp)
+
+    if not dry_run:
+        os.makedirs(backup_dir, exist_ok=True)
+        ensure_mongodb_indexes(client=client, db_name=db_name)
+
+    summary: dict[str, dict] = {}
+
+    for filename, collection_name in COLLECTION_FILE_MAP.items():
+        path = os.path.join(db_dir, filename)
+        if not os.path.exists(path):
+            summary[collection_name] = {
+                'source_file': filename,
+                'total': 0,
+                'migrated': 0,
+                'skipped': 0,
+                'status': 'missing',
+            }
+            continue
+
+        records = _load_tinydb_records(path)
+
+        if not dry_run:
+            shutil.copy2(path, backup_dir)
+
+        collection = db[collection_name]
+        to_insert: list[dict] = []
+        skipped = 0
+
+        for record in records:
+            doc_id = record.get('id')
+            if not doc_id:
+                skipped += 1
+                continue
+
+            if not dry_run:
+                existing = collection.find_one({'_id': doc_id})
+                if existing:
+                    skipped += 1
+                    continue
+
+            to_insert.append(_doc_to_mongo(record))
+
+        if not dry_run and to_insert:
+            try:
+                collection.insert_many(to_insert, ordered=False)
+            except Exception as exc:  # pragma: no cover - defensive
+                print(
+                    f'  Warning: error inserting into {collection_name}: {exc}',
+                    file=sys.stderr,
+                )
+                raise
+
+        summary[collection_name] = {
+            'source_file': filename,
+            'total': len(records),
+            'migrated': len(to_insert),
+            'skipped': skipped,
+            'status': 'migrated' if not dry_run else 'dry-run',
+        }
+
+    return summary
+
+
+def _print_summary(summary: dict) -> None:
+    """Print a human-readable migration summary."""
+    print('\nMigration Summary')
+    print('-' * 70)
+    print(f'{"Collection":<30}{"Total":>8}{"Migrated":>10}{"Skipped":>10}{"Status":>10}')
+    print('-' * 70)
+    total_records = 0
+    total_migrated = 0
+    total_skipped = 0
+    for collection_name, info in summary.items():
+        print(
+            f'{collection_name:<30}'
+            f'{info["total"]:>8}'
+            f'{info["migrated"]:>10}'
+            f'{info["skipped"]:>10}'
+            f'{info["status"]:>10}'
+        )
+        total_records += info['total']
+        total_migrated += info['migrated']
+        total_skipped += info['skipped']
+    print('-' * 70)
+    print(
+        f'{"TOTAL":<30}'
+        f'{total_records:>8}'
+        f'{total_migrated:>10}'
+        f'{total_skipped:>10}'
+    )
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description='Migrate TinyDB JSON files to MongoDB.'
+    )
+    parser.add_argument(
+        '--dry-run',
+        action='store_true',
+        help='Analyze files and report counts without writing to MongoDB.',
+    )
+    args = parser.parse_args()
+
+    if os.environ.get('USE_MONGODB', 'false').lower() != 'true':
+        print('Set USE_MONGODB=true to run the migration.', file=sys.stderr)
+        sys.exit(1)
+
+    if not os.environ.get('MONGO_URI'):
+        print('MONGO_URI is required when USE_MONGODB=true.', file=sys.stderr)
+        sys.exit(1)
+
+    if args.dry_run:
+        print('Dry run: no data will be written to MongoDB.')
+
+    summary = migrate(dry_run=args.dry_run)
+    _print_summary(summary)
+
+
+if __name__ == '__main__':
+    main()

backend/scripts/run_integration_tests.ps1

@@ -0,0 +1,79 @@
+<#
+.SYNOPSIS
+    Run the MongoDB adapter integration tests against a local Docker MongoDB container.
+
+.DESCRIPTION
+    Starts a temporary MongoDB container, runs a targeted pytest suite with
+    USE_MONGODB=true, then stops and removes the container.
+
+.EXAMPLE
+    cd backend
+    .\scripts\run_integration_tests.ps1
+#>
+[CmdletBinding()]
+param(
+    [string]$ContainerName = 'chore-db-integration-test',
+    [int]$HostPort = 27017,
+    [string]$Image = 'mongo:8',
+    [string]$DbName = 'chore_db_test',
+    [string]$TestPath = 'tests/test_mongo_adapter.py'
+)
+
+$ErrorActionPreference = 'Stop'
+
+$mongoUri = "mongodb://localhost:${HostPort}/${DbName}"
+
+function Test-ContainerRunning {
+    $containers = docker ps --filter "name=$ContainerName" --format '{{.Names}}' 2>$null
+    return $containers -contains $ContainerName
+}
+
+function Wait-MongoReady {
+    param([int]$TimeoutSeconds = 30)
+    $start = Get-Date
+    while (((Get-Date) - $start).TotalSeconds -lt $TimeoutSeconds) {
+        try {
+            $null = docker exec $ContainerName mongosh --eval 'db.adminCommand({ ping: 1 })' --quiet 2>$null
+            if ($LASTEXITCODE -eq 0) {
+                return
+            }
+        } catch {
+            # Container or mongosh may not be ready yet.
+        }
+        Start-Sleep -Seconds 1
+    }
+    throw "MongoDB container did not become ready within ${TimeoutSeconds} seconds."
+}
+
+# Clean up any leftover container from a previous aborted run.
+if (Test-ContainerRunning) {
+    Write-Host "Removing existing container '$ContainerName'..."
+    docker rm -f $ContainerName | Out-Null
+}
+
+Write-Host "Starting MongoDB container '$ContainerName' on port $HostPort..."
+docker run -d `
+    --name $ContainerName `
+    -p "${HostPort}:27017" `
+    $Image | Out-Null
+
+try {
+    Wait-MongoReady
+    Write-Host "MongoDB is ready. Running integration tests..."
+
+    $env:USE_MONGODB = 'true'
+    $env:MONGO_URI = $mongoUri
+    $env:MONGO_DB_NAME = $DbName
+    $env:DB_ENV = 'test'
+    $env:DATA_ENV = 'test'
+
+    pytest $TestPath
+    if ($LASTEXITCODE -ne 0) {
+        throw "Integration tests failed with exit code $LASTEXITCODE."
+    }
+} finally {
+    Write-Host "Stopping and removing container '$ContainerName'..."
+    docker rm -f $ContainerName | Out-Null
+}
+
+Write-Host "Integration tests complete."

backend/tests/conftest.py

@@ -1,5 +1,7 @@
 import os
 os.environ['DB_ENV'] = 'test'
+os.environ['USE_MONGODB'] = 'true'
+os.environ['MONGO_URI'] = 'mongomock'
 os.environ.setdefault('SECRET_KEY', 'test-secret-key')
 os.environ.setdefault('REFRESH_TOKEN_EXPIRY_DAYS', '90')
 os.environ.setdefault('DIGEST_TOKEN_SECRET', 'test-digest-secret')
@@ -18,5 +20,11 @@ TEST_REFRESH_TOKEN_EXPIRY_DAYS = 90
 @pytest.fixture(scope="session", autouse=True)
 def set_test_db_env():
     os.environ['DB_ENV'] = 'test'
+    os.environ['USE_MONGODB'] = 'true'
+    os.environ['MONGO_URI'] = 'mongomock'
     os.environ['SECRET_KEY'] = TEST_SECRET_KEY
     os.environ['REFRESH_TOKEN_EXPIRY_DAYS'] = str(TEST_REFRESH_TOKEN_EXPIRY_DAYS)
+    # Ensure indexes are created once for the test session.  This is safe to
+    # call repeatedly because MongoDB treats index creation as idempotent.
+    from db.db import ensure_mongodb_indexes
+    ensure_mongodb_indexes()

backend/tests/test_mongo_adapter.py

@@ -0,0 +1,201 @@
+import os
+import pytest
+from tinydb import Query
+
+from db.db import (
+    MongoLockedTable,
+    _query_to_mongo_filter,
+    child_db,
+    task_db,
+    users_db,
+    refresh_tokens_db,
+)
+from db.mongo_client import get_mongo_client, get_mongo_db_name
+
+
+# All tests in this module require the mongomock-backed MongoDB adapter.
+pytestmark = pytest.mark.skipif(
+    os.environ.get('USE_MONGODB', 'false').lower() != 'true',
+    reason='MongoDB adapter tests require USE_MONGODB=true',
+)
+
+
+@pytest.fixture(autouse=True)
+def clean_mongo_collections():
+    """Truncate relevant collections before each test."""
+    child_db.truncate()
+    task_db.truncate()
+    users_db.truncate()
+    refresh_tokens_db.truncate()
+    yield
+
+
+class TestQueryTranslation:
+    """Unit tests for TinyDB Query -> MongoDB filter translation."""
+
+    def test_simple_equality(self):
+        q = Query()
+        assert _query_to_mongo_filter(q.id == 'abc') == {'_id': {'$eq': 'abc'}}
+
+    def test_field_other_than_id(self):
+        q = Query()
+        assert _query_to_mongo_filter(q.user_id == 'u1') == {
+            'user_id': {'$eq': 'u1'}
+        }
+
+    def test_and_query(self):
+        q = Query()
+        mongo_filter = _query_to_mongo_filter(
+            (q.id == 'abc') & (q.user_id == 'u1')
+        )
+        assert mongo_filter == {'_id': {'$eq': 'abc'}, 'user_id': {'$eq': 'u1'}}
+
+    def test_or_query(self):
+        q = Query()
+        mongo_filter = _query_to_mongo_filter(
+            (q.user_id == 'u1') | (q.user_id == None)  # noqa: E711
+        )
+        assert mongo_filter == {
+            '$or': [
+                {'user_id': {'$eq': 'u1'}},
+                {'user_id': {'$eq': None}},
+            ]
+        }
+
+    def test_and_with_nested_or(self):
+        q = Query()
+        mongo_filter = _query_to_mongo_filter(
+            (q.id == 'abc') & ((q.user_id == 'u1') | (q.user_id == None))  # noqa: E711
+        )
+        assert mongo_filter == {
+            '_id': {'$eq': 'abc'},
+            '$or': [
+                {'user_id': {'$eq': 'u1'}},
+                {'user_id': {'$eq': None}},
+            ],
+        }
+
+
+class TestMongoCrud:
+    """CRUD tests against the mongomock-backed MongoLockedTable."""
+
+    def test_insert_maps_id_to_underscore_id(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice', 'age': 8})
+
+        raw = get_mongo_client()[get_mongo_db_name()]['children'].find_one(
+            {'_id': 'c1'}
+        )
+        assert raw is not None
+        assert raw['_id'] == 'c1'
+        assert 'id' not in raw
+        assert raw['name'] == 'Alice'
+
+    def test_get_returns_document_without_underscore_id(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice', 'age': 8})
+
+        doc = child_db.get(Query().id == 'c1')
+        assert doc is not None
+        assert doc['id'] == 'c1'
+        assert doc['name'] == 'Alice'
+        assert '_id' not in doc
+
+    def test_get_none_when_missing(self):
+        assert child_db.get(Query().id == 'missing') is None
+
+    def test_search_with_query(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice', 'user_id': 'u1'})
+        child_db.insert({'id': 'c2', 'name': 'Bob', 'user_id': 'u1'})
+        child_db.insert({'id': 'c3', 'name': 'Carol', 'user_id': 'u2'})
+
+        results = child_db.search(Query().user_id == 'u1')
+        assert len(results) == 2
+        assert {r['id'] for r in results} == {'c1', 'c2'}
+
+    def test_search_with_or(self):
+        task_db.insert({'id': 't1', 'name': 'Default', 'user_id': None})
+        task_db.insert({'id': 't2', 'name': 'User task', 'user_id': 'u1'})
+
+        q = Query()
+        results = task_db.search((q.user_id == 'u1') | (q.user_id == None))  # noqa: E711
+        assert len(results) == 2
+
+    def test_update_modifies_matching_documents(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice', 'points': 0})
+        child_db.insert({'id': 'c2', 'name': 'Bob', 'points': 0})
+
+        modified = child_db.update({'points': 10}, Query().id == 'c1')
+        # TinyDB returns a list of updated document ids; the adapter mirrors that.
+        assert modified == ['c1']
+
+        doc = child_db.get(Query().id == 'c1')
+        assert doc['points'] == 10
+
+        other = child_db.get(Query().id == 'c2')
+        assert other['points'] == 0
+
+    def test_update_does_not_overwrite_id(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice'})
+
+        child_db.update({'id': 'c2', 'name': 'Alice Smith'}, Query().id == 'c1')
+        # The id field must remain unchanged; update should have stripped id.
+        assert child_db.get(Query().id == 'c1')['name'] == 'Alice Smith'
+        assert child_db.get(Query().id == 'c2') is None
+
+    def test_remove_deletes_matching_documents(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice'})
+        child_db.insert({'id': 'c2', 'name': 'Bob'})
+
+        deleted = child_db.remove(Query().id == 'c1')
+        # TinyDB returns a list of removed document ids; the adapter mirrors that.
+        assert deleted == ['c1']
+
+        assert child_db.get(Query().id == 'c1') is None
+        assert child_db.get(Query().id == 'c2') is not None
+
+    def test_all_returns_all_documents(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice'})
+        child_db.insert({'id': 'c2', 'name': 'Bob'})
+
+        docs = child_db.all()
+        assert len(docs) == 2
+        assert all('_id' not in d for d in docs)
+
+    def test_truncate_removes_all_documents(self):
+        child_db.insert({'id': 'c1', 'name': 'Alice'})
+        child_db.truncate()
+        assert child_db.all() == []
+
+    def test_insert_multiple(self):
+        ids = child_db.insert_multiple([
+            {'id': 'c1', 'name': 'Alice'},
+            {'id': 'c2', 'name': 'Bob'},
+        ])
+        assert sorted(ids) == ['c1', 'c2']
+        assert len(child_db.all()) == 2
+
+    def test_unique_token_index(self):
+        refresh_tokens_db.insert({'id': 'r1', 'token': 'abc', 'user_id': 'u1'})
+        refresh_tokens_db.insert({'id': 'r2', 'token': 'def', 'user_id': 'u1'})
+
+        # mongomock does not enforce unique indexes by default, but we verify
+        # both records are readable.
+        assert refresh_tokens_db.get(Query().token == 'abc') is not None
+        assert refresh_tokens_db.get(Query().token == 'def') is not None
+
+    def test_user_id_secondary_index_is_created(self):
+        # Insert and query via the secondary index path used by the app.
+        users_db.insert({'id': 'u1', 'email': 'a@example.com', 'verified': True})
+        users_db.insert({'id': 'u2', 'email': 'b@example.com', 'verified': False})
+
+        found = users_db.search(Query().verified == True)  # noqa: E712
+        assert len(found) == 1
+        assert found[0]['id'] == 'u1'
+
+
+class TestAdapterApi:
+    """Tests that the adapter exposes the expected LockedTable-compatible API."""
+
+    def test_close_is_noop(self):
+        # Existing cleanup fixtures call ``*_db.close()``; ensure it does not
+        # raise for the MongoDB-backed adapter.
+        child_db.close()

frontend/.env.test

@@ -0,0 +1,4 @@
+# MongoDB configuration for end-to-end tests.
+# Uses mongomock so E2E tests do not require a running MongoDB server.
+USE_MONGODB=true
+MONGO_URI=mongomock

frontend/playwright.config.ts

@@ -8,12 +8,9 @@ import {
 } from './e2e/e2e-constants'
 
 /**
- * Read environment variables from file.
+ * E2E MongoDB configuration is hardcoded below.  Developers can override
- * https://github.com/motdotla/dotenv
+ * values via environment variables; see .env.test for an example.
  */
-// import dotenv from 'dotenv';
-// import path from 'path';
-// dotenv.config({ path: path.resolve(__dirname, '.env') });
 
 /**
  * See https://playwright.dev/docs/test-configuration.
@@ -288,6 +285,8 @@ export default defineConfig({
           'BNKkHdq45uLigohSG7c1TwlAo7ETncoRVLQK02LxHgu2P1DgSJD9njRMfbbzUsaTQGllvLBz7An1WiWsNYQhvKE',
         VAPID_PRIVATE_KEY: 'jNiZJT0UO4H861KmnCt874Fg6p5jDAyYKS4V2MZf8bQ',
         PROCESS_PLATFORM: process.platform,
+        USE_MONGODB: process.env.USE_MONGODB || 'true',
+        MONGO_URI: process.env.MONGO_URI || 'mongomock',
       },
     },
   ],

specs/feat-database-migration.md

@@ -0,0 +1,164 @@
+# Feature: Migrate Backend Persistence from TinyDB to MongoDB Atlas
+
+## Overview
+
+**Goal:** Replace the TinyDB JSON-file persistence layer in the Flask backend with MongoDB Atlas while keeping the existing API surface, model definitions, and test infrastructure largely unchanged. Introduce a migration script that exports existing TinyDB data into MongoDB.
+
+**User Story:** As a developer and operator, I want the application to run on a managed, scalable document database so that concurrent access, multi-worker deployments, and production hosting are reliable.
+
+**Rules:**
+- Backend routes and response shapes must remain unchanged.
+- Python models in `backend/models/` keep their `from_dict()` / `to_dict()` interface.
+- Every mutation must continue to emit SSE events through the existing broadcaster.
+- The migration must be reversible (keep TinyDB files as a backup) and idempotent.
+- All existing tests must continue to pass; new MongoDB-backed tests must be added.
+
+---
+
+## Data Model Changes
+
+### Backend Model
+- No changes to dataclass fields or `to_dict()` / `from_dict()` output.
+- The model's `id` field is stored as MongoDB's `_id` primary key. The original `id` field is removed from storage so documents do not contain duplicate values.
+- On read, `_id` is mapped back to `id` and the document is wrapped as a TinyDB-compatible `Document` with `doc_id` set.
+- `created_at` and `updated_at` remain epoch floats.
+
+### Frontend Model
+- No changes. The API contract is preserved.
+
+---
+
+## Backend Implementation
+
+### 1. Database client initialization
+- Create `backend/db/mongo_client.py`.
+- Do **not** create a `MongoClient` at module import; it is created lazily on first access.
+- `get_mongo_client()` returns a cached process-level singleton so tests, scripts, and Flask requests all share the same client.
+- For Gunicorn multi-worker deployments, `backend/gunicorn.conf.py` defines a `post_fork` hook that calls `init_mongo_client()` so each worker process gets its own client after forking.
+- Fail-fast settings for development:
+  ```python
+  MongoClient(
+      os.getenv("MONGO_URI"),
+      serverSelectionTimeoutMS=5000,
+      connectTimeoutMS=5000,
+      maxPoolSize=20,
+  )
+  ```
+
+### 2. Collection mapping
+- One MongoDB collection per existing TinyDB file:
+  - `children` ← `children.json`
+  - `tasks` ← `tasks.json`
+  - `routines` ← `routines.json`
+  - `routine_items` ← `routine_items.json`
+  - `routine_schedules` ← `routine_schedules.json`
+  - `routine_extensions` ← `routine_extensions.json`
+  - `rewards` ← `rewards.json`
+  - `images` ← `images.json`
+  - `pending_rewards` ← `pending_rewards.json` (legacy)
+  - `pending_confirmations` ← `pending_confirmations.json`
+  - `users` ← `users.json`
+  - `tracking_events` ← `tracking_events.json`
+  - `child_overrides` ← `child_overrides.json`
+  - `chore_schedules` ← `chore_schedules.json`
+  - `task_extensions` ← `task_extensions.json`
+  - `refresh_tokens` ← `refresh_tokens.json`
+  - `push_subscriptions` ← `push_subscriptions.json`
+  - `digest_action_tokens` ← `digest_action_tokens.json`
+
+### 3. `LockedTable` MongoDB adapter
+- Refactor `backend/db/db.py` so that each exported `*_db` object continues to expose the same methods: `search(...)`, `get(...)`, `all()`, `insert(...)`, `insert_multiple(...)`, `update(...)`, `remove(...)`, `truncate()`.
+- Implement a new `MongoLockedTable` class that mirrors the existing `LockedTable` API but delegates to a MongoDB collection.
+- Keep `LockedTable` (TinyDB) as a fallback selectable by `USE_MONGODB=true|false` (default `false`).
+- Store the model `id` field as MongoDB `_id` on writes; do not keep a duplicate `id` field in storage.
+- Restore the model `id` field from `_id` on reads and return TinyDB-compatible `Document` objects with `doc_id` set so existing callers that rely on `doc_id` continue to work.
+- Support `update(fields, doc_ids=[...])` and callable updater functions (`update(lambda rec: ..., cond)`) to match TinyDB behavior.
+- Reuse function names where possible to minimize API file changes.
+
+### 4. Indexes
+- Create indexes on application startup (`backend/main.py`) and in the pytest session fixture.  Index creation is intentionally **not** done at module import time so the `MongoClient` remains lazily initialized.
+- Required indexes:
+  - Unique primary key is provided by MongoDB `_id` (mapped from model `id`); no separate `id` index is created.
+  - `user_id` — secondary on `children`, `tasks`, `routines`, `routine_items`, `rewards`, `images`, `tracking_events`, `pending_confirmations`, `chore_schedules`, `task_extensions`, `refresh_tokens`, `push_subscriptions`, `digest_action_tokens`.
+  - `child_id` — secondary on `child_overrides`, `chore_schedules`, `task_extensions`, `pending_confirmations`, `tracking_events`.
+  - `entity_id` + `entity_type` — compound secondary on `pending_confirmations`, `tracking_events`, `child_overrides`.
+  - `token` — unique on `refresh_tokens` and `digest_action_tokens`.
+
+### 5. Migration script
+- Create `backend/scripts/migrate_to_mongodb.py`.
+- Reads each TinyDB JSON file in `data/db/` (or `test_data/db/` when `DB_ENV=test`).
+- Inserts each `_default` record into the corresponding MongoDB collection, mapping TinyDB integer keys to the record's own `id` field.
+- Skips records that already exist by `id` (idempotent).
+- Backs up TinyDB files to `data/db/backups/<timestamp>/` before first run.
+- Prints a summary of migrated records per collection.
+- Usage:
+  ```bash
+  cd backend
+  python -m scripts.migrate_to_mongodb [--dry-run]
+  ```
+
+### 6. Environment variables
+- `MONGO_URI` — required when `USE_MONGODB=true`.
+- `MONGO_DB_NAME` — optional database name; defaults parsed from `MONGO_URI` or falls back to `chore_db`/`chore_db_test`/`chore_db_e2e` based on `DB_ENV`/`DATA_ENV`.
+- `USE_MONGODB` — `true` to use MongoDB backend; `false` to keep TinyDB.
+
+### 7. Default data initialization
+- `backend/db/default.py` (`initializeImages`, `createDefaultTasks`, `createDefaultRewards`) must work with the new MongoDB-backed `*_db` objects. No API changes.
+
+### 8. Scheduler compatibility
+- Background schedulers (`account_deletion_scheduler`, `digest_scheduler`, etc.) read/write through the same `*_db` objects, so they require no changes once the adapter is in place.
+
+### 9. Tracking event ordering
+- `db/tracking.py` adds a hidden monotonic `_seq` field to tracking events on insert and sorts by `(occurred_at, created_at, _seq)` descending. This guarantees deterministic ordering when consecutive events share the same timestamp (common in fast tests).
+
+### 10. Gunicorn / Docker deployment
+- `backend/gunicorn.conf.py` provides the `post_fork` hook required by the wiki plan.
+- `backend/Dockerfile` now loads the config via `-c gunicorn.conf.py`.
+
+---
+
+## Backend Tests
+
+- [x] Add `mongomock` to test dependencies and create `backend/tests/test_mongo_adapter.py` that verifies CRUD operations against `mongomock`.
+- [x] Update `backend/tests/conftest.py` to set `USE_MONGODB=true` and point `MONGO_URI` at `mongomock` for the default test run.
+- [x] Ensure every existing API test still passes with the MongoDB adapter by running `pytest tests/`.
+- [x] Add integration test script `backend/scripts/run_integration_tests.ps1` that starts a local MongoDB Docker container, runs a targeted pytest suite, and tears down the container.
+- [x] Add `frontend/.env.test` with MongoDB configuration and update Playwright webServer env to pass `USE_MONGODB`/`MONGO_URI` to the Flask backend; E2E database is reset via the existing `/auth/e2e-seed` endpoint.
+
+---
+
+## Frontend Implementation
+
+- No frontend changes are required. The API contract remains unchanged.
+
+## Frontend Tests
+
+- [ ] Run `npx playwright test` and confirm E2E tests pass against the MongoDB-backed Flask backend with the isolated `chore_db_e2e` database.
+
+---
+
+## Future Considerations
+
+- After a production burn-in period, remove the TinyDB fallback and `LockedTable` code.
+- Add database connection health-check endpoint.
+- Consider MongoDB transactions for multi-document operations (e.g., child deletion cascades).
+
+---
+
+## Acceptance Criteria (Definition of Done)
+
+### Backend
+- [x] `backend/db/db.py` exports MongoDB-backed `*_db` objects when `USE_MONGODB=true`.
+- [x] `backend/db/mongo_client.py` implements lazy client initialization and a Gunicorn-compatible init hook.
+- [x] All existing API endpoints continue to work without route or response changes.
+- [x] `backend/scripts/migrate_to_mongodb.py` migrates existing TinyDB JSON files idempotently.
+- [x] Required unique and secondary indexes are created on application startup or migration.
+- [x] Unit tests pass with `mongomock` (`pytest tests/`).
+- [ ] Integration tests pass against a local Docker MongoDB container (requires a running Docker daemon; script verified to start/stop container when daemon is available).
+
+### Frontend
+- [x] E2E setup and representative parent-mode smoke tests pass against the isolated `chore_db_e2e` mongomock database, with the DB reset via `/auth/e2e-seed`.
+
+### Operations
+- [x] `MONGO_URI` and optional `MONGO_DB_NAME` environment variables are documented.
+- [x] Rollback instructions exist to switch back to TinyDB by setting `USE_MONGODB=false`.