10 / 15 · Data Structure
Records, fields and relationships
Use this data dictionary when building the stores, API payloads and sync flow. It shows what the main records look like and where they belong.
The existing app has no SQL database. A table below describes a collection or record shape, not a request to add a relational server database. Field names are technical identifiers; all examples are fictional. This chapter documents the implementation checked on 26 October 2026, with reconstruction requirements and compatibility limits marked explicitly.
On narrow screens, scroll each table sideways to read every column. Tables can also receive keyboard focus.
1 · Storage map
| Logical collection | Key / location | Content and ownership |
|---|---|---|
| Users | users.json → users[userKey] | Server account registry; identity and account status. |
| Sessions | sessions.json → [sessionId] | Server login sessions; signed session ID in an HttpOnly cookie. |
| Notes | IndexedDB; optional Drive .md | Readable titles, writing and structured editor blocks belong to the browser and the user’s Drive. |
| Folders | Local graph / folder:<id>; data.json → noteFolders or folders | Separate Notes and Files trees; folder names and hierarchy can be server metadata. |
| Pending notes, bases and tombstones | Account Drive IndexedDB records | Durable outgoing work, last confirmed document and deleted markers. |
| Drive note index | data.json → driveNotes.files[noteId] | Technical metadata only after legacy migration; never a durable note projection. |
| Media metadata | data.json → files[name] or googleDriveFiles[fileId] | Playback, annotations and source information. Original Drive file bytes stay in Drive. |
| Drive connection | data.json → googleDriveConnection | Server-side encrypted refresh token and connection metadata. |
| Canvases | document-assets/<docId>/canvas-<canvasId>.json | Separate server-side canvas document; may contain user text and references. |
| Sync tickets | encrypted-sync/<ticket>.json | Temporary encrypted packet and result; remove when result is consumed. |
Each server data location is relative to the authenticated account’s storage root, normally APP_DATA_DIR/users/<userKey>/. The legacy owner can use APP_DATA_DIR/ itself. The global users and sessions files live at that top level. Never choose an account root from an untrusted request field.
2 · Relationships and identifiers
| Relationship | Cardinality / reference | Rule |
|---|---|---|
| User → sessions and account stores | One user to many sessions and records | Ownership comes from authenticated userKey, not from a note ID alone. |
| Folder → child folders | One to many through parentId | Only the collection root has no parent; reject cycles and invalid parents. |
| Notes folder → notes | One to many through DocumentRecord.folderId | Notes root is folder-notes; Files root is folder-home. Do not mix the trees. |
| Note ↔ Drive file | Optional technical mapping noteId → fileId | A logical note ID is different from Google’s file ID and revision string. |
| Note → base / pending / deleted | Zero or one current record per note and key prefix | A pending document: null means delete. A token identifies a specific queued version. |
| Note ↔ media sources | Many to many through editor blocks and source references | Store references/URLs, not duplicate GB-sized file bytes. These are embedded references, not a SQL join table. |
| Note → canvases → elements / connectors | One to many; Canvas.docId and endpoint IDs | Connectors reference element IDs within the same canvas. A noteRef.docId points to another note. |
User (userKey)
├─ Session [sessionId]
└─ Account storage
├─ NotesFolder [id] ── parentId → NotesFolder
│ └─ Note [id, folderId]
│ ├─ editor blocks → media source / other note
│ ├─ Pending / Base / Deleted [noteId]
│ ├─ DriveIndex [noteId] → Google Drive .md [fileId]
│ └─ Canvas [docId, id] → Elements → Connectors
└─ FilesFolder [id] → Media metadata → local file / Drive sourceUnless specified otherwise, app timestamps are numbers in milliseconds since the Unix epoch. Drive revision values are opaque strings; never compare them as integers. Positions are numeric sibling order. Missing optional fields and explicit null have different meanings: preserve the distinction during round trips.
3 · Note record and editor content
DocumentRecord is the browser/API document shape. Core fields are required for a normalized note; optional fields must survive supported round trips.
| Field / key | Type and presence | Meaning / example |
|---|---|---|
id | string · core | Stable logical note ID, for example doc-example. Use IDs compatible with endpoint validation: letters, digits, underscores and hyphens. |
title | string · core | Note title. This is readable note content, including when shown in a list. |
folderId | string · core | Existing Notes folder ID, default folder-notes. |
content | ProseMirror JSON object · core | {type:"doc", content:[...]}; structured nodes, marks and attributes, not a plain HTML string. |
createdAt, updatedAt | number · core | Creation and last modification time in milliseconds. |
trashedAt | number | null · core | Legacy trash timestamp; null means active. New permanent deletion uses tombstones. |
driveFileId, driveRevision | string · optional | Decorated Drive mapping and confirmed remote version; excluded from encoded note metadata. |
driveReadError | string · optional | Read failure information; never treat an unreadable note as a confirmed empty document. |
questions, goalReferences | array · optional | Structured note content. The source types are any[]; define and validate the supported entries before implementing these extensions. Do not invent a fixed schema from the array names. |
goalStatus | enum · optional compatibility | uncompleted | in_progress | completed | cancelled. Retained data does not authorize adding task-management UI. |
goalUrgency, goalImportance | high | low · optional compatibility | Retained fields, not mandatory product features. |
trashedVia | string · optional runtime field | Origin folder for legacy trash operations; used by the implementation even though absent from the declared DocumentRecord type. |
forcedLanguage | string | null · optional legacy | Preserve existing imported data where needed; do not build automatic translation or new forced-language controls. |
Editor node structure
| Field / key | Type and presence | Meaning / example |
|---|---|---|
type | string | Schema node name such as doc, heading, paragraph or text. |
content | node[] · optional | Recursive ordered child nodes. A paragraph can contain text nodes. |
text | string · text nodes | The actual text. |
attrs | object · schema-dependent | For example heading {level:1}, media source attributes or a canvas reference. Validate against the editor schema. |
marks | array of mark objects · optional | Inline formatting/link marks; preserve supported type and attrs. |
{
"id": "doc-example",
"title": "Research notes",
"folderId": "note-folder-example",
"content": {
"type": "doc",
"content": [
{
"type": "heading",
"attrs": {
"level": 1
},
"content": [
{
"type": "text",
"text": "Research notes"
}
]
},
{
"type": "paragraph",
"content": [
{
"type": "text",
"text": "My observations about the source file."
}
]
}
]
},
"createdAt": 1793000000000,
"updatedAt": 1793000001000,
"trashedAt": null
}4 · Folder tables
The server keeps dictionaries noteFolders[id] and folders[id]. The browser uses Notes workspaces in its local graph or folder:<id> records in Drive mode.
| Field / key | Type and presence | Meaning / example |
|---|---|---|
id, name | string · core | Stable folder ID and readable display name. |
parentId | string | null · normalized server shape | null only for the collection root; a child references a folder in the same tree. Browser workspace summaries may omit a root parent. |
position | number · normalized server / Drive folder | Order among siblings; do not derive identity from ordering or name. |
createdAt, updatedAt | number · normalized server shape | Milliseconds. |
trashed | object · optional server marker | {at:number, folderId:string, viaFolder:string}; preserve deletion origin during legacy restore. |
trashedAt, trashedVia | number | null; string · optional browser fields | Local workspace summaries also use these trash markers. Map them deliberately; they are not identical to the server marker object. |
{
"id": "note-folder-example",
"name": "Sources",
"parentId": "folder-notes",
"position": 0,
"createdAt": 1793000000000,
"updatedAt": 1793000000000
}5 · Browser tables and outgoing operations
Drive mode: account database
IndexedDB filesandnotes-drive-<userKey>, version 1, contains the object store records. Keys are external strings; the stored value is the record itself, not a {key,value} wrapper.
| Key prefix | Value shape | Lifetime / meaning |
|---|---|---|
note:<id> | DocumentRecord | Current local document; render before a background refresh. |
base:<id> | DocumentRecord | Confirmed merge ancestor, needed for a three-way merge. Contains note content: browser only. |
pending:<id> | {id, document, baseDocument?, action?, token} | Latest local operation. document is a full document or null for deletion; action may be restore; token is a fresh string. |
deleted:<id> | {id:string} | Deletion tombstone; blocks stale fetches from restoring a deleted note. |
sent:<id> | string | Operation token already sent; supports duplicate suppression. It does not replace the confirmed base. |
folder:<id> | Folder record | Cached Notes workspace. |
folder-op:<sequence> | {key,id,kind,body} | Ordered outgoing action; kind is create, rename, move, delete or restore. Body depends on that action. Sequence is padded to 12 digits. |
deleted-folder:<id> | {id:string} | Folder deletion tombstone. |
meta:* | Value depends on key | Technical state including folder sequence, last sync time, upload result and legacy-import marker; no single universal meta-record type. |
Write the local document and its pending operation in the same IndexedDB transaction. An acknowledgement may clear only the operation whose token still matches. Keep newer edits, failed operations and tombstones through reloads. Use account-specific browser locks and sync messages.
Browser-only mode and compatibility
The existing browser-only store opens paper_documents, version 4. It is still used when Drive mode is inactive and can later be imported into the Drive store. Local-first sync is disabled in its current constructor; these stores do not imply a second server sync protocol.
| Object store | Key / index | Value shape |
|---|---|---|
documents | keyPath id | Local document projection. |
local_first_datasets | keyPath datasetId | {datasetId, payload:string, updatedAt:number}. IDs include graph and doc:<id>; payload is serialized local-first state. |
local_first_pending_updates | keyPath id; index datasetId | {id,datasetId,payload,createdAt,deviceId?,tabId?}. IDs/payload are strings; time is numeric. |
local_first_sync_cursors | keyPath datasetId | {datasetId,seq:number,updatedAt:number}. |
local_first_state | keyPath key | Internal state entries, including device identity; validate each supported key. |
The browser-only database name currently has no userKey suffix. Do not interpret that as proof of account isolation. A rebuild must namespace all readable local notes and queues per account, migrate existing data safely and verify account switching. Changing this documentation does not fix the existing runtime limitation.
6 · Server JSON and technical Drive index
data.json is normalized through an explicit top-level whitelist. Adding a service key without updating that whitelist loses it on the next write.
| Top-level key | Shape | Purpose |
|---|---|---|
version | number, currently 1 | Envelope version. |
files, googleDriveFiles | object dictionaries | Local and Drive media metadata respectively. |
folders, noteFolders | object dictionaries | Files and Notes folder metadata. |
driveNotes | object | Activation, folder mapping, technical file index and allocated Drive IDs. |
googleDriveConnection | object | null | Drive authorization connection; see below. |
notesFolderMigrationVersion | number | Folder migration marker. |
noteStorageVersion | optional number; 1 after acknowledgement | Legacy note-content transfer marker. |
documents | object, empty after migration | Legacy content source only; do not create new server notes here. |
playlists, papers | object dictionaries | Existing feature/compatibility collections; not a new required core workflow. |
reports | array, capped at 200 | Technical reports; never include note content or secrets. |
Drive state
| Field / key | Type and presence | Meaning / example |
|---|---|---|
active | boolean | Whether Drive is the note sync backend. |
folderId, folderName, email | string · optional state metadata | Chosen Drive note folder and connection identity. |
files | dictionary keyed by logical note ID | Values restricted to the technical fields below after migration. |
allocations | dictionary of note ID → Drive file ID | Stable allocated IDs for retry-safe creation. |
Persisted Drive file entry
| Field / key | Type and presence | Meaning / example |
|---|---|---|
fileId | string | Remote Google Drive file ID. |
revision | string | Opaque version last observed/confirmed. |
etag | string · optional | Conditional request token. |
hash | string · optional | SHA-256 fingerprint of normalized Markdown. A hash is metadata, not encryption. |
deleted | boolean · optional | Known deleted mapping. |
updatedAt | number · optional | Last document modification time, milliseconds. |
source | string · optional | Origin marker, for example local or external. |
lastToken | string · optional | Push token used for duplicate retry detection. |
This is the actual allowlist from driveFileMetadata. No title, content, baseContent, questions or goalReferences in newly persisted entries. Legacy entries may remain until a receipt-checked transfer is acknowledged; never delete them before verifying the browser import.
7 · Users, sessions and Drive connection
| Field / key | Type and presence | Meaning / example |
|---|---|---|
users[userKey] | User registry record | provider:"google", sub, email, name, picture, userKey (strings); numeric createdAt, updatedAt, lastLoginAt; disabledAt:number|null. onboardedAt is optional, null until completed for new accounts. |
userKey | 32-character string | Current Google key: first 32 hex characters of SHA-256 of issuer + ":" + sub. A data namespace, not a password or an authorization decision. |
legacyOwnerUserKey | string | null · registry envelope | Legacy root ownership mapping; do not reassign old data just because an account is deleted. |
sessions[sessionId] | {user,updatedAt} | Persisted authenticated user snapshot and numeric last activity; runtime also keeps refresh/persistence timing. Legacy string users may be read for compatibility, not created as a new login method. |
googleDriveConnection | object | null | {email,refreshToken,connectedAt,uploadFolderId?}. Refresh token is encrypted at rest; time is numeric, remaining fields strings. No real token examples. |
Google sign-in and Drive consent are separate flows. Store credentials only on the server; browser payloads expose connection status, not refresh tokens. Session IDs and OAuth secrets must never become fixture content in this package.
8 · Media, annotations and canvas
Media metadata
Local records are keyed by safe filename; Drive records by Google file ID. Sparse records are valid. A Drive upload returns the fuller shape below; a listing may derive display fields dynamically rather than persist each field.
| Field / key | Type and presence | Meaning / example |
|---|---|---|
id, name, path | string · source-dependent | For Drive upload records, ID is the Google file ID and path is the display filename. Local identity is a single safe filename. |
source, route | string · source-dependent | For Drive: source:"gdrive", route:"gdrive:<fileId>". Keep source identity when inserting media in notes. |
type, mimeType, size, mtime | string, string, number, number · optional metadata | Media kind/MIME type, byte count, modification time in milliseconds. Large bytes are not embedded in a record. |
folderId | string · optional | Library folder placement; distinguish it from the chosen remote Drive parent folder. |
url, downloadUrl, thumbUrl | string · derived/source-dependent | Authenticated source endpoints; for example /google-drive/raw/<fileId> and /google-drive/thumb/<fileId>. Empty thumbnail URL is allowed. |
playbackSeconds, lastPlayedMs | number · optional | Resume position in seconds and last played time in milliseconds. |
annotations | array · optional | Entries {id:string,time:number,text:string}; integer seconds, text currently limited to 200 characters. |
note | string · optional media annotation | Separate per-media annotation stored on the server. This is not a top-level editor note; do not silently move editor content here. |
Canvas document
| Field / key | Type and presence | Meaning / example |
|---|---|---|
format, id, docId, title | string · normalized canvas | Format is notesandfiles-canvas-v1; docId owns the canvas. |
createdAt, updatedAt | number | Milliseconds. |
elements | array | Every element has id,x,y,w,h. Shape: kind:"shape",shape,text,colour; note link: kind:"noteRef",docId,title; file link: kind:"fileRef",src,mediaKind,name. Coordinates/dimensions are numbers; other listed fields are strings. |
connectors | array | {id,fromId,toId,fromSide,toSide,label,arrow}. Sides: auto/top/right/bottom/left; arrow: none/end/both. |
| Element enums | strings | Shape: rectangle/rounded/ellipse/diamond; colour: neutral/blue/green/amber/rose/violet; mediaKind: audio/video/image. |
Canvas JSON and media annotations are existing server-side content stores with their own privacy scope. They are not covered by a claim that all application data is browser/Drive-only. Do not add a duplicate server copy of editor note content to either collection.
9 · Drive Markdown and encrypted tickets
Drive file format
A note is a readable Markdown file with a single filesandnotes: JSON metadata line inside frontmatter. The encoder removes content, driveRevision, driveFileId and driveReadError from that metadata. Editor blocks have a Base64-encoded JSON node comment plus readable Markdown; this preserves supported structured nodes while allowing external edits.
---
filesandnotes: {"id":"doc-example","title":"Research notes","folderId":"note-folder-example","createdAt":1793000000000,"updatedAt":1793000001000,"trashedAt":null}
---
<!-- filesandnotes:block <Base64 JSON node; schematic placeholder> -->
# Research notes
<!-- /filesandnotes:block -->The placeholder above is schematic, not an importable file. Base64 is not encryption. If a block’s Markdown changes outside the app, the decoder reparses that Markdown; malformed metadata/block markers produce an error rather than an empty overwrite.
Temporary encrypted packet
| Field / key | Type and presence | Meaning / example |
|---|---|---|
Envelope v | number, 1 | Envelope version. |
Envelope iv, tag, data | Base64 strings | AES-256-GCM nonce, authentication tag and ciphertext. Random 12-byte nonce; server-derived encryption key. |
Decrypted identity, status, body, result? | Internal object | Identity is a request fingerprint; status queued/done; body contains upserts[], deletes[], folders[]. Result appears after processing and may contain note content. Only the encrypted envelope is written to the ticket file. |
A ticket is a random 32-character hex ID. Its account-scoped file is deleted when a completed result is consumed. Ticket acceptance confirms receipt, not a successful Drive write. The server can decrypt these packets; this is not end-to-end encryption. See 08 Security for risks and outstanding hardening.
Detailed implementation boundaries
Use the normalized document schema with 11 API Contracts for wire payloads, 13 Markdown Compatibility for supported node attributes/defaults, and 12 Failure & Recovery for lifecycle decisions. The generic structural node schema does not make unknown node types supported. Extension arrays need their own declared validation when selected.
10 · Data contract prompt and acceptance checks
Read prompt/data-structure.html before implementing stores or API payloads.
Create explicit shared record schemas for the chosen scope. Keep logical note IDs, folder IDs, userKey, Drive file IDs and revision strings distinct. Use synthetic fixtures only.
Implement account-scoped browser stores; do not reproduce the legacy browser-only database naming limitation. Preserve optional compatibility fields without creating new product workflows.
Keep note titles, bodies, questions, references and merge bases in the browser and optional user Drive. Persist only the documented technical Drive allowlist on the server. Keep canvas/media annotation storage explicit.
Commit document + pending operation atomically. Clear a queued version only after its matching acknowledgement. Keep tombstones and failed work after reload. Define migrations before changing schema versions.
Verify record round trips, folder integrity, two-account isolation, stale acknowledgements, retry-safe Drive creation, offline reload, deletion races and legacy import acknowledgement. Do not erase user data to make a migration pass.
Report schema decisions, actual test evidence and any unsupported extension fields in build-status.md.- Round-trip the synthetic note and folder without changing IDs, nulls, block order or supported attributes.
- Reject invalid IDs, folder cycles and cross-account access; test browser-only and Drive modes separately.
- Reload offline with pending work, then replay it. A stale token must not clear a newer edit.
- Inspect new server files after saving a unique note sentence and title: neither may appear as readable note content. Inspect technical Drive entries against the allowlist; canvas/media fixtures use their separate scope.
- Verify supported Markdown blocks and external edits, then verify that malformed data cannot overwrite a good note with a blank one.
- Use 06 Verification for the broader product scenarios. Use A01–A12 and synthetic fixtures in 15 Acceptance & Fixtures. These are checks to perform in the generated app, not claims that this documentation edit ran them.