Skip to main content

🔄 Cross-Device Backup & Restore

Follows the durable tutorials (Filecoin steps 5-6) with @lighthouse-ai/cloud-sync - works for batched and memwal.

Why pointers?​

rebuild() re-reads every blob - slow as history grows and impossible when you do not want to list storage at all. Snapshots fix the speed (one CID holds the whole index); the pointer service fixes the bookkeeping (fresh devices discover that CID without you copying it). Together they give you new laptop, same memory in two calls.

Prerequisites​

  • A durable memory from any previous tutorial (Filecoin, Walrus, S3, or memwal) with at least one flushed memory.
  • The pointer client installed: npm install @lighthouse-ai/cloud-sync dotenv.
  • The pointer service URL + auth:
CM_API_URL=https://memory-api.lighthouse.storage
npx -y @lighthouse-ai/cloud-sync login   # once - saves the Clerk JWT to ~/.memory-sdk-cloud/token (0600)

Without CM_API_URL the flow below has no service to talk to; without login the helpers throw Missing Clerk session JWT - pass a dashboard Show + copy session token string as token instead (headless-friendly).

Step 1 - Snapshot the index (one CID for everything)​

const snap = await memory.snapshotIndex()
// batched: { cid, entries, gatewayUrl }
// memwal:  { cid, pinned, entries, gatewayUrl }

This pins the entire local index - entries and pending (batched) - as one blob (mem-index.<namespace>.json or memwal-index.<network>.<namespace>.json). Anyone with the CID can read it (plaintext), but the pointer service only ever stores the CID, never contents.

Run this after remember/flush or on a cron - every backup overwrites the previous pointer (MVP keeps latest per namespace × network).

Step 2 - Back up the pointer​

Helper path (recommended):

import { backupIndex } from '@lighthouse-ai/cloud-sync'

const backed = await backupIndex(memory, {
  apiUrl: process.env.CM_API_URL!,
  namespace: 'demo-filecoin',
  network: 'ipfs-filecoin',   // any string - tracked separately per network
  engine: 'batched',
})
console.log('cloud backup:', backed.pointer)

Raw path (same thing):

await fetch(`${API}/v1/pointers`, {
  method: 'PUT',
  headers: { 'content-type': 'application/json', authorization: `Bearer ${token}` },
  body: JSON.stringify({ snapshotCid: snap.cid, engine: 'batched', namespace: 'demo-filecoin', network: 'ipfs-filecoin' }),
})

Token precedence per call: explicit token → saved token file → error (Missing Clerk session JWT…). The demo passes no token, so it exercises the saved-file path.

Raw HTTP reference (field table, list endpoint, full shapes)

Save with curl (same call the helper makes):

curl -X PUT $API/v1/pointers \
  -H "authorization: Bearer $JWT" \
  -H "content-type: application/json" \
  -d '{"snapshotCid":"baf…","engine":"batched","namespace":"demo-filecoin","network":"ipfs-filecoin","memories":42}'
FieldRequiredConstraintsDefault
snapshotCidYes10–128 chars-
engineNo1–32 charsmemwal
namespaceNo1–128 charsdefault
networkNo1–32 charsmainnet
memoriesNoint 0–10,000,000-
{
  "pointer": {
    "id": "clerk:<sub>",
    "pointerKey": "demo-filecoin#ipfs-filecoin",
    "snapshotCid": "baf…",
    "engine": "batched",
    "namespace": "demo-filecoin",
    "network": "ipfs-filecoin",
    "memories": 42,
    "updatedAt": "2026-07-22T14:59:03.427Z"
  }
}

Pointer key is <namespace>#<network>. MVP keeps the latest CID per key (PUT overwrites). The service stores only the CID — never contents. 400 snapshotCid … required means the CID was missing or outside 10–128 chars.

List everything for the user (filters optional — omit both to list all):

curl "$API/v1/pointers?namespace=demo-filecoin&network=ipfs-filecoin" \
  -H "authorization: Bearer $JWT"
# { "pointers": [ { "snapshotCid": "baf…", … } ] }

Step 3 - Restore on a fresh device (no CID copied)​

import { restoreIndex } from '@lighthouse-ai/cloud-sync'

const viaCloud = new BatchedEngine(await createStorage('lh-ipfs-filecoin', { apiKey }), {
  namespace: 'demo-filecoin', dataDir: viaCloudDir, embedder,
})
const restored = await restoreIndex(viaCloud, { apiUrl, namespace: 'demo-filecoin', network: 'ipfs-filecoin' })
console.log('restore via pointer:', restored.result) // { added, total }
console.log('recall:', await viaCloud.recall('what theme does the user like?'))

restoreIndex does GET /v1/pointers/current?namespace=&network= → rebuildLocal(snapshotCid) (both query params default to default / mainnet when omitted; 404 means no backup yet for that pair).

curl "$API/v1/pointers/current?namespace=demo-filecoin&network=ipfs-filecoin" \
  -H "authorization: Bearer $JWT"
# { "pointer": { "snapshotCid": "baf…", … } }

Merges skip ids the device already has; namespace (and for memwal, network) mismatches throw rather than mixing data.

The demo uses a third directory (fresh-filecoin-cloud) to prove the pointer alone suffices - the CID never appears in the restore code.

Step 4 - Choose your recovery path​

SituationCallNeeds
New device, pointer backed uprestoreIndex()API URL + JWT + storage creds
New device, no pointerrebuild() (batched) / restore() (memwal)Storage / relayer creds only
Have a CID (chat, docs, logs)rebuildLocal(cid)Storage creds + the CID

rebuildLocal is also the manual fallback when the pointer service is unreachable but someone pasted you a snapshot CID.

Troubleshooting​

SymptomFix
No snapshot saved for namespace … yet (404)Run backup first - restore cannot invent a pointer.
Snapshot is for "alice", but this engine is "bob"Namespace mismatch - snapshots never cross namespaces. Align MEMORY_NAMESPACE / constructor namespace.
Memwal Snapshot is for mainnet/…, but this store is testnet/…Network mismatch - creds and snapshots are per-network.
Missing Clerk session JWTnpx -y @lighthouse-ai/cloud-sync login once, or pass token explicitly.
400 snapshotCid … requiredThe CID was missing or outside 10–128 chars — pass the full CID from snapshotIndex().
401 Missing Bearer token / Invalid or expired tokenGet a fresh session token, or re-run login.

Next steps​