💾 Storages

Storages decide where the batched engine puts bytes. (Memwal owns its blobs - no storage choice.) Pick with MEMORY_STORAGE or createStorage(kind, opts).

import '@lighthouse-ai/store-lighthouse'
import { createStorage } from '@lighthouse-ai/core'

const storage = await createStorage('lh-ipfs-filecoin', { apiKey: process.env.LIGHTHOUSE_API_KEY })
// 'lh-ipfs-walrus' | 'walrus' alias | 'filecoin' alias | 's3' | 'local-fs'

Comparison

	lh-ipfs-walrus	lh-ipfs-filecoin (default)	s3	local-fs
Backing	Walrus blobs on Sui	IPFS + Filecoin deals	AWS S3 / R2 / MinIO	Local folder
Needs	LIGHTHOUSE_API_KEY (Sui-wallet key)	LIGHTHOUSE_API_KEY (any)	S3_BUCKET (+ endpoint/creds)	dir or MEMORY_DIR (offline)
Upload endpoint	upload-walrus.lighthouse.storage	upload.lighthouse.storage	your bucket/endpoint	-
Gateway	gateway-walrus.lighthouse.storage	gateway.lighthouse.storage	S3_PUBLIC_BASE_URL or s3://bucket/prefix/blobs/<cid>.json	file://…
Quota accounting	~63 MB per blob (erasure coding)	Actual file size	Your bucket	Your disk
Deletable / listable	Yes / Yes	Yes / Yes	Yes / Yes	Yes / Yes
Walrus blob IDs	✅ (storage.getBlobIds(cid))	Returns []	Returns []	Returns []

Short aliases walrus and filecoin (plus bare lighthouse, which follows MEMORY_NETWORK/MEMORY_STORAGE) are accepted anywhere a storage kind is.

local-fs - offline folder

import '@lighthouse-ai/store-local-fs'
const storage = await createStorage('local-fs', { dir: './blobs' })

• Needs dir (or MEMORY_DIR). No keys, no network.
• Keeps blobs plus CID pointer files (<cid>.ptr.json → fileName) under the state dir. uploadJson() computes the same CIDv1 raw + sha2-256 as the other adapters and returns { cid, gatewayUrl, size } — same content = same CID.
• forget() is a hard delete; getBlobIds() returns [].

Use it for tests and offline dev: MEMORY_STORAGE=local-fs MEMORY_EMBEDDER=keyword.

s3 - AWS S3 / R2 / MinIO

import '@lighthouse-ai/store-s3'
const storage = await createStorage('s3', {}) // reads S3_* from env

Env var	Purpose
S3_BUCKET	Required. Bucket name
S3_REGION / AWS_REGION	Region (e.g. us-east-1)
S3_ENDPOINT	Custom endpoint (R2: https://<account>.r2.cloudflarestorage.com; MinIO: http://localhost:9000)
S3_FORCE_PATH_STYLE	1 for MinIO / R2
S3_PREFIX	Key prefix (default memory-sdk)
S3_PUBLIC_BASE_URL	CDN base for gatewayUrl() (else s3://bucket/prefix/…)
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY	Credentials (MinIO defaults work locally)

Keys are content-addressed: <prefix>/blobs/<cid>.json plus <prefix>/names/<fileName> alias (default prefix memory-sdk). uploadJson() computes the same CIDv1 raw + sha2-256 as the other adapters and returns { cid, gatewayUrl, size } — same content = same CID. forget() is a hard delete (alias always removed; blob removed when unreferenced). getBlobIds() returns [].

lh-ipfs-walrus / lh-ipfs-filecoin - Lighthouse

import '@lighthouse-ai/store-lighthouse'
const walrus = await createStorage('lh-ipfs-walrus', { apiKey })
const filecoin = await createStorage('lh-ipfs-filecoin', { apiKey })

• Both need LIGHTHOUSE_API_KEY (get one at Files App). Walrus keys must come from a Sui-wallet login.
• MEMORY_NETWORK selects the gateway variant for bare lighthouse kind.
• forget() stops renewal - content stays readable until the period expires, then is reclaimed.
• Only lh-ipfs-walrus returns Walrus-native blob IDs:

await storage.getBlobIds(cid) // walrus: ['…']; filecoin/s3/local-fs: []

Blobs are content-addressed: same content = same CID. CIDs are CIDv1 raw + sha2-256 and must stay stable - changing the scheme invalidates stored links.

Capabilities

Every adapter exposes:

storage.name // 'lh-ipfs-walrus' | 'lh-ipfs-filecoin' | 's3' | 'local-fs'
storage.capabilities // { deletable, listable, blobIds }
storage.gatewayUrl(cid)
await storage.uploadJson(value, fileName)
await storage.fetchJson(cid)
await storage.listAllFiles() // [{ id, cid, fileName, createdAt }]
await storage.deleteFile(fileId)
await storage.getBlobIds(cid)

Use listAllFiles() + fileName.startsWith('mem-batch.<namespace>.') when implementing custom rebuild logic - that is exactly what BatchedEngine.rebuild() does.