Skip to main content

Upload Backup Data and Manage Snapshots

Once you have a dump on disk, Backup() (Go), backup() (JS), or baas backup (CLI) uploads it as a new snapshot. The same client manages the full snapshot lifecycle: list, inspect, prune, delete, and restore.

import (
    sdkclient "github.com/lighthouse-web3/baas-go-sdk/client"
    sdktypes  "github.com/lighthouse-web3/baas-go-sdk/types"
)

All snippets assume an authenticated client (see Authentication).

Upload a backup​

Call Backup() on the dump file or the whole db-dumps directory. For daily jobs, keep one stable dump path (for example ./db-dumps/app.dump or ./db-dumps/app.sql) and overwrite that file each run before calling Backup().

snapshot, err := client.Backup([]string{"./db-dumps"}, &sdktypes.BackupOptions{
	Description: "nightly database backup",
	Tags: map[string]string{
		"type": "db-backup",
		"env":  "prod",
		"db":   "app_db",
	},
	Hostname: "backup-runner-01",
})
if err != nil {
	log.Fatal(err)
}

log.Printf("snapshotId=%s totalSize=%d chunks=%d", snapshot.SnapshotID, snapshot.TotalSize, snapshot.TotalChunks)

Tip: using a fixed filename in the dump directory enables dedup-friendly, incremental uploads while still creating a fresh snapshot each run.

Watch live progress (optional)​

opts := &sdktypes.BackupOptions{
	Description: "nightly",
	OnProgress: func(e sdktypes.ProgressEvent) {
		log.Printf("[%s] %d/%d stored=%dB ratio=%.2f",
			e.Phase, e.Current, e.Total, e.StoredBytes, e.CompressionRatio)
	},
}
snapshot, err := client.Backup([]string{"./db-dumps"}, opts)

Useful BackupOptions​

FieldMeaning
DescriptionHuman label for the snapshot.
TagsArbitrary map[string]string metadata.
HostnameOverride the recorded hostname (defaults to the machine's).
SourceIDLogical source identity. Leave empty and the SDK manages it (see below).
ParentSnapshotIDHint the previous snapshot for faster incremental diffing.
DisablePackCompressionTurn off zstd compression for already-compressed data.
EncryptionEnable client-side encryption — see Encryption.
WorkspaceIDBack up into a different workspace for this call only.

Sources & SourceID​

If you don't set SourceID, the SDK creates/reads a stable id from a .lighthouse/source_id file inside the backup target path. This is what lets the portal group every snapshot of the same machine+path under one Backup Source. Keep that .lighthouse directory and successive backups line up as a coherent history; set an explicit SourceID only if you want to control grouping yourself.

Verify in the portal: open Backup Sources → your source → the new snapshot is at the top of the list. See View backups & snapshots in the portal.

List recent snapshots​

resp, err := client.ListSnapshots("", 20) // cursor, limit
if err != nil {
	log.Fatal(err)
}
for _, s := range resp.Snapshots {
	log.Printf("id=%s createdAt=%d desc=%s size=%d", s.SnapshotID, s.CreatedAt, s.Description, s.TotalSize)
}
// resp.Cursor (if non-nil) → pass back as the cursor arg to get the next page.

Inspect one snapshot​

s, err := client.GetSnapshot("snapshot-id")
if err != nil {
	log.Fatal(err)
}
log.Printf("rootTreeHash=%s paths=%v tags=%v encrypted=%v",
	s.RootTreeHash, s.Paths, s.Tags, s.Encryption != nil)

Prune snapshots (retention)​

Pruning applies a retention policy in one call — keep the latest N and/or drop everything older than a cutoff. Always dry-run first.

import "time"

keep := 14
before := time.Now().AddDate(0, 0, -30).UTC().Format(time.RFC3339) // older than 30 days

// Step 1: DRY RUN — see what would be deleted, delete nothing.
preview, err := client.PruneSnapshots(sdktypes.PruneRequest{
	KeepLatest: &keep,
	Before:     before,
	DryRun:     true,
})
if err != nil {
	log.Fatal(err)
}
log.Printf("would delete %d snapshot(s): %v", preview.Count(), preview.SnapshotIDs)

// Step 2: execute for real.
result, err := client.PruneSnapshots(sdktypes.PruneRequest{
	KeepLatest: &keep,
	Before:     before,
	DryRun:     false,
})
if err != nil {
	log.Fatal(err)
}
log.Printf("pruned %d snapshot(s)", result.Count())

How retention combines: when both are set, a snapshot is deleted only if it is both outside the keepLatest newest set and older than before. Set just one for a simpler policy. PruneResponse.Count() (Go) / pruneCount(response) (JS) returns candidates on a dry run and deletions on a real run.

Delete one snapshot​

err = client.DeleteSnapshot("snapshot-id")
if err != nil {
	log.Fatal(err)
}

This deletes the snapshot record; content-addressed chunks still referenced by other snapshots are retained automatically. It is irreversible — confirm the id with getSnapshot first if you are scripting deletions.

Restore a snapshot (recovery drill)​

Restoring downloads the snapshot's chunks, (decrypts →) decompresses → verifies integrity against expected hashes, then reassembles the original files into a target directory. A recovery drill = restore into a scratch directory and confirm the contents, without touching production data.

import (
	"os"
	"path/filepath"
)

target, _ := os.MkdirTemp("", "lh-restore-drill-*")
log.Printf("restoring into %s", target)

err = client.Restore("snapshot-id", target, &sdktypes.RestoreOptions{
	OnProgress: func(e sdktypes.ProgressEvent) {
		log.Printf("[%s] %d/%d", e.Phase, e.Current, e.Total)
	},
})
if err != nil {
	log.Fatal(err)
}

// Sanity-check the restored tree.
_ = filepath.Walk(target, func(p string, info os.FileInfo, err error) error {
	if err == nil && !info.IsDir() {
		log.Printf("restored: %s (%d bytes)", p, info.Size())
	}
	return nil
})

Then restore the database from the dump inside the scratch directory you restored into — $target in the Go/JS examples above, or ./restore-check in the CLI example:

  • PostgreSQL: pg_restore ... "$RESTORE_DIR/db-dumps/app.dump"
  • MySQL: mysql ... < "$RESTORE_DIR/db-dumps/app.sql"
  • Other databases: see the matching tutorial.

Recovery-drill checklist:

  1. ✅ Restore into a fresh, empty directory (never over live data).
  2. ✅ Leave checksum verification on (don't set SkipChecksumVerification).
  3. ✅ Confirm file count, sizes, and spot-check contents.
  4. ✅ Note the wall-clock time — that's your real RTO (recovery time).
  5. ✅ Delete the scratch directory when done.

Run this drill on a schedule (e.g. monthly) so you find restore problems before a real outage.

Client-side encryption (optional)​

With encryption enabled, data is encrypted on your machine (AES-GCM) before upload; the server only ever stores ciphertext and a wrapped key. You hold a passphrase-protected keyfile (the Tenant Master Key, TMK). Lose the keyfile/passphrase → the data is unrecoverable — there is no server-side reset.

import "github.com/lighthouse-web3/baas-go-sdk/encrypt"

// Create a keyfile once.
_, err := encrypt.GenerateKeyfile("/secure/lh.keyfile", os.Getenv("LH_KEYFILE_PASSPHRASE"), nil)
if err != nil {
	log.Fatal(err)
}

enc := &sdktypes.EncryptionOptions{
	KeyfilePath: "/secure/lh.keyfile",
	Passphrase:  os.Getenv("LH_KEYFILE_PASSPHRASE"),
}

// Backup with encryption.
snapshot, err := client.Backup([]string{"./db-dumps"}, &sdktypes.BackupOptions{
	Description: "nightly-encrypted",
	Encryption:  enc,
})

// Restore an encrypted snapshot — supply the same keyfile/passphrase.
err = client.Restore("snapshot-id", target, &sdktypes.RestoreOptions{
	Encryption: enc,
})

A snapshot's Encryption / encryption field (set) tells you whether it was encrypted. The SDK also supports TMK rotation (client.RotateTMK(...) in Go, client.rotateTMK(...) in JS) to re-wrap a snapshot's data key under a new master key without re-encrypting any data blobs.