Step 1: Upload your first file using Lighthouse SDK
Firstly, you'll need to get a picture of your favorite pupper whose picture you'll want to store on the decentralized web.
The Lighthouse SDK is a JavaScript library that allows you to upload files to the Filecoin network. It's open source and available here
A. Uploading a file is as simple as:
import lighthouse from"@lighthouse-web3/sdk";// ... other codeconstuploadResponse=awaitlighthouse.upload('/path/to/adorable/dog.jpg','YOUR_API_KEY');
Previously, if the file of your puppy was too small, it would encounter issues being stored on the chain due to size minimums enforced by on-chain deal makers. The SDK helps you get around this by adding mock data (in deal parameters below) to your file to meet the minimum size requirements.
B. To upload a file with replication:
Replication is the process of making multiple copies of your file and storing them on the Filecoin network. This ensures that if one storage provider goes down, you'll still be able to retrieve your file from another storage provider.
import lighthouse from"@lighthouse-web3/sdk";// ... other code// Indicates that the deal to maintain your file will be replicated to a total of two copies on the network.constdealParams= { num_copies:2,};// The `false` indicates that we're uploading a single file.// Returns a CID (Content ID) for your file that you can use for PoDSI verification.constuploadResponse=awaitlighthouse.upload('/path/to/adorable/dog.jpg','YOUR_API_KEY',false, dealParams);
Step 2: Set Deal Parameters
Note: Deal parameters are currently supported on the Calibration testnet. If you don't specify deal parameters, then deal is made on Filecoin mainnet
When uploading a file, you can customize how it's stored in Lighthouse using the deal parameters:
num_copies: Decide how many backup copies you want for your file. The Max limit is 3. For instance, if set to 3, your file will be stored by 3 different storage providers.
repair_threshold: Determines when a storage sector is considered "broken" if a provider fails to confirm they still have your file. It's measured in "epochs", with 28800 epochs being roughly 10 days.
renew_threshold: Specifies when your storage deal should be renewed. It's also measured in epochs.
miner: If you have preferred miners, list their addresses here. For testing, it's recommended to use t017840.
network: This should always be set to 'calibration' (for RAAS services to function) unless you want to use the mainnet.
add_mock_data: This field is used to make smaller files reach the minimum file size accepted on the Lighthouse calibration test network (1 MB). If your file is less than the minimum size, add_mock_data will append a mock file to ensure it meets the storage requirements. The value indicates the size in MB. For instance, if your file is 256KB, the add_mock_data should be set to 2 to the minimum target.
The term "epoch" can be thought of as a time unit in the filecoin network under which various operations occur, like PoST, PoRep, etc., with 2880 epochs equivalent to a day.
constpath="/path/to/file.jpg"constapiKey="thisisaateststring"constdealParam_default= {"network":"calibration"}// adds mock data for satisfying minimum file sizeconstdealParam_mock= {"add_mock_data":4,"network":"calibration"}// To ignore a deal parameter set it as nullconstdealParam_ignore= {"replication_num_copies":null,"repair_threshold":null,"renewal_threshold":null,"network":"calibration"}// Default parameters set. All RaaS workers enabled, any miners can take the deal. 2 MiB mock file added.constresponse=awaitlighthouse.upload(path, apiKey,false,dealParam_default);//this should be used if the user wants to bundle in a 4MiB mock file with their user submission.constresponse=awaitlighthouse.upload(path, apiKey,false, dealParam_mock);//this needs to be used by the self hosted RaaS module, and the aggregator SDK after the event gets emitted. Turns off all RaaS workers. 2 MiB mock file added.
constresponse=awaitlighthouse.upload(path, apiKey,false, dealParam_ignore);
Step 3: Understanding PoDSI: Getting the PoDSI for your file
Now that you've registered the picture of your puppy, how would you know that it's actually being maintained on the Filecoin network? This is where the PoDSI comes in. The PoDSI is a proof that your file is being maintained on the Filecoin network.
Proof of Data Segment Inclusion (PoDSI) is like a certificate of authenticity. It assures that your file is safely tucked inside a special package, known as a "deal", made by the Lighthouse Node. This node combines several files, gives them a unique ID, offers proof of their inclusion, and even throws a mini-proof of the entire package's structure.
The time between uploading and being able to get your PoDSI should only be a few minutes. You can get the PoDSI for your file by calling the getProof function in one of the following ways:
via Axios in node.js
let response =awaitaxios.get("https://api.lighthouse.storage/api/lighthouse/get_proof", { params: { cid: lighthouse_cid, network:"testnet"// Change the network to mainnet when ready }})
or via curl
# Assumes that uploaded your file to mainnet.# Alternatively, if you are using testnet, add &network=testnet to the end of the URL.curlhttps://api.lighthouse.storage/api/lighthouse/get_proof?cid=<puppy_CID>
curl example:
# An example of how to get the PoDSI for a file uploaded to testnetcurl https://api.lighthouse.storage/api/lighthouse/get_proof?cid=QmYTaCnjNrrKCwXzC8ZLiiNJ78rsobXtfKwN8s9qCLBzVA&network=testnet
The response, an example of a PoDSI proof on Calibration, should look something like this:
The proof contains information that can be used to confirm whether your file was included in a specific aggregated data bundle.
The dealInfo provides details about the file's storage deal. If the "dealId" is null, it means that the storage deal has been initiated but the miner hasn't started the sealing process yet.
Step 4: Get your deal ID of your upload
When you upload the picture of your puppy, the on-chain deal that is made to store it on the Filecoin network is assigned a unique deal ID. You can get this deal ID from the PODSI response above.
Under the hood, the node infrastructure is working hard to ensure that your file is included on-chain. The process of deal making can take up to about a day.
Step 5: Download your file using the fileโs CID
Now that your file is stored on the Filecoin network, you can retrieve it using its CID. You can do this by calling the download function in one of the following ways:
via CLI:
# Assumes that you have lighthouse-cli installed. If not, feel free to download it using # npm install -g @lighthouse-web3/sdkcurl-ofileNamehttps://gateway.lighthouse.storage/ipfs/<cid>
or via Code:
constlighthouseDealDownloadEndpoint= https://gateway.lighthouse.storage/ipfs/'let response =awaitaxios({ method:'GET', url:`${lighthouseDealDownloadEndpoint}${lighthouse_cid}`, responseType:'stream',});try {constfilePath=awaitthis.saveResponseToFile(response, downloadPath);console.log(`File saved at ${filePath}`);return filePath} catch (err) {console.error(`Error saving file: ${err}`);}saveResponseToFile(response, filePath) {constwriter=fs.createWriteStream(filePath);// Pipe the response data to the fileresponse.data.pipe(writer);returnnewPromise((resolve, reject) => {writer.on('finish', () =>resolve(filePath));writer.on('error', (err) => {console.error(err);reject(err); }); });}
You can also work with PODSI, and deal making on-chain with help of smart contracts which we will discuss thoroughly in next section.