Decentralized applications are gradually gaining popularity over the last years. Powered by blockchain technologies like Ethereum, they bring new solutions for problems related to trust and security. This article will present an example of such a problem and show how to solve it using a decentralized solution.
What will we build?
Let’s suppose we want to build a digital assets trading platform, for example, allowing people to exchange files. Our platform should support both sides of the deal. Authors should be able to publish files and receive payments from their purchasers. On the other side, purchasers should gain access to the specific publications they paid for.
Both parties of the transaction also demand a particular level of security. Publishers want to share their files only with customers who paid. All unauthorized use of the published content should be maximally limited. On the other hand, purchasers need to be sure they receive access to the acquired goods once their payments come to the seller.
The usual approach is to design the trading platform as a trusted intermediary. It has to take care of providing correct access to the published content and handle the cashflow between purchasers and publishers.
Such a centralized system is fairly common but has also several drawbacks. Trading parties may not want to pass real money through a platform controlled by an institution they don’t know and don’t trust. Also, a central platform is a single point of failure itself. For example, it could be hijacked by attackers who exploit a vulnerability and steal user funds. This is the moment when a decentralized approach emerges as a promising remedy and we will try to use it in our implementation.
To create our decentralized marketplace, we will use the well known Ethereum platform. Specifically, we will write a smart contract and use it to provide the core system functionality. We will also use IPFS as a distributed storage layer. Last but not least, we will use Go to create a CLI client providing a convenient interaction with the system.
All of the code presented in the examples come from my GitHub repository.
System architecture
As I mentioned above, our system consists of several moving parts. The diagram below presents all of them along with their mutual relations:
Let’s walk through the components in detail. The best way to understand how they interact with each other is to look from the use case perspective.
Publishing a file
The publisher uses its own CLI instance to trigger the publish action. Then, the CLI performs a sequence of steps to make it happen. Specifically:
- Generates a random symmetric key. It will be used as the file access key.
- Encrypts the file using that access key.
- Uploads the encrypted file to IPFS.
- Receives the file CID from IPFS.
- Encrypts the access key using the publisher’s Ethereum public key.
- Publishes the file CID with the encrypted access key and file price to the Ethereum smart contract.
Once the last step is done, the publisher CLI should be switched to the watch mode manually. In that mode, it observes all purchase requests registered by the smart contract and responds to the ones related to its files.
Starting a purchase request
On the other hand, the purchaser uses its own CLI to run the purchase process of the desired file. First, the CLI performs a purchase request consisting of the following steps:
- Submits a purchase request transaction to the Ethereum smart contract. That transaction contains the file CID and the Ethereum public key of the purchaser. It also transfers the required payment to the smart contract.
- Starts observing smart contract events waiting for the publisher’s response.
It’s worth noting that our smart contract acts as an escrow. Once the purchaser transfers the funds, they become locked, under control of the smart contract. To withdraw its payment, the publisher must provide a valid purchase response first.
Finalizing the purchase
To respond to a purchase request, the publisher CLI must be in the watch mode. Once the purchase request is confirmed on-chain, the purchaser CLI begins the response process:
- Downloads the encrypted file access key from the smart contract.
- Decrypts the access key using the publisher’s Ethereum private key.
- Takes the purchaser’s Ethereum public key from the smart contract.
- Encrypts the access key anew. This time using the purchaser’s Ethereum public key.
- Submits the encrypted access key to the smart contract fulfilling the purchase request.
- Withdraws the payment once the purchase response is confirmed on-chain.
The last part is performed by the purchaser CLI awaiting for publisher’s response. Once the publisher submits the response, the CLI performs some last steps to get the file:
- Gets the encrypted file access key, submitted by the publisher, from the smart contract.
- Decrypts the access key using purchaser’s Ethereum private key
- Downloads the encrypted file from IPFS.
- Decrypts the file using the decrypted access key.
- Saves the decrypted file in the target directory.
At this point, the entire process is completed and both parties of the deal should be satisfied. You may notice the described system contains several design simplifications. One of them is the fact the purchaser has no guarantee the file it purchases contains the expected content. The system doesn’t contain any mechanisms allowing the purchaser to get content proof from the publisher. Another simplification is the fact the purchaser could not make a cashback in case the publisher doesn’t respond to the purchase. All simplifications are here to make the system easier for demonstration purposes.
Smart contract implementation
Having our marketplace architecture in mind, let’s dive into the implementation details. I think the best starting point is our smart contract code presented below:
pragma solidity 0.6.2;
import "@openzeppelin/contracts/payment/PullPayment.sol";
contract IpfsMarket is PullPayment {
struct Publication {
// Address of the author.
address author;
// Encrypted author's access key.
bytes authorAccessKey;
// Price in WEIs.
uint256 price;
// List of all purchasers.
address[] purchasers;
// Public keys submitted by the purchasers which should be used
// to encrypt the access key during the purchase answer.
mapping (address => bytes) purchasersPublicKeys;
// Encrypted access keys for purchasers submitted by the author.
mapping (address => bytes) purchasersAccessKeys;
}
mapping (string => Publication) publications;
event PurchaseCreated(
string cid,
bytes publicKey
);
event PurchaseAnswered(
string cid,
address purchaser,
bytes accessKey
);
function publish(
string memory cid,
bytes memory accessKey,
uint256 price
) public {
require(
publications[cid].author == address(0),
"Publication with given CID already exists"
);
Publication memory publication;
publication.author = msg.sender;
publication.authorAccessKey = accessKey;
publication.price = price;
publications[cid] = publication;
}
function getPrice(string memory cid) public view returns (uint256) {
require(
publications[cid].author != address(0),
"Publication with given CID doesn't exists"
);
return publications[cid].price;
}
function purchase(
string memory cid,
bytes memory publicKey
) public payable {
require(
publications[cid].author != address(0),
"Publication with given CID doesn't exists"
);
require(
publications[cid].author != msg.sender,
"Only non-authors can purchase"
);
address purchaser = publicKeyToAddress(publicKey);
require(
purchaser == msg.sender,
"Sender must use their own public key"
);
require(
publications[cid].purchasersPublicKeys[purchaser].length == 0,
"Purchase can be made only once"
);
require(
publications[cid].price == msg.value,
"Incorrect payment amount"
);
publications[cid].purchasers.push(purchaser);
publications[cid].purchasersPublicKeys[purchaser] = publicKey;
emit PurchaseCreated(cid, publicKey);
}
function publicKeyToAddress (
bytes memory publicKey
) private pure returns (address) {
require (
publicKey.length == 64,
"Incorrect public key length"
);
return address(bytes20(uint160(uint256(keccak256(publicKey)))));
}
function answerPurchase(
string memory cid,
address purchaser,
bytes memory accessKey
) public {
require(
publications[cid].author != address(0),
"Publication with given CID doesn't exists"
);
require(
publications[cid].author == msg.sender,
"Only publication author can answer to the purchase"
);
require(
publications[cid].purchasersPublicKeys[purchaser].length != 0,
"Could not answer a non existing purchase"
);
require(
publications[cid].purchasersAccessKeys[purchaser].length == 0,
"Purchase answer can be made only once"
);
publications[cid].purchasersAccessKeys[purchaser] = accessKey;
_asyncTransfer(publications[cid].author, publications[cid].price);
emit PurchaseAnswered(cid, purchaser, accessKey);
}
function hasPurchased(string memory cid) public view returns (bool) {
require(
publications[cid].author != address(0),
"Publication with given CID doesn't exists"
);
return publications[cid].purchasersPublicKeys[msg.sender].length > 0;
}
function getAccessKey(string memory cid) public view returns (bytes memory) {
require(
publications[cid].author != address(0),
"Publication with given CID doesn't exists"
);
if (publications[cid].author == msg.sender) {
return publications[cid].authorAccessKey;
}
return publications[cid].purchasersAccessKeys[msg.sender];
}
function isAuthor(string memory cid) public view returns (bool) {
require(
publications[cid].author != address(0),
"Publication with given CID doesn't exists"
);
return publications[cid].author == msg.sender;
}
}Basic building blocks
The aforementioned contract has several building blocks. The first one is a structure named Publication. This structure represents a published file and holds all pieces of information needed to perform the purchase process.
Next, the smart contract defines several methods. The majority of them are view functions that allow accessing the smart contract state at no cost. Examples of view functions are getPrice or isAuthor.
The most important contract methods are the ones that mutate its state. These methods are publish, purchase, and answerPurchase. Each of them submits a separate Ethereum transaction once invoked.
It’s worth noting that each transaction consumes some amount of gas. Such gas is a measure of the computational effort the network takes to execute all operations included in the transaction. The gas cost is paid by the transaction sender.
You may have also noticed that some of these methods emit dedicated events like PurchaseCreated or PurchaseAnswered. Those events are mostly used by the off-chain clients but they are also useful for debugging purposes.
Payments processing
The most interesting parts of the contract are the ones responsible for payments. When the purchaser wants to buy a publication, it has to invoke the purchase method and pay for the publication within the same transaction. The purchase method has a special payable modifier which allows it to receive payments while being called. The correct amount of the payment is forced by a require condition which checks it against the publication price. Once all preconditions are met, the purchaser’s payment lands on the smart contract account.
At this point, the publisher doesn’t receive any payment yet. First, the publisher has to provide a proper purchase response by calling the answerPurchase method. Once the answer is correct, the contract invokes the _asyncTransfer method which makes the publisher’s payment available for a withdrawal. In the end, the publisher must call the withdrawPayments method to receive funds on its account.
Both _asyncTransfer and withdrawPayments methods are defined in the PullPayment contract provided by the OpenZeppelin library. Our IpfsMarket contract derives from it and gains an implementation of the pull-payment strategy, where the paying contract doesn’t interact directly with the receiver account. This approach is considered as best practice when it comes to sending funds and helps protecting against several threats, for example, reentrancy attacks.
CLI client implementation
We now have an understanding of the Ethereum chain part. To make the system complete, we need a tool that will interact with the smart contract appropriately. This task is handled by the Go client. Let’s look at its package structure first:
The cmd package contains definitions of all commands exposed by the client. Specifically, these commands are publish, watch, and purchase.
Each command invokes a corresponding service from the pkg/service package. Each service implements a concrete use case by orchestrating several basic building blocks defined in the pkg/file package.
Apart from that, the CLI client has two packages responsible for third party integrations. The first one is the pkg/chain which contains the implementation of an Ethereum chain client. That client uses a Go contract binding generated by the abigen tool, to interact with the IpfsMarket contract. The second package is pkg/storage which contains an IPFS client implementation providing the way to interact with the IPFS storage.
Last but not least, there is also the pkg/cipher package which contains implementations of the AES-GCM and ECIES algorithms. The first one is used to encrypt the file content before sending it to IPFS. The latter encrypts the file access key before submitting it to the IpfsMarket contract.
Publish command entry point
Let’s now look at the implementation details of the publish command. In this article, I will describe only one command to keep the post size at a reasonable level. All other commands have the same structure as the described one. If you are interested in the full code, please refer to the GitHub repository.
The entry point of the publish command is defined in the cmd/publish.go file and looks as follows:
package cmd
import (
"fmt"
"github.com/lukasz-zimnoch/ipfs-market/configs"
"github.com/lukasz-zimnoch/ipfs-market/pkg/service"
"github.com/urfave/cli"
"math/big"
)
const defaultFilePrice = 100000000000000000 //0.1 ETH
var PublishCommand = cli.Command{
Name: "publish",
Action: Publish,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "file,f",
Usage: "file name",
},
&cli.Int64Flag{
Name: "price,p",
Value: defaultFilePrice,
Usage: "file price in WEI",
},
},
}
func Publish(c *cli.Context) error {
config, err := configs.ReadConfig(c.GlobalString("config"))
if err != nil {
return err
}
filePath := c.String("file")
filePrice := big.NewInt(c.Int64("price"))
fileCid, err := service.Publish(config, filePath, filePrice)
if err != nil {
return err
}
// pass the file CID to stdout
fmt.Println(fileCid)
return nil
}We use the urfave/cli library to expose the publish command. Then, we tie that command to the Publish function which reads passed flags and the config file. That config file defines several properties needed to perform the process and should be a YAML with the following structure:
ethereum:
url: "<ethereum-node-url>"
privateKey: "<ethereum-private-key>"
ipfsMarketContract: "<ipfs-market-contract-address>"
storage:
url: "<ipfs-node-url>"
workdir: "<working-directory-path>"Sample config files are defined in the configs package. This package also contains a config.go file that defines the ReadConfig function. That function unmarshals the config YAML and maps it to a Go struct which can be used directly in the code.
Publish service
The aforementioned Publish function points directly to the Publish service defined in the pkg/service/publish.go file:
package service
import (
"fmt"
"github.com/lukasz-zimnoch/ipfs-market/configs"
"github.com/lukasz-zimnoch/ipfs-market/pkg/chain"
"github.com/lukasz-zimnoch/ipfs-market/pkg/cipher"
"github.com/lukasz-zimnoch/ipfs-market/pkg/file"
"github.com/lukasz-zimnoch/ipfs-market/pkg/storage"
"math/big"
)
func Publish(
config *configs.Config,
filePath string,
filePrice *big.Int,
) (string, error) {
key, err := cipher.GenerateSymmetricKey()
if err != nil {
return "", fmt.Errorf("could not generate symmetric key: [%v]", err)
}
aesGcmCipher, err := cipher.NewAesGcm(key)
if err != nil {
return "", fmt.Errorf("could not create AES-GCM cipher: [%v]", err)
}
ipfsStorage := storage.NewIpfs(config.Storage.URL)
uploader := file.NewUploader(aesGcmCipher, ipfsStorage)
fileCid, err := uploader.Upload(filePath)
if err != nil {
return "", fmt.Errorf("could not upload file: [%v]", err)
}
ethPrivateKeyBytes, err := chain.DecodeEthPrivateKey(config.Ethereum.PrivateKey)
if err != nil {
return "", fmt.Errorf("could not decode ethereum private key: [%v]", err)
}
eciesCipher := cipher.NewEcies(ethPrivateKeyBytes)
ethereum, err := chain.NewEthereumClient(
config.Ethereum.URL,
ethPrivateKeyBytes,
config.Ethereum.IpfsMarketContract,
)
if err != nil {
return "", fmt.Errorf("could not create ethereum client: [%v]", err)
}
publisher := file.NewPublisher(eciesCipher, ethereum)
err = publisher.Publish(fileCid, key[:], filePrice)
if err != nil {
return "", fmt.Errorf("could not publish file: [%v]", err)
}
return fileCid, nil
}That service performs its main task by leveraging several building blocks. First, it generates a 32 bytes symmetric key using the cipher.GenerateSymmetricKey function. Under the hood, that function uses a cryptographically secure random number generator from the crypto/rand package.
Then, our service uses that symmetric key to create an instance of the AES-GCM cipher. As the next step, the service obtains an IPFS storage handler using the node URL provided in the config file. Both components are then injected into the file.Uploader which accepts them as implementations of the Cipher and Storage interfaces respectively. These components are used inside the file.Uploader as follows:
package file
import (
"fmt"
"io/ioutil"
)
const maxFileByteSize = 1048576
type Uploader struct {
cipher Cipher
storage Storage
}
func NewUploader(cipher Cipher, storage Storage) *Uploader {
return &Uploader{
cipher: cipher,
storage: storage,
}
}
func (u *Uploader) Upload(filePath string) (string, error) {
logger.Infof("starting uploading file [%v]", filePath)
fileBytes, err := ioutil.ReadFile(filePath)
if err != nil {
return "", fmt.Errorf("could not read file [%v]: [%v]", filePath, err)
}
if len(fileBytes) > maxFileByteSize {
return "", fmt.Errorf(
"file [%v] exceeds maximum file size of [%v] bytes",
filePath,
maxFileByteSize,
)
}
logger.Infof(
"file [%v] has been read successfully; size: [%v] bytes",
filePath,
len(fileBytes),
)
fileEncryptedBytes, err := u.cipher.Encrypt(fileBytes)
if err != nil {
return "", fmt.Errorf("could not encrypt file [%v]: [%v]", filePath, err)
}
logger.Infof(
"file [%v] has been encrypted successfully; "+
"size after encryption: [%v] bytes",
filePath,
len(fileEncryptedBytes),
)
fileCid, err := u.storage.Store(fileEncryptedBytes)
if err != nil {
return "", fmt.Errorf("could not store file [%v]: [%v]", filePath, err)
}
logger.Infof(
"file [%v] has been stored successfully; "+
"content identifier: [%v]",
filePath,
fileCid,
)
return fileCid, nil
}As we can see, the Uploader reads the file from disk, encrypts it using the provided Cipher, and stores the encrypted content using the Storage handler. In the end, the Uploader returns the file CID to the Publish service.
Having the file CID, our service continues its work by creating an ECIES cipher instance using the Ethereum private key taken from the config. It also creates an Ethereum chain handle using the provided node URL, private key, and IPFSMarket contract address.
In the last step, the Publish service injects the ECIES cipher and the Ethereum handle into the file.Publisher which takes them as implementations of the Cipher and Chain interfaces respectively. Then, our service calls the publisher.Publish method with the file CID, file access key, and the file price as arguments. Internals of the file.Publisher are presented below:
package file
import (
"fmt"
"math/big"
)
type Publisher struct {
cipher Cipher
chain Chain
}
func NewPublisher(cipher Cipher, chain Chain) *Publisher {
return &Publisher{
cipher: cipher,
chain: chain,
}
}
func (p *Publisher) Publish(cid string, accessKey []byte, price *big.Int) error {
logger.Infof("starting publishing CID [%v]", cid)
encryptedAccessKey, err := p.cipher.Encrypt(accessKey[:])
if err != nil {
return fmt.Errorf("could not encrypt key: [%v]", err)
}
transactionHash, err := p.chain.Publish(cid, encryptedAccessKey, price)
if err != nil {
return fmt.Errorf("could not publish CID [%v]: [%v]", cid, err)
}
logger.Infof(
"CID [%v] has been published successfully; "+
"transaction hash on-chain: [%v]",
cid,
transactionHash,
)
return nil
}In the beginning, the file.Publisher encrypts the file access key using the provided Cipher. Next, it uses the Chain handle to invoke the Publish method along with the file CID, encrypted access key, and file price as arguments. In our case, this action calls the publish method from the IPFSMarket contract.
That’s all, the file has been published and can be the subject of the purchase process.
Conclusions
In this post, I have described a sample digital marketplace project which shows practical usage of decentralized platforms like Ethereum and IPFS. I hope this article will help to understand the pros and cons of decentralization and will inspire you to learn more about blockchain-based technologies. I also strongly encourage you to check the full code in my GitHub repository.
