Logo kremalicious

The Commons Marketplace in Pacific Network

We’ve been busy over the last few months, further developing the Commons Marketplace alongside creating the Pacific PoA Network.

The Commons Marketplace in Pacific Network

This article was originally posted on Medium in the Ocean Protocol blog.

When we started with the Commons Marketplace, we built it on top of our Nile Testnet. Besides showcasing the technology behind Ocean Protocol, it helped us to verify what we had built as well as to further improve on the Ocean components. Building our own reference marketplace with a specific use case, combined with the latest developed underlying technology, proved to be valuable in spotting and fixing the issues we were running into in Nile while building the Pacific Mainnet.

Since then, we’ve had the chance to analyze real-world usage of the marketplace, giving us valuable insights into expectations of the users and developers that are utilizing Ocean Protocol technology.

🏄‍♀️ Consume Flow Improvements

Downloading an asset file, a process we call consuming, is at the heart of what Ocean Protocol offers to ensure control of a registered data set by the publisher. Only under the conditions set by the publisher, a consumer will be allowed to do so. Revealing those conditions requires entering a cryptographic signing process first, no matter the conditions.

When an asset is registered into Ocean Protocol, the file URLs are encrypted and the encrypted URLs are stored in the data set metadata, the unencrypted URLs are never stored. Only entering the consume flow will decrypt and unveil these file URLs. This process proved to be unstable and often took a long time to complete in Nile.

We tweaked all of the relevant Ocean Protocol components in the background (Aquarius, Brizo, Secret Store, Squid-js) to be able to provide a faster consume flow. We also added more real-time feedback during the whole process in the UI. In combination, these steps introduced a much more stable consume flow and a dramatic decrease in real and perceived time required to download an asset in Pacific.

Granular feedback messages and faster process than before. Multiple months of work for one GIF.

It is still not as fast as we would like it to be, and we will continue to make improvements on the speed over time.

🔥 Burner Wallets: Commons Without Any Setup

Since the launch of Commons, a Web3-capable browser with a wallet setup was required before full functionality became available. Without a wallet, and without a Web3-capable browser, the search feature was possible, but publishing or downloading data sets were not, because those processes require a cryptographic signing process — something which can only be done with a Web3 wallet.

We found this to be one of the main pain points users had when interacting with a marketplace providing free and publicly available data. Using the right browser, relying on a browser plugin, setting up a wallet, possibly managing crypto tokens, and connecting to the right network proved to be tricky for the target groups we’d like to have using Ocean.

We’ve started adding some improvements to remedy this, and so that Ocean developers will be able to debug more quickly:

New Ocean Versions component on commons.oceanprotocol.com/about

But telling users to read some version numbers and act on it can’t be the solution.

We went back to the drawing board, created some concepts and prototypes, and in the end built a solution providing the smoothest user experience by eliminating any setup: introducing the so-called burner wallet concept. If you just want to browse and download some assets, this solution is all you need and you don’t even have to think about it.

What Is a Burner Wallet

A burner wallet is a frictionless but disposable wallet, often used for signing transactions only, or to move small amounts of tokens around. They are usually hierarchical deterministic wallets (HD wallet), generating a set of public/private keys from a random seed phrase. But the keys are not protected as much as in a “normal” wallet, and in our case the seed phrase is stored in your browser’s cache, hence the low-security aspect.

It's not about money, it's about sending a message

Luckily, this doesn’t matter for interactions in the Commons Marketplace, where all assets are free and don’t require any transfer of tokens. All that is required to interact with all Ocean Protocol features in Commons is a wallet for signing during the register & consume requests.

Conceptually, a burner wallet’s private keys are supposed to be destroyed (“burned”) after the task the wallet has been created for is finished, hence the name burner wallet. Technically, we do not use the original burner wallet concept (that is, running against the xDai sidechain), but make use of the excellent principle: a disposable auto-generated wallet, ready to go without any user interaction.

Meet The New Default

Any visitor to the Commons Marketplace will be switched to a burner wallet by default. In the background, we create a wallet in your browser from a random mnemonic with Truffle’s HD Wallet and pass that wallet to Web3 as the provider. We then load its first generated account and also send some Pacific ETH (not to be confused with Ethereum Mainnet ETH) from our Faucet to that account, in order to cover eventual Gas costs for publishing and consuming.

Extended Web3 status component with burner wallet

We try to give you some persistence across sessions though, by storing that seed phrase in your browser’s localStorage, and on next visit we’ll generate your wallet again from this seed phrase, so you’ll get the same account again. This persistence is needed so we can provide you with some basic level of personalization, e.g. the History page will show the assets published by you, meaning by the Web3 account you are using.

Phew, that’s a lot of tech going on in the background — so let’s get back to the UI! All those tasks are moved aside and are not visible to end users. They essentially transform the previous view where consuming was not possible without a Web3 browser or MetaMask:

Non-Web3 capable browser

Into this view, unlocking the consume flow even for non-Web3 capable browsers without any user action required:

Non-Web3 capable browser with burner wallet

Disclaimer: Being low-security and disposable by design, a burner wallet should not be used if you want to keep track of all your published and downloaded assets, or if you want to hold tokens in it. We strongly suggest not using any burner wallet generated in Commons to move or hold any tokens. The burner wallet only exists in your browser and is geared towards usage in Commons only. We don’t ever store any account information so we are not able to restore a lost burner wallet.

MetaMask for the Pros

If you require more security, you can optionally switch to using MetaMask with an account that only you control. If we detect your browser is able to use MetaMask (i.e. is Web3-capable), we show an action for switching the wallet you’re using in Commons:

Wallet status

In the wallet selector you can switch to MetaMask:

Choose your preferred ride.

After selecting MetaMask, the UI switches to your first account in it. If you use MetaMask for the first time with Commons, you will be prompted by MetaMask to approve reading access to your accounts:

MetaMask approval

Approving this request results in your first MetaMask account being selected, displayed and ready to use in Commons:

Wallet status with MetaMask

We also store your selection and, on your next visit, we switch to MetaMask automatically.

Additionally, we allow you to export your burner wallet by providing the seed phrase you can use to import the wallet into MetaMask:

Seed phrase for exporting a burner wallet

This is helpful if you have already made some interactions in Commons, published and consumed some assets, and now want to keep your history and transactions in an account you control.

🌊 From Nile Testnet to Pacific Mainnet

Since its launch, the commons.oceanprotocol.com interface was pointed towards Nile components so all assets registered through it were living in Nile. But there is no way to simply move a whole data catalogue (which is a combination of metadata stored in Aquarius & Brizo) from one Ocean network to the other.

So we had no other choice other than re-seeding all existing assets stored in Nile into Pacific. This means all assets got a new DID, and some assets might have been lost in the transition. But the old data catalogue in Nile is still accessible under https://commons.nile.dev-ocean.com.

After we registered all Commons assets again in Pacific, we switched the commons.oceanprotocol.com interface to point to Pacific, showing the new data catalogue.

🧪 Commons In Multiple Networks

With Pacific running alongside our Testnets, we needed to add changes so we could deploy the Commons front-end in any of our networks, and everything would just work as intended without further configuration or introduction of a whole bunch of environment variables. The interaction and feedback when using MetaMask needed an overhaul for that, and we also improved our Ocean network detection, now properly differentiating between them and adopting UI copy accordingly.

This all provides much more helpful feedback in case of network & account misconfigurations, e.g. if you use MetaMask and are connected to the wrong network:

Wallet status with wrong network

For this check we fetch the network from the Brizo endpoint configured in Commons client, then check that against the network you’re connected to in MetaMask. This allows Ocean devs to deploy the front-end into any network, and it will figure out the correct network combination automatically, and show relevant help messages.

We consider the Pacific installation to be the main live environment, but have Commons deployed in our Testnets as well for, well, testing purposes:

They all use a different Aquarius & Brizo instance in different networks in the background, so they all have a different data catalogue.

As a developer, this also means you can use the Nile & Duero installations as a front-end for testing asset registering. You can register assets programmatically with our Squid libraries pointed at the respective Aquarius & Brizo instances in those networks. Then check out the registered asset in Commons in the respective network. Note that all Commons installations are restricted to free assets, assets with a set prize are filtered out and won’t be displayed.

The Ocean Versions component in those Commons installations under their About pages allows you to see which endpoints are in use:

Ocean component endpoints in use by Commons in Nile

And you can use the Commons client & server locally on your machine to fully customize it for your Ocean testing purposes, and as a blueprint to build your own marketplace on top of Ocean Protocol.

📡 Channels: AI for Good

A feature to enhance data discoverability and introduce a first basic curation mechanism. Channels are curated collections of existing data sets from multiple categories, showing them in one prominent view.

Our first channel is AI For Good, created as part of the AI for Good Global Summit 2019 in Geneva. AI for Good is an initiative to promote the use of artificial intelligence for taking action on the United Nations Sustainable Development Goals (SDGs) such as fighting poverty, climate change, improving healthcare, etc. The AI for Good Commons channel is aimed at helping scale AI for Good by enabling relevant open datasets to be published and shared and connecting AI problem owners with problem solvers. All curated by the Ocean Protocol team.

AI For Good as a Featured Channel on commons.oceanprotocol.com

Think you have an asset which would fit here? Let us know @oceanprotocol.

In the background the feature works with tags, which are part of asset metadata defined in OEP-8. But they are not exposed in the UI yet, making adding the required tag a manual process.

⛲️ All of the Features From Before

With all those described changes above, we’ve made Ocean Protocol easier and more reliable to use. Commons allows you to access thousands of datasets for free that have been registered on the Ocean network. If you are looking for quality data, you can easily use the Commons to search for and find publicly available datasets that are free of charge.

If you are interested in sharing your data, you can use the Commons to publish data into the Ocean Protocol Network.

If you have large amounts of data that you are interested in sharing, please contact us about supporting you to perform bulk data registration.

Learn more about the Commons Marketplace

This article was originally posted on Medium in the Ocean Protocol blog.


Have a comment?

Hit me up @krema@mas.to


Found something useful?

Say thanks with BTC or ETH


Edit on GitHub

Contribute to this post