As Web3 becomes more and more popular, the industry is seeing an increase in interest from both retail users and institutions. The trustless nature of blockchain technology allows multiple parties involved in a transaction to execute with 100% guarantees for each side once conditions are met. That creates enormous opportunities for new business cases to be developed. At the time of writing, decentralized finance’s (DeFi) total value locked (TVL) is $41 billion, according to DefiLlama,
Decentralized finance (DeFi) total value locked (TVL) (Source).
Blockchain oracles have been in the Web3 space since 2015, bridging the gap between deterministic siloed blockchains and probabilistic real-world data, allowing multiple use cases on the blockchain that could not have been possible otherwise. By design dApps are supposed to be trustless, always running in the way they were designed. Oracles are a critical piece of the infrastructure ensuring that data can be trusted before it reaches the blockchain. For the smart contracts that rely on external data, execution oracles have to be fast, reliable, decentralized, and resistant to any type of attack.
The purpose of this report is to dive deeper into oracles’ value, identify bottlenecks, explore innovations in the space, and provide recommendations on designing the best oracle setup to ensure optimal protocol performance with the most accurate data feeds in the shortest time.
First, let’s do a quick recap on blockchain oracles. Oracles are decentralized applications that gather, validate, and deliver off-chain data to smart contracts on the blockchain. Similarly, they can do the same for delivering on-chain data to off-chain systems. Oracles are middleware connecting smart contracts on blockchains to off-chain data providers, sources and systems. Without oracles, smart contract applications would be limited to executing using only on-chain data.
If an oracle is corrupted, the correctness of the result of the execution of the smart contract will be compromised, potentially causing enormous losses. Blockchain oracles are a crucial part of the ecosystem.
Flash loan attacks, orchestrated oracle manipulation, and lengthy latency during extremely volatile times add up to the complexity of the infrastructure a dApp has to monitor when building its protocols.
Latency and frequency are two key parameters that determine the performance of an oracle, and a formula can be more complex taking multiple parameters into consideration.
Latency is the time taken for an off-chain data feed to be available to use for a smart contract on-chain after triggering a condition that requires off-chain data with a transaction. Latency can also be used in the context of data freshness, i.e., how old the last data feed is prior to being published on-chain. The latency formula depends on multiple factors.
Latency depends on both the underlying blockchain and oracle settings.
Frequency is how often the price is updated on a blockchain. In other words, how often the price update triggers (Deviation Threshold, Heartbeat Threshold or requester contract) publishing a new price. In a highly volatile market, the frequency of updates might be bigger because the triggering parameter such as the Deviation Threshold moves more often. The more frequent, almost real-time updates, especially during times of high volatility might contribute to network congestion if the blockchain throughput is low.
A relayer is a general term for a third party that relays some information from one party to another. In the context of blockchain, a relayer submits a user’s transaction to the blockchain network on their behalf and pays the associated gas fee. Oracles usually operate across multiple blockchains and one option for oracle architecture to achieve cross-chain interoperability is to use a third-party relayer design to transmit data across blockchains.
Relayers in oracle design can be used to bridge reported data to other blockchains (Source).
Some potential drawbacks to using relayer architecture are increased latency (users must wait for data to first be delivered to the primary blockchain, then they must wait for it to be bridged to a secondary blockchain or Layer2 network) and responsiveness, as the the relay model requires a set of highly available and incentivized third-party relayers to bridge oracle data from one chain to another.
Data sources are third parties that have access to the information in real-time, and can be divided into various categories
A qualitative data collection approach has been used to further deep dive into the existing oracle landscape. A semi-structured interview for the case study was selected to gather information about oracle use by the ten largest DeFi protocols in the market accounting for hundreds of millions of TVL.
All Chainlink, Binance Oracle, Pyth Network and Band Protocol documentation in service of the above-mentioned protocols has been reviewed and analyzed as a part of the case study.
A semi-structured interview was selected to gather information about oracle use by various DeFi protocols in the market. Participants were selected based on their TVL and trading volume.
The purpose of this paper is to identify and analyze the bottlenecks in the industry, as well as discover new options and provide recommendations on oracle use.
The limitations of the case study: CTOs and lead engineers of the largest subset of DeFi protocols were selected to interview. Smaller DeFi lending and borrowing protocols and small DEXs were not included. The list of questions for the interview is provided below.
The results from the survey using keywords analysis and transcribed data provided insights on how DeFi protocols are using oracles, what the limitations and challenges are, and sheds the light on how DeFi protocols mitigate risks relative to the industry.
The very first blockchain oracle was centralized and served the industry well by supplying the necessary data to the blockchain. But as the industry matured, different oracle designs emerged to solve myriad issues with the centralized model. Analysis of documentation from a variety of oracle providers indicate that their designs vary depending on multiple parameters (eth source):
1. Number of Data Sources. Oracles that are connected to multiple sources and generate the average price from different sources are called aggregated price oracles. For example, Chainlink, Binance Oracle, Pyth Network, Band Protocol are all aggregated price feed oracles as compared to Uniswap which is a single source oracle.
2. Location of Data Source. Data sources for oracles can be on-chain or off-chain. Some of the largest off-chain data providers are the largest CEXsconnected through APIs to oracles nodes where data is pulled, validated, signed, and published on-chain. The largest on-chain data sources are DEXs such as PancakeSwap for BNB chain and Uniswap for Ethereum. DEXs provide prices based on the invariant curve exchange rate for cryptocurrency pairs.
3. Centralized or Decentralized. Oracles are classified as centralized or decentralized depending on their trust model and consensus mechanism. One of the very first oracles in the space on Ethereum called Provable (formerly Oraclize) is a centralized oracle provider but now most are decentralized. .
4. Push or Pull. Oracles that automatically update cryptocurrency prices on chain are called push oracles and oracles that need an active request to update cryptocurrency prices are called pull oracles. Push oracles publish prices on-chain when triggered by one of two indicators:
Deviation Threshold: If the cryptocurrency price is different from the previous price by more than 0.1% -1% (varies for different pairs) then the push oracle is activated to update the price on-chain.
Heartbeat Threshold: If the cryptocurrency price doesn’t change within 1-10 minutes (depending on the parameters set) then the push oracle is activated to update the price on-chain.
5. Type of Data Source. Oracles can specialize in many types of data including cryptocurrency prices, commodities prices, FX prices, trade, weather, sports outcomes and statistics, identity, DNS lookups, and more.
While most of the oracles in the space are off-chain and decentralized, time-weighted average price (TWAP) oracles are different. TWAP oracles give the average price of a token for a determined period of time versus oracles that provide mean or weighted average prices aggregated from multiple data sources at a given moment. TWAP oracles are based on DEX prices and use the exchange rate of token A to token B as the price-determining factor. DEXs are the only source of truth for the price of the tokens that are not listed and traded on larger exchanges and there are no other providers available. For example, Uniswap TWAP V2.
The time-weighted average price (TWAP) calculation methodology supported by the Uniswap V2 automatic market maker (AMM) (Source).
A TWAP oracle has several limitations. It is a lagging indicator, so if a cryptocurrency price is volatile, the TWAP will not accurately reflect the price, which results in a higher risk of under-collateralization. TWAP oracles pull their price data from a single source only, making it more likely that low-cap asset prices off of a particular DEX are not representative of the broader market price. TWAP oracles do not provide data about off-chain trading pairs and they are not a scalable solution that mirrors the design of the underlying protocols that they service.
That said, TWAP oracles do have an unspoken benefit: they are the only source of price feeds for high-risk, low-cap tokens. Protocols that are built around isolated trading low cap tokens benefit from having access to a TWAP oracle but, as mentioned previously, the data in question is subject to natural (or malicious) market manipulation and must be used with caution.
At the time of the writing, Uniswap TWAP oracle team was working on researching on other improvements such as Time Weight Median Price (TWMP), wide-range liquidity, and limit orders to be introduced to Uniswap TWAP v.3.
Uniswap time weighted average price (TWAP) oracle total value secured (TVS) November 2022 (Source).
Currently, there are four major market participants in the oracle space:
Chainlink total value secured (TVS) (Source).
Chainlink is one of the largest oracle solutions in the market. It was established in 2017 and has been a source of truth for over 1500 dApps across 15 blockchains. At the time of this writing, the Total Value Secured by Chainlink was approximately $9.4 billion.
Chainlink is an off-chain decentralized oracle network that serves over 200 pairs on Ethereum and more than 100 pairs on the BNB chain. Chainlink data expands far outside of crypto pricing offerings and includes weather data, sports data, FX and commodities.
Chainlink has additional features such as data automation, VRF/RNG, Proof of Reserve, NFT price feeds, and Cross-chain interoperability protocol. Also, dApps can access any external data through AnyAPI adaptors.
Pyth Network total value secured (TVS) (Source).
Pyth Network was launched in 2021 and is the first oracle to popularize the pull mechanism for price feed updates. Currently, more than 70 projects are deployed to use Pyth Network which uses its own network to make sure the underlying blockchain does not affect the reliability of the oracle and that it is always running.
Pyth Network is a decentralized off-chain aggregate price oracle that publishes data off-chain 2-3 times per second for everyone to read it. That data is published on-chain only after the request for a price contract has been made. The gas fee is paid by whoever first called the price update and it is available for the rest to use cost-free once on-chain. End-users of Pyth data can elect to pay data fees to gain protection against a potential oracle failure.
Pyth Network uses a weighted average aggregate price coming from multiple sources, some of them exclusive. Pyth Data Providers are fully transparent and available to read.
Binance Oracle was launched in October 2022 after being in design and production for more than nine months. Binance Oracle has implemented a few modifications to create more resilient and faster push oracles. At the time of this writing, Binance Oracle is deployed to the beta testnet and has onboarded its first customers.
A few significant improvements to Binance’s oracle design were added to make the price feeds faster and more secure:
Band Protocol’s total value secured (TVS) (Source).
Band Protocol is a Cosmos-based oracle supporting 20 blockchains through the inter-blockchain protocol (IBC), a scalable oracle that has its own network to process all data. Band Protocol is also a decentralized off-chain aggregator oracle supporting more than 90 crypto symbols and 12 forex trading pairs.
Band Protocol has built its own relayer network and is able to ensure fast cross-chain communication to publish data to the different blockchains using the IBC bridge.
API3 is moving from the third-party oracle model to the first-party data providers directly on-chain. An off-chain first-party oracle connects data from any API to a smart contract through Airnode. DAO-governed, Airnode is Web3 middleware that connects any web API directly to any blockchain application. Airnodes are a piece of cloud service infrastructure that allow data providers to deploy their existing Web2 API onto the blockchain, creating what API3 calls a dAPI (Decentralized API).
Airnode Web3 middleware connects any web API directly to any blockchain application (Source).
The API3 team manages the endpoints and a multi-sig mechanism is used for extra security signing transactions. API3 can also provide individual data sets for users that require full control over the curation of the data feeds they use.
Umbrella is a Layer2 oracle built on a sidechain. Umbrella solves the scalability problem in oracles by leveraging a Layer2 solution and utilizes Merkle trees for batching transactions to save on gas fees.
Umbrella leverages a Layer2 solution and utilizes Merkle trees for batching transactions (Source).
Blockchain oracles have been live since 2015 – the same year that Ethereum smart contracts were introduced – and have since gone through many iterations and improvements. A few new emerging trends in oracle use have been identified both during analyzing case studies and following emerging technologies.
Privacy enabling zero-knowledge proofs (ZKP) are hot topics that have emerged over the course of 2021-2022, solving inherent blockchain problems like lack of confidentiality and inability to control private data. Blockchain oracles are no exception to this trend. The industry is seeking a solution to reveal and verify the truth without disclosing private information.
For example, Chainlink is working on a ZKP-based oracle solution called DECO, a privacy-preserving oracle protocol developed at Cornell University and later acquired by Chainlink. Oracle nodes can prove facts about data sourced from trusted servers without revealing the data on-chain, while also proving the source of the data since the TLS chain of custody is maintained.
One of DECO’s applications is a verifiable credential oracle that acts as a source of truth for biometric data and allows selective data disclosure paired with digital identity.
Oracle nodes can prove facts about data sourced from trusted servers without revealing the data on-chain.
For the average Web3 user the closest understanding of oracle is the bridge between Web2 real-world data and Web3 dApps. However, oracles can not only act as price or weather data feed providers but also can be used as a source of truth for inter-blockchain messaging itself.
A few prospective solutions are working on a ZK proof for cross-chain messaging where oracles act as a core part of the middleware to prove and verify that the data transmitted is true and can be trusted. An oracle node generates the ZK proof for the state of the smart contract so that data can be transferred across blockchains.
The fastest available data on-chain is necessary and widely used in the DeFi world however, DEX interviews revealed that there is no actual demand for the real-time speed of pushing price feeds – it has to be fast, but it doesn’t have to be ultra-fast. There are two different approaches to data delivery speed with respect to pull oracles:
It is important to keep in mind that oracle’s minimum latency is still a blockchain’s block time to finality when transactions are finalized, which will probably remain the primary limitation to data delivery speed.
Some protocols developed their own implementations of TWAP oracles with added features such as using moving averages to smooth abrupt price movements. Others have built custom pools as an oracle on the AMM/DEX. TWAP oracles might see increased demand in the future if markets move towards decentralized exchanges, which may be more likely after recent market volatility in November 2022.
Oracles are critical middleware infrastructure that enable myriad use cases for the blockchain. Competition amongst legacy oracle providers has pushed them to constantly innovate, add resiliency to the oracle ecosystem, and drive adoption for their services in new and better ways. Binance Oracle’s entrance into the space introduces a new player with enhanced speed and security.
While capital continues to flow to DEXs, the collapse of crypto markets in November 2022 may delay the adoption of further advances in oracle development such as verified credentials and ZK cross chain messaging. Nevertheless, oracle innovation continues to unfold, bringing ever more utility to blockchain over time.