Uniswap v3 TWAP Oracle Deep Dive Pt. 2

The initial Uniswap Grants was announced here

Overview

Chaos Labs is a cloud platform that specializes in Economic Security for DeFi protocols. At Chaos Labs we build agent- and scenario- based simulations to battle test protocols in adversarial and chaotic (🤗) market conditions. In our quest to build state of the art simulations and security tooling we also partner with DeFi protocols to build valuable tools for the DeFi developer ecoysystem. This series of posts is a technical deep dive on Uniswap v3 TWAP oracles. Our implementation for configuring Uniswap v3 TWAP return values can be found here. PRs are welcome!

Intro

In our previous blog post, we covered the fundamentals of Uniswap V3 TWAP oracles. This blog post will focus on:

• TWAP storage layout
• Hardhat capabilities
• TWAP architecture and implementation
• Configuring return values of Uniswap V3 oracles within a forked environment.

But before we deep dive let’s understand why oracle price configuration is a critical component of high fidelity simulations.

Why Oracle Price Configuration is Impactful

DeFi protocols control billions of dollars worth of assets on-chain and require thorough simulation execution, risk evaluation and testing. Protocols and dApps evolve in a variety of ways such as:

• Protocol launches
• Parameter updates
• New functionality introduced via governance proposals
• New protocol integrations or support for new assets

All on-chain code is extremely sensitive and must undergo thorough evaluation before being deployed to mainnet.

A core belief at Chaos Labs is that an effective simulation environment is as close to the production environment as possible. Assumptions and functionality stubbing create a larger deviation from the condition users and protocols will face on-chain. Deviations decrease reliability and security guarantees while ultimately leaving engineering teams in the dark.

Now let's zoom in on Oracles. Contracts and dApps treat oracle return values as a source of truth and valid application input. Oracle return values are often the decisive factor, in determining internal state transitions and critical control flow execution.

As we aim to test our dApps in an environment that is as close to production as possible we are faced with the challenge of recreating desired states in our simulations. Edge cases or protocol fragility are often the most interesting to simulate. Consider instances of:

• High market volatility
• Cascading prices
• Adversarial interactions
• Dynamic network congestion
• Varying transaction pick-up latency

This is especially true for DeFi protocols that often boast heavy protocol and infrastructure dependencies. As we know, oracles are key pieces of infrastructure and stubbing out dApp <> oracle communication as is common in unit tests is insufficient.

To this end, we need to have the ability to configure oracles as they represent a core application dependency. Having granular control over oracle return values will allow us to verify that our protocols performs as expected, even under harsh or adversarial conditions.

So how do we configure Uniswap v3 TWAP Oracle return values price within the context of a fork?

How Do We Achieve Oracle Price Configuration

The Uniswap documentation is a good place to start. Let’s begin by examining the Uniswap pool implementation to understand where and how the price is stored within the contract. The answer to that question is - it depends. It depends on several parameters:

• The TWAP interval for which we query the price
• The trading history of the pool
• The size of the Observations array assigned to the pool.

Let’s start with the most common case of a TWAP interval that is bigger than zero. We’re looking for the average price in the Uniswap pool over a period of X seconds, such that X > 0.

Note: It is usually not safe to use very short or 0-time intervals since it leaves you more susceptible to price tampering

In order to get the average price, we need to know what the price was at two points in time:

• Now
• X seconds ago

Our previous post highlighted that we don't save the price itself within the pool. We utilize the tick to derive the price. Two important data structures are used to keep track of the ticks:

• The Observations array
• Slot0

struct Observation {
// the block timestamp of the observation
uint32 blockTimestamp;
// the tick accumulator, i.e. tick * time elapsed since the pool was first initialized
int56 tickCumulative;
// the seconds per liquidity, i.e. seconds elapsed / max(1, liquidity) since the pool was first initialized
uint160 secondsPerLiquidityCumulativeX128;
// whether or not the observation is initialized
bool initialized;
}

struct Slot0 {
        // the current price
        uint160 sqrtPriceX96;
        // the current tick
        int24 tick;
        // the most-recently updated index of the observations array
        uint16 observationIndex;
        // the current maximum number of observations that are being stored
        uint16 observationCardinality;
        // the next maximum number of observations to store, triggered in observations.write
        uint16 observationCardinalityNext;
        // the current protocol fee as a percentage of the swap fee taken on withdrawal
        // represented as an integer denominator (1/x)%
        uint8 feeProtocol;
        // whether the pool is locked
        bool unlocked;
    }

Slot0’s main purpose is to maintain the current state of the pool. The main fields of interest for us are:

• tick
• observationIndex,
• observationCardinality

We will see how using these and the observations array we can extrapolate the TWAP.

Some basic concepts we need to understand first:

• Each Uniswap pool is deployed with an Observation array of length 1, and it can be extended later on through a dedicated function call, where the caller will pay the gas fees for extending the contract's storage.
• Observations array is a cyclic buffer storing the latest observations up to the current size of the array defined by observationCardinality. Thus increasing the size of the array allows for supporting longer TWAP intervals.

Now lets see what happens when request two ticks with X seconds separating between them through observe function:

function observe(uint32[] calldata secondsAgos) external view
  returns (int56[] memory tickCumulatives, uint160[] memory secondsPerLiquidityCumulativeX128s);

We can see that for each timestamp we get as input we use binary search to find the closest recorded observations in the observations array. We then proceed to extrapolate the observation for the exact timestamp (if there isn’t any) using the two adjacent recordings. Eventually, the observe function returns two extrapolated observations for the exact timestamp requested if there is an observation recording older than the requested time.

With two observations we can obtain the time-weighted average tick which leads us to the time-weighted average price.

Now that we understand how the oracle works, we can get to the fun part :) How do we approach configuring the prices returned?

The approach we have taken with our implementation was to override the Observation recordings in the contract to achieve any price we desire. We can create new recordings by executing trades in the pool, but we want to change only the price returned and not alter the whole state of the contract as it can result in undesired behavior. Instead, we decided to rewrite the contract memory directly with our own chosen values. This requires two operations:

• Reverse calculating the observation's storage layout and their locations in the array to result in price P for TWAP X.

• A way to alter the contract storage, where the Observations reside.

The first requirement can be achieved with our newfound knowledge of Uniswap pools. How do we solve for the second requirement? There are several ways to accomplish this. Let's examine a solution which utilizes Hardhat capabilities.

Hardhat Capabilities

Hardhat is a powerful tool for EVM developers. In this article, we will focus on its ability to create EVM network forks. Hardhat exposes control endpoints on the forked node that allows developers to do some really cool stuff. Let's focus on the - hardhat_setStorageAt feature.

Using this API we can set the storage of a contract, given that we know what we want to store and where. But as the documentation mentions, this is not straightforward. That doesn’t deter us though!

Storage Layout

In order to modify contract storage directly, we need to understand Ethereum memory and Solidity variable layout in storage. We will only cover the parts required for our use case with Uniswap oracles, but we highly recommend learning more via the solidity docs.

Going back to the Uniswap pool contract - we know we want to change the value stored in Slot0 and the observations array. Where are they in the contract storage layout?

State variables of contracts are stored in storage in a compact way such that multiple values sometimes use the same storage slot. Except for dynamically-sized arrays and mappings (see below), data is stored contiguously item after item starting with the first state variable, which is stored in slot 0

Slot0 is the first state variable in the contract, and so, as its name suggests - it is stored in slot 0 of the contract storage.

One thing to note is that every storage slot is 32 bytes long. Therefore, if a structure is bigger than 32 bytes it will be stored in multiple slots. However, if a struct is smaller than 32 bytes it can be stored with the next variables into the same slot for memory efficiency. So when we work to place the variables in their respective slots we need to account for two things:

• Initialization order within the contract
• Byte size.

Following this process, we conclude that Slot0 is placed at slot 0 (surprise 😅) and as its full size is exactly 256 bits = 32 bytes it ends at slot 1.

As for the observations array, it starts at slot 8 of the contract storage. Each observation is also exactly 32 bytes, and so observation X can be found at slot 8+X of the contract storage.

We can also use the Solidity compiler to make this process easier for us. When compiling the contract we can add flags to the compiler to produce the contract storage layout as part of the compilation output. Below we can see an example of such output:

So we know where there the variables are stored. Are we done? Not quite!

Taking a deeper look into the Observation struct, we see that it contains additional fields. In our case, we only want to change the value of the tickCumulative variable of each observation. The same applies for Slot0. However, we'll focus on the observations for this step:

Each Observation has the following fields with varying bit sizes:

blockTimestamp, tickCumulative, secondsPerLiquidityCumulativeX128,initialized.

With a bit of math, we can conclude the bits in our 32 bytes slots that contain our targeted tickCumulative.

Another concept we must familiarize ourselves which is Endianness - the order or sequence of bytes of digital data in computer memory. We won't dive into Endianness in Ethereum here. However, if you’re interested - this is a great blog post about it!

In short - Ethereum uses the two endianness format depending on the variable type:

• Big-endian format for strings and bytes
• Little-endian format for other types (numbers, addresses, etc..).

We finally conclude that tickCumulative for observation X is stored at slot X+8, in bytes 21 to 28 of the slot.

Using the code snippet below we can then override an Observation tickCumulative:

async function OverrideObservationTick(n: BigNumber, index: number): Promise<string> {
    const slot = 8 + index;
    const add = `0x${slot.toString(16)}`;
    const st = (await this.provider.send("eth_getStorageAt", [this.poolAddress, add])) as string;
    const tickHex = numberToStorage(n, 14);
    const newSt = "0x" + st.slice(2, 44) + tickHex + st.slice(-8);
    await this.provider.send("hardhat_setStorageAt", [this.poolAddress, add, newSt]);
    return newSt;
  }

What's next? ⏭️

We're collaborating wiht Uniswap to build a developer tool that eases the usage and testing of v3 TWAP oracles based on our previous work. Stay tuned 😉

About the Uniswap Grant Program

If you want to learn more about the Uniswap Grants Program, check out their blog.