# Cosmos Hub 4, v9-Lambda Upgrade, Instructions
This document describes the steps for validator and full node operators for the successful execution of the v9-Lambda Upgrade (opens new window), which contains the following main new features/improvement:
- Interchain-Security (opens new window) v1.0.0 (opens new window) provider module. See the ICS Spec (opens new window) for more details.
- cosmos-sdk (opens new window) to v0.45.13-ics (opens new window). See CHANGELOG.md (opens new window) for details.
- ibc-go (opens new window) to v4.2.0 (opens new window). See v4.2 Release Notes (opens new window) for details.
- tendermint (opens new window) to 0.34.26 (opens new window). See CHANGELOG.md (opens new window) for details.
- packet-forward-middleware (opens new window) to v4.0.4 (opens new window).
- E2E ccv tests (opens new window). Tests covering new functionality introduced by the provider module to add and remove a consumer chain via governance proposal.
- integration ccv tests (opens new window). Imports Interchain-Security's
TestCCVTestSuiteand implements Gaia as the provider chain.
TOC:
- Cosmos Hub 4, v9-Lambda Upgrade, Instructions
# On-chain governance proposal attains consensus
Proposal #187 (opens new window) is the reference on-chain governance proposal for this upgrade, which is still in its voting period. Neither core developers nor core funding entities control the governance, and this governance proposal has passed in a fully decentralized way.
# Upgrade will take place March 14-16, 2023
The upgrade will take place at a block height of 14470501. The date/time of the upgrade is subject to change as blocks are not generated at a constant interval. You can stay up-to-date using this live countdown (opens new window) page.
# Chain-id will remain the same
The chain-id of the network will remain the same, cosmoshub-4. This is because an in-place migration of state will take place, i.e., this upgrade does not export any state.
# Preparing for the upgrade
# System requirement
32GB RAM is recommended to ensure a smooth upgrade.
If you have less than 32GB RAM, you might try creating a swapfile to swap an idle program onto the hard disk to free up memory. This can allow your machine to run the binary than it could run in RAM alone.
# Backups
Prior to the upgrade, validators are encouraged to take a full data snapshot. Snapshotting depends heavily on infrastructure, but generally this can be done by backing up the .gaia directory.
If you use Cosmovisor to upgrade, by default, Cosmovisor will backup your data upon upgrade. See below upgrade by cosmovisor section.
It is critically important for validator operators to back-up the .gaia/data/priv_validator_state.json file after stopping the gaiad process. This file is updated every block as your validator participates in consensus rounds. It is a critical file needed to prevent double-signing, in case the upgrade fails and the previous chain needs to be restarted.
# Testing
For those validator and full node operators that are interested in ensuring preparedness for the impending upgrade, you can run a v8-Rho local testnet (opens new window) or join in our v9-Lambda public-testnet (opens new window).
# Current runtime, cosmoshub-4 (pre-v9-Lambda upgrade) is running Gaia v8.0.1
The Cosmos Hub mainnet network, cosmoshub-4, is currently running Gaia v8.0.1 (opens new window). We anticipate that operators who are running on v8.0.1, will be able to upgrade successfully. Validators are expected to ensure that their systems are up to date and capable of performing the upgrade. This includes running the correct binary, or if building from source, building with go 1.18.
# Target runtime, cosmoshub-4 (post-v9-Lambda upgrade) will run Gaia v9.0.0
The Cosmos Hub mainnet network, cosmoshub-4, will run Gaia v9.0.0 (opens new window). Operators MUST use this version post-upgrade to remain connected to the network.
# v9-Lambda upgrade steps
There are 2 major ways to upgrade a node:
- Manual upgrade
- Upgrade using Cosmovisor (opens new window)
- Either by manually preparing the new binary
- Or by using the auto-download functionality (this is not yet recommended)
If you prefer to use Cosmovisor to upgrade, some preparation work is needed before upgrade.
# Method I: Manual Upgrade
Make sure Gaia v9.0.0 is installed by either downloading a compatable binary (opens new window), or building from source. Building from source requires go 1.18.
Run Gaia v8.0.1 till upgrade height, the node will panic:
Stop the node, and switch the binary to Gaia v9.0.0 and re-start by gaiad start.
It may take several minutes to a few hours until validators with a total sum voting power > 2/3 to complete their node upgrades. After that, the chain can continue to produce blocks.
# Method II: Upgrade using Cosmovisor
Please Read Before Proceeding
Using Cosmovisor 1.2.0 and higher requires a lowercase naming convention for upgrade version directory. For Cosmovisor 1.1.0 and earlier, the upgrade version is not lowercased.
For Example:
Cosmovisor =<1.1.0:/upgrades/v9-Lambda/bin/gaiad
Cosmovisor >=1.2.0:/upgrades/v9-lambda/bin/gaiad
| Cosmovisor Version | Binary Name in Path |
|---|---|
| 1.3 | v9-lambda |
| 1.2 | v9-lambda |
| 1.1 | v9-Lambda |
| 1.0 | v9-Lambda |
# Manually preparing the Gaia v9.0.0 binary
# Preparation
Install the latest version of Cosmovisor (1.3.0):
Verify Cosmovisor Version
Create a cosmovisor folder:
create a Cosmovisor folder inside $GAIA_HOME and move Gaia v8.0.1 into $GAIA_HOME/cosmovisor/genesis/bin
build Gaia v9.0.0, and move gaiad v9.0.0 to $GAIA_HOME/cosmovisor/upgrades/v9-lambda/bin
Then you should get the following structure:
Export the environmental variables:
Start the node:
Skipping the invariant checks is strongly encouraged since it decreases the upgrade time significantly and since there are some other improvements coming to the crisis module in the next release of the Cosmos SDK.
# Expected upgrade result
When the upgrade block height is reached, Gaia will panic and stop:
This may take 7 minutes to a few hours. After upgrade, the chain will continue to produce blocks when validators with a total sum voting power > 2/3 complete their node upgrades.
# Auto-Downloading the Gaia v9.0.0 binary (not recommended!)
# Preparation
Install the latest version of Cosmovisor (1.3.0):
Create a cosmovisor folder:
create a cosmovisor folder inside gaia home and move gaiad v8.0.1 into $GAIA_HOME/cosmovisor/genesis/bin
Export the environmental variables:
Start the node:
Skipping the invariant checks can help decrease the upgrade time significantly.
# Expected result
When the upgrade block height is reached, you can find the following information in the log:
Then the Cosmovisor will create $GAIA_HOME/cosmovisor/upgrades/v9-lambda/bin and download the Gaia v9.0.0 binary to this folder according to links in the --info field of the upgrade proposal 97.
This may take 7 minutes to a few hours, afterwards, the chain will continue to produce blocks once validators with a total sum voting power > 2/3 complete their nodes upgrades.
Please Note:
- In general, auto-download comes with the risk that the verification of correct download is done automatically. If users want to have the highest guarantee users should confirm the check-sum manually. We hope more node operators will use the auto-download for this release but please be aware this is a risk and users should take at your own discretion.
- Users should use run node on v8.0.1 if they use the cosmovisor v1.3.0 with auto-download enabled for upgrade process.
# Upgrade duration
The upgrade may take a few minutes to several hours to complete because cosmoshub-4 participants operate globally with differing operating hours and it may take some time for operators to upgrade their binaries and connect to the network.
# Rollback plan
During the network upgrade, core Cosmos teams will be keeping an ever vigilant eye and communicating with operators on the status of their upgrades. During this time, the core teams will listen to operator needs to determine if the upgrade is experiencing unintended challenges. In the event of unexpected challenges, the core teams, after conferring with operators and attaining social consensus, may choose to declare that the upgrade will be skipped.
Steps to skip this upgrade proposal are simply to resume the cosmoshub-4 network with the (downgraded) v8.0.1 binary using the following command:
gaiad start --unsafe-skip-upgrade 14470501
Note: There is no particular need to restore a state snapshot prior to the upgrade height, unless specifically directed by core Cosmos teams.
Important: A social consensus decision to skip the upgrade will be based solely on technical merits, thereby respecting and maintaining the decentralized governance process of the upgrade proposal's successful YES vote.
# Communications
Operators are encouraged to join the #cosmos-hub-validators-verified channel of the Cosmos Hub Community Discord. This channel is the primary communication tool for operators to ask questions, report upgrade status, report technical issues, and to build social consensus should the need arise. This channel is restricted to known operators and requires verification beforehand. Requests to join the #cosmos-hub-validators-verified channel can be sent to the #general-support channel.
# Risks
As a validator performing the upgrade procedure on your consensus nodes carries a heightened risk of double-signing and being slashed. The most important piece of this procedure is verifying your software version and genesis file hash before starting your validator and signing.
The riskiest thing a validator can do is discover that they made a mistake and repeat the upgrade procedure again during the network startup. If you discover a mistake in the process, the best thing to do is wait for the network to start before correcting it.