Node and Runtime Upgrade Proposal #3

Engineering
1

Hi validators,

We are ready to release a new version of the Centrifuge Chain that catches up with latest supported Substrate version (v2.0.0-rc6).
Here are the details of the release: Release Centrifuge v2.0.0 Release - RC6 · centrifuge/centrifuge-chain · GitHub

There has been many changes in Substrate between our current version and the latest one that make this upgrade not as seamless as we would have liked. These are the main changes:

  • Internal child storage hierarchy has changed. This requires a fresh sync for all the nodes so the new structure can be generated during that phase.
  • Hashers of core pallets have changed, therefore a list of migrations need to be done at the time of the runtime upgrade.

The first change is the most disruptive, since it requires a two step migration process, involving all validators/nodes before the runtime upgrade can be executed.

In a nutshell, every validator must follow the below steps:

  • Node Upgrade: Here are detailed steps on how to perform it - Cetrifuge Chain Node Upgrade Guide - HackMD
  • Once all validators have updated their nodes with the new version and are producing blocks in the new nodes, we will trigger the runtime upgrade.

It is really important that all validators perform this in a timely manner, otherwise those that have not, might get their stake slashed.

Timeline:

Flint & Amber Node Upgrade (In Parallel):

  • Centrifuge is bringing up new bootstrap nodes (same IPs and NodeIDs) by Thursday 1st October UTC EOD, do not start your new validators until the new bootnodes are up, they will not sync. (We will notify once they are up)
  • [ACTION ITEM] All validators in both networks to have performed the Node Upgrade by Friday 9th October EOD UTC
  • After 1 week of stability, Centrifuge will propose the runtime upgrade to RC6. Done by Friday 16th October EOD UTC

Mainnet:

  • Centrifuge is bringing up new bootstrap nodes (same IPs and NodeIDs) by Friday 23rd October UTC EOD, do not start your new validators until the new bootnodes are up, they will not sync. (We will notify once they are up)
  • [ACTION ITEM] All validators to have performed the Node Upgrade by Friday 30th October EOD UTC
  • After 1 week of stability in mainnet env, Centrifuge will propose the runtime upgrade to RC6. Done by Friday 6th November EOD UTC

Let us know if you have any questions,
Thank you!

6 Likes
2

Here are the new Bootnodes for Flint and Amber:

Flint Bootnodes:
- --bootnodes=/ip4/35.246.197.135/tcp/30333/p2p/12D3KooWBF1RdctVztxLLzEwUiuMtqEDVicCjVCS8eyxh71nonxz
- --bootnodes=/ip4/34.89.219.76/tcp/30333/p2p/12D3KooWNHRdve4U1rsZsDnTKbY8C94Y7VJTrifBy9P2LXLDhCnG

Amber Bootnodes:
- --bootnodes=/ip4/34.107.94.59/tcp/30333/p2p/12D3KooWQ3YL8sP2M1S9PVNj8JjmACUjawwRPckzL8fJmXZ7YRPY
- --bootnodes=/ip4/35.242.247.213/tcp/30333/p2p/12D3KooWECFKJirQiVHeidSkwrEocPr7wkUw3ijbJKNyvCVd3xcu
3

We are ready to start the Mainnet runtime upgrade. Feel free to follow the steps outlined here.
Using the Mainnet bootnodes to sync.

We released a patched version for the mainnet deployment: https://github.com/centrifuge/centrifuge-chain/releases/tag/v2.0.0-rc6.0
Please use this one.

As explained in the overall upgrade plan, we will trigger the upgrade by Friday 30th EOD UTC, so all validators need to be running and producing blocks using the latest version v2.0.0-rc6.0 by then.
Please allow 1 day for your session keys to be switched from one node to another, so make sure that by Thursday 29th EOD you triggered the controller session key swap as described in the guide above.

Thank you!

4

Lazy Payouts Update

In addition to the main changes mentioned in the original post, so-called “lazy payouts” were also introduced in runtime upgrade #3. This change was introduced as a part of Substrate V2. As a result, there is a change to how rewards are paid out to validators and nominators.

This change requires that payouts are manually done for each validator. This then automatically pays out all of the nominators that have staked to that validator.

Linked here is a simple guide to do this using the Centrifuge portal. We recommend that validators do this themselves every 2 days.

Some validators have been working on their own auto-payout. Please feel free to post any known work below!