Update on CFG rewards claiming for Tinlake

Hey everyone,

wanted to give a status update on the issues with claiming CFG rewards and underlying problems. We have been working hard on fixing the underlying issues and finding a way forward but are not there yet.

The most important confirmation up-front, no CFG rewards are or will be lost. This is a matter of aggregating and displaying the rewards in the UI where they are also claimed. CFG rewards keep on accruing correctly and can be claimed once the underlying issues are solved.

What happened:
The index service (”subgraph”) based on The Graph we use for aggregating data on Tinlake failed recently. We keep backup infrastructure in place for these case, but unfortunately the backup subgraph does not entail the latest reward change (technically, the split of TIN and DROP reward rates). Thus, at the moment, everything in the UI is displayed correctly, except claiming TIN/DROP rewards.

We are currently upgrading our infrastructure to a more recent version from the Graph. Once the upgrade to the newer endpoints is finished, everything will work as before. This upgrade is taking longer than expected but we currently expect for this to be resolved in the next 2-3 weeks.

We will keep you posted on any developments that may change this timeline to better or worse and of course as soon as claiming is available again.

We understand that this is not great and are working hard to fix this asap. Thanks for your patience.

10 Likes

Hi @denniswell,

Thanks for the info. Is there any update you can give on this topic?

Hey all, update on CFG rewards/subgraph. Not the update we’ve hoped for but we’ll stay on it.

The subgraph we’ve deployed entailing the split rewards rate is still synching incredibly slowly. We haven’t been able to speed this up yet unfortunately but are working on it, looking into the underlying infrastructure as well, talking to quicknode…

As described above, we are working in parallel on upgrading the subgraph to the new endpoints. Still running into bugs and squashing and it may still take a few days to get it done. The speed looks good but its really hard to say how much faster it is at this stage.

We start thinking about non-technical workarounds to fall back on old subgraph without the split and old rates but this would require new governance votes and we have avoided this path for a reason…

While all of this is far from ideal, we still want to take a bit more time to look into making the upgraded version work. Once we have more tangible outlook beginning of next week, I will update again.

Please be assured that we are working hard on a solution and this is as frustrating for us as it is for you and our investors/issuers. Thanks for your patience and ongoing support.

And obviously, to ensure everyone, no CFG rewards are or will be lost. This is a matter of aggregating and displaying the rewards in the UI where they are also claimed. CFG rewards keep on accruing correctly and can be claimed once the underlying issues with the subgraph are solved.

3 Likes

Hey all, we are making good progress on integrating the new endpoints and speeding up the existing subgraphs, but are less confident to get this done in the next 2-3 weeks
We are looking into two workarounds now:

  • (i) Calculating the CFG rewards through a script and plugging this in the UI.This would allow either a one time claim of earned rewards for everyone, perhaps we can even make it work daily

  • (ii) Plugging in an old subgraph, reverting the reward rates to the old set-up without split. Would require another governance vote.
    We’ll need to make a call mid next week. Will keep you in the loop. Thanks for your patience

2 Likes

We’ve made good progress on two fronts.
(i) We’ve written a script on top of previous (working) subgraphs that implements the split reward rates. With this script we can manually calculate the rewards and we can use this to enable claiming earned rewards in the UI up until the day of the execution of the script. We hope to have this up and running by beginning of next week. Note, for the avoidance of doubt, this will enable investors to claim once, so it is a workaround to claim the CFG earned until the day of deployment, but at least this will allow investors to claim rewards up until that day
(ii) We have deployed a new subgraph with the updated endpoints that is currently synching. We expect this to synch within 10-14 days. We’ve been there before but are confident that this version will fully synch now. Once this is up and running, we can plug it in and claiming rewards should work as before. Will keep you updated on progress here. If this fails or takes longer than expected, we can fall back on the manual solution above if needed for a second time, to enable claiming more CFG

3 Likes

What happened on the script deployment ? What progress on sub-graph syncing?
We are approaching the 2month anniversary of the cfg parachain launch, still not been able to claim rewards on Tinlake after this length of time is a total debacle. How can you expect to convince new people to invest when one of the main interface to the protocol, indicating reward rates is broken for so long!
image

Good day Chipman
First of all, I would like to ask you sorry for this trouble and issue.
I can confirm, that the script is working and yesterday my rewards were updated correctly.
But please note that the script can not update rewards instantly for all users, so will request some time to update all data correctly.

As I can see another user also was able to see his rewards updated. You can find his message here:

Sorry for this issue, hope that soon script could sync all data and finally all investors will be able to claim their rewards.
If you have any other questions feel free to ask.

1 Like

Good day Centrifuge community!

While we are not over the finish line yet, we are very close now. The new subgraph is working with the correct numbers. We are currently squashing bugs on integrating it into the UI. I know we’ve been there before, but I hope we can share finally good news next week.

3 Likes

Status update on claiming CFG rewards. The hopefully last remaining piece of the puzzle is to change the address that holds the funds to be distributed to the contributors. This had changed during a refactoring for the Parachain launch. See proposal Council Motion 6: Migrate Tinlake Claims Source Account. Once the democracy vote has passed, we’d run the last tests and hope to ship soon unless we run into more issues. We are currently optimistic that claiming should be back up beginning of next week.

1 Like

It is finally happening. Claiming CFG rewards is fixed and now live in production on Tinlake.io. Please note, that within the new set-up claiming CFG rewards (finally) requires a CentChain confirmation so you’d need some CFG on your CentChain account. If you have problems claiming, please get in touch here, or ideally through support@centrifuge.io.
Thanks, everyone for your patience.

1 Like

Status update: Claiming CFG is technically working, the underlying CFG account is running low on CFG preventing larger claims at the moment. This may effect some with larger amounts of CFG to claim. We are working on a fast-track governance proposal to mint the required CFG into the account. Once passed and minted everything should work. Currently expecting this by tomorrow eob US time. Will update tomorrow. So close. Thanks for your patience

2 Likes

The public referendum has been enacted & CFG rewards minted (more background here Council Motion 8: Mint Tinlake LP Rewards). Claiming CFG rewards should work now. If there are any issues please contact support@centrifuge.io. Happy weekend everyone and thanks for your patience

1 Like