Corrupted node - missing channel.backup

1

Hi -

Background

I recently logged into my umbrel after 6 days of no use, was assuming it was active.

When I returned to the node, I logged in, balanced weren’t showing and hard drive was filled 950gb of system data

I tried pulling logs via settings, they failed to load, then I trigged a shut down which hanged and I manually unplugged after 15 minutes.

After startup umbrel UI indicated docker images failed to start

I logged in via ssh, and noticed the hard drive now is unexpectly indicating nearly 100% free space

I ran debug script, then saw docker images were failing to start due to missing config
So I ran scripts indicated here Error: Failed to start containers

After doing that, when accessing via Umbrel UI
I either see “welcome to umbrel” with a disabled “loading” button
or Sometimes the docker services startup error page loads

Running fsck reported no errors
I believe I have lost my channel data
when I look at the file tree, I don’t see files I’d expect to find

  • lnd/data/graph/mainnet/ only has channel.db but file size seem small
  • db/umbrel-seed is empty

Tomorrow I’ll try with a fresh SD card, but I worry the issue lies with the SSD

——

The support I need

I’d very much appreciate some support with these two pieces.

  • is there any additional triage steps I can do?

  • I’d like to initiate force close on my channels but I’m not confident my channels.backup is up to date, can I request the latest backup ( I do have a backup of the seed ) ?

I appreciate the support, there is a significant amount of funds within these channels. I’d like to close them asap.

Thank you

2

Here is my current debug log

3

Yes you can request a copy of channels.backup from @lukechilds or @mayank, you will need to contact them in PM and provide some details to prove the ownership of that wallet.

Once you have the backup file, will be better to start from scratch building that node, with restoring wallet.
Here you have a guide

1 Like
4

I ended up using an out-of-date SCB, and reached out to umbrel for latest.
The “out-of-date” SCB closed most channels.

I provided umbrel team a backup ID, they were able to support me with a more recent SCB.
I’ve now ran this and happy to see all except 1 channel has closed.

I will need to work with my peer using chantools -> zombierecovery to close the remaining channel as they have also suffered from a failure.

Thanks for your support @DarthCoin - I saw your information a lot when learning triage steps and it was very much appreciated.

1 Like
5

Nice to see that you “escaped” from the nightmare of restoring.
Yes zombie channels is a thing and is not desired at all.