I’ve run an umbrel node using raspberry pi 4 (8GB RAM) and 2TB crucial SSD for a few months.
Since last night, it takes a lot longer to show wallet and channel infos in the dashboard and all the channels are indicated to be offline.
After reading through the archive, I tried the followings first:
reflash the microsd card
reboot
lncli unlock (in response to the error message indicating the wallet is locked).
No change is seen, so I am thinking of going through the recovery process.
To make sure not to screw things up, I wanna ask some questions beforehand.
I found the following channel backup file: /home/umbrel/umbrel/lnd/data/chain/bitcoin/mainnet/channel.backup`
This file is shown to be made in the most recent date when a new channel was added. Since then, there were a few routing transactions. In this case, the backup file is different from the file downloaded from the app? In other words, are channel balances are included in the channel backup file?
If different, which one should I use in the recovery process?
I understand that a successful recovery process closes all the channels and reallocate the funds to the onchain wallet.
Currently, the onchain wallet is also problematic, showing “invalid btc address” messages whenever trying to send funds. Does this have nothing to do with the recovery process?
I want to try with blixt app first to save time, but if I eventually try with the same raspi 4, I just need to erase all files in the SSD drive and restore the channel file after going through the installation with 24 recovery phrases. Am I right?
I attach some parts of the debug message file just in case there are simpler ways
Thank you.
Memory monitor logs
./scripts/memory-monitor: line 30: /bin/sleep: Input/output error
./scripts/memory-monitor: line 23: awk: command not found
./scripts/memory-monitor: line 23: free: command not found
./scripts/memory-monitor: line 24: ((: > 90 : syntax error: operand expected (error token is "> 90 ")
./scripts/memory-monitor: line 30: /bin/sleep: Input/output error
./scripts/memory-monitor: line 23: awk: command not found
./scripts/memory-monitor: line 23: free: command not found
2022-02-02 01:42:07 Memory monitor running!
2022-02-02 02:02:26 Memory monitor running!
2022-02-02 02:18:49 Memory monitor running!
middleware | umbrel-middleware
middleware | Unable to list invoices, wallet locked, unlock it to enable full RPC access
middleware | /v1/lnd/lightning/invoices
middleware | LndError: Unable to list invoices
…
middleware | umbrel-middleware
middleware | LndUnlocker: Wallet unlocked!
middleware | ::ffff:10.21.21.2 - - [Wed, 02 Feb 2022 03:53:03 GMT] “GET /v1/bitcoind/info/sync HTTP/1.0” 200 88 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0”
middleware |
LND logs
Attaching to lnd
lnd | 2022-02-02 03:42:26.354 [INF] LTND: We’re not running within systemd
lnd | 2022-02-02 03:42:26.354 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create a wallet, lncli unlock to unlock an existing wallet, or lncli changepassword to change the password of an existing wallet and unlock it.
lnd | 2022-02-02 03:42:28.240 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | 2022-02-02 03:42:28.975 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | 2022-02-02 03:42:30.440 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
lnd | 2022-02-02 03:42:32.645 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
No, there’s no need to erase everything. Is enough just to reset the user data and start with a fresh wallet, keeping the blockchain sync.
But… if your node has a problem on that data (could that the reason is not starting well), then is better to start fresh with new download of blockchain. I know is painful to wait, but is the best way to be sure is clean. Data corruption always endup in days of frustration and waiting, when you could just start clean.
In Umbrel troubleshooting manual there’s a section that explain how to reset user data and start with a new fresh wallet, user, password, seed etc. but keeping the blockchain.
Yes, I strongly recommend to take your seed + channels.backup file into your mobile device with Blixt and recover your funds. That is the easiest way, meanwhile you are rebuilding a new Umbrel node.
Thank you for your kind answer.
As you suggested, I first tried to reset the user data with maintaining the blockchain.
Onchain fund was shown in a couple of hours.
I understand that I should wait for a while to get funds returned from my lightning channels. Is there any file in which I can check if the forced channel close is actually in progress or if the command has actually been called?
I am asking because the following empty lists print in response to lncli pending channels.
{
“total_limbo_balance”: “0”,
“pending_open_channels”: [
],
“pending_closing_channels”: [
],
“pending_force_closing_channels”: [
],
“waiting_close_channels”: [
]
}
I downloaded the channel backup file from the app link and used it for the recovery. Is there a way of checking if the backup file is okay?
your reset the Umbrel user data and start with restoring from seed and channels?
you download the backup channels file from which “app”? Umbrel dashboard or which app? Not all backups are compatible when is about to restore Umbrel in command line.
if you do not follow exactly the procedure of recovering funds from channels, you will not get it back.
do not use that recovering procedure twice. If you did it once and the process of closing channels was started, DO NOT do it again.
Yes. I reset user data and started with restoring with seeds and channels.
Sorry for confusing you. I downloaded it from the umbrel dashboard right before the recovery procedure.
okay. I did it exactly as described here
Yes. I am just waiting. I am seeing some of the channels closed in 1ml.com, but any corresponding fund has not been returned. Is this a usual case?
The instruction says “After you run this, wait for 1 minute. You should now be able to see your channels being closed on http://umbrel.local/lightning .”
-----> I did not see any peer appear on the dashboard during the procedure.
but, I saw the onchain balance change occasionally (like when trying loop-out) during the procedure.
Also says, “You should wait patiently until the funds are coming back to your onchain wallet. It will take at least 40 blocks.”
-----> I hope this is my case.
I checked the blockchain explorer and this is what I found.
Some of my channels were closed after I had started the recovery procedure.
According to the block chain explorer, the funds were first sent to a P2WSH address. And then the fund in the peer side has been returned to a P2WPKH address , but mine is not. It is hanging in the P2WSH address.
My question regarding this is:
Are the funds in both sides returned simulataneously if everything went well?
Is there any log file in which I can find a clue about the detail on the channel close.
There are other channels seen as active in 1ml.com.
What can I do with these channels?
I’ll wait for now, but can you let me know what happen if I redo the recovery after wiping out the node (or using blixt)
To check the status of a closing channel you better run the CLI as it was explained in the troubleshooting manual here - see the section about how to check the closed channels status.
You can start force closing on them if you see that are in limbo. Could be sometimes, when the peer is not responding. It will be more costly but you could release the funds.
Remember: in a channel dispute will be returned ONLY the funds that were on your side at the moment of closure. Sometimes all the balance of the channel could be on the peer side, so you get NOTHING.
Important is that you see all channels closed in your node listing.