Hi, please, help. All my channels are offline in Lightning Node App after updating to Umbrel 0.5.1. It’s been 1 hr since the update and all channels still show offline.
I have updated to Lightning Node LND 0.15.1-beta and channels are still offline.
In Thunderhub, I see negative Sats available. Is this due to a stuck LND transaction?
Available -26,955
Any help is greatly appreciated. Or if just more time has to pass for Lightning Node to sync, please advise.
Thanks.
I restarted Umbrel and Lightning Node restarted as well and the channels came back online.
I have a similar problem. My node apparently lost its Internet connection a week ago while I wasn’t paying attention to it. I have restarted it, the “Bitcoin Node” application is all synced up with the blockchain, but my “Lightning Node” application shows all channels offline, and is spinning on the update to v0.15.2-beta (i.e., the green “Updating” button is throbbing). Restarting the node seems to have no benefit. Am I just being impatient? How many hours is reasonable to wait to see the update finish, and Lightning channels come online?
What version of Umbrel are you running? Are you running Umbrel 0.5.1?
Yes. I’ve been running 0.5.1 almost since it released.
I had issues with my channels offline again 5 days ago while Bitcoin Node was synced this time. Updating “Lightning Node” from to v0.15.1-beta to v0.15.2-beta seemed to fix that problem. But you’re saying that the update to v0.15.2-beta isn’t going through?
Correct. I wonder if the fact that channels haven’t opened yet is related to the update not going through? The currently running LND shows as 0.15.1-beta, and definitely is using CPU:
$ docker ps | rg lightninglabs/lnd
e990927c0bec lightninglabs/lnd:v0.15.1-beta "lnd --listen=0.0.0.…" 30 minutes ago Up 30 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:9735->9735/tcp, :::9735->9735/tcp, 0.0.0.0:10009->10009/tcp, :::10009->10009/tcp lightning_lnd_1
“procs” output:
PID:▲ User │ TTY CPU MEM CPU Time │ Command
236233 xxxx │ 191.4 2.8 00:04:50 │ lnd --listen=0.0.0.0:9735My Lightning Node UI shows 0 connection as well. The log shows, I think, normal startup, at least as far as TOR network is concerned:
lightning
tor_server_1 | Oct 16 19:29:24.812 [notice] Opening Socks listener on 127.0.0.1:9050
tor_server_1 | Oct 16 19:29:24.843 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050
tor_server_1 | Oct 16 19:29:25.000 [notice] Bootstrapped 0% (starting): Starting
tor_server_1 | Oct 16 19:29:25.000 [notice] Starting with guard context "default"
tor_server_1 | Oct 16 19:29:26.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
tor_server_1 | Oct 16 19:29:26.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
tor_server_1 | Oct 16 19:29:26.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
tor_server_1 | Oct 16 19:29:27.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
tor_server_1 | Oct 16 19:29:27.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
tor_server_1 | Oct 16 19:29:27.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
tor_server_1 | Oct 16 19:29:27.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus
tor_server_1 | Oct 16 19:29:29.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
tor_server_1 | Oct 16 19:29:30.000 [notice] Bootstrapped 40% (loading_keys): Loading authority key certs
tor_server_1 | Oct 16 19:29:32.000 [notice] Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors
tor_server_1 | Oct 16 19:29:32.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6206, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.)
tor_server_1 | Oct 16 19:29:32.000 [warn] Tried connecting to router at 51.81.155.81:9001 ID=<none> RSA_ID=FB0DDFE915CB93D9489D3E644EF46B8F4E51967B, but RSA + ed25519 identity keys were not as expected: wanted FB0DDFE915CB93D9489D3E644EF46B8F4E51967B + no ed25519 key but got BB4BBDD4895E9481A8F16C311D22CC2BB7B1D6BF + WluugT10MBptUc+cpcW7U/Cgg+g5QVF0o4iNG+WWKW0.
tor_server_1 | Oct 16 19:29:33.000 [notice] Bootstrapped 50% (loading_descriptors): Loading relay descriptors
tor_server_1 | Oct 16 19:29:35.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
tor_server_1 | Oct 16 19:29:41.000 [notice] Bootstrapped 55% (loading_descriptors): Loading relay descriptors
tor_server_1 | Oct 16 19:29:44.000 [notice] Bootstrapped 62% (loading_descriptors): Loading relay descriptors
tor_server_1 | Oct 16 19:29:45.000 [notice] Bootstrapped 67% (loading_descriptors): Loading relay descriptors
tor_server_1 | Oct 16 19:29:46.000 [notice] Bootstrapped 73% (loading_descriptors): Loading relay descriptors
tor_server_1 | Oct 16 19:29:46.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
tor_server_1 | Oct 16 19:29:47.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
tor_server_1 | Oct 16 19:29:47.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
tor_server_1 | Oct 16 19:29:48.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
tor_server_1 | Oct 16 19:29:48.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
tor_server_1 | Oct 16 19:29:48.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
tor_server_1 | Oct 16 19:29:48.000 [notice] Bootstrapped 100% (done): DoneVery likely so. I don’t know and couldn’t tell you what else to do from here. Perhaps Umbrel support team can advise better, in my experience you’ll hear from them on weekdays by 6am PT.
I think my node might be hitting the LND “bug” that prompted the Umbrel folks to push the new version of LND. From my docker logs for the LND container:
2022-10-16 19:32:04.848 [INF] LNWL: Started rescan from block 00000000000000000007477b90ff0ea53cdd1db88e03799af18ff58df0cebaa7 (height 757921) for 92 addresses
2022-10-16 19:32:05.622 [ERR] LNWL: Unable to complete chain rescan: readScript: script witness item is larger than the max allowed size [count 33970, max 11000]
2022-10-16 19:32:09.912 [INF] INVC: New invoice subscription client: id=1
2022-10-16 19:33:39.725 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:33:40.073 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:52:11.423 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:52:11.603 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:52:29.786 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:52:30.096 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:52:33.177 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:52:33.585 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:53:19.446 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:53:19.860 [ERR] RPCS: [/lnrpc.Lightning/GetNodeInfo]: rpc error: code = NotFound desc = unable to find node
2022-10-16 19:54:09.689 [INF] RPCS: New transaction subscription
2022-10-16 19:55:15.688 [INF] RPCS: Canceling transaction subscriptionAlso, I tried the steps in troubleshooting to try and manually update the LND container image, but I think I’m specifying it wrong:
$ sudo ./scripts/update/update --repo lightninglabs/lnd#v0.15.2-beta
Cloning into '/tmp/umbrel-update'...
Username for 'https://github.com': dwvisser
Password for 'https://dwvisser@github.com':
remote: Repository not found.
fatal: repository 'https://github.com/lightninglabs/lnd.git/' not found
fatal: not a git repository: '/tmp/umbrel-update//.git'
mv: cannot stat '/tmp/umbrel-update//info.json': No such file or directory
jq: error: Could not open file /tmp/umbrel-update//info.json.orig: No such file or directory
Update path doesn't seem to be an Umbrel install.The problem resolved this morning. I logged into my Umbrel GUI, and noticed that it would allow me to attempt the LND upgrade. I crossed my fingers and did it. Within seconds the update completed, and now, some minutes later, my channels are connected again. (Well, one is being closed by the other party, but I only had about 10000 sats in that one anyway.)
The first lesson I’ve learned: make darn sure your Umbrel is connecting to the Internet before you try to update any app.
The second lesson: If you do get a “stuck” update because of this, it will eventually “unstick” itself.
Glad to hear that Lightning Node update followed through. I am seeing more of my channels periodically go offline now than any time in the past 1 year. Hopefully it’s a sign of LND pushing new software at the moment and will get to a more stable state in the future again.
How are you upgrading the Lightning Node to v0.15.x-beta? I’m using Umbrel, having the problem you described for about a week now, and noticed that while my Umbrel install is 0.5.1, my LND is 0.14.1-beta?!? Are you using the Umbrel release??
Thanks.
Login to to Umbrel --> Settings. Do you see any available updates in the top right corner?
Sadly, no. I’ve clicked on the Umbrel “check for updates” with nothing new right now.
That’s why I was surprised about your LND version being a full version ahead of me…
Try this: Login to Umbrel --> App Store --> Lightning Node. See if there is a way to update directly.
That was a good tip. In the App Store → Lightning Node, it shows that the updated Lightning Node is in there:
0.15.2-beta, but in the Umbrel main screen Lightning Node app, it shows the old version:
So maybe there is an upgrade failure somewhere along the way. I’ll chase this vector for a while and see if I can find any tips in the forums. Thank you!
Arggghhh…
Looks like I had to manually update the software… That’s insane that Umbrel doesn’t automatically do this for me!
Let’s see if this resolves my issues.
The LND is now updated. Waiting to see if it takes overnight to start working again. Thanks for pointing me in the (hopefully) right direction!
