What’s up with this?
Anyone else upgrade and no boot after? Stuck on loading.
I have to admit, getting kind of sick of Umbrel man, too fragile.
1 Like
Yeah, happened this morning. Feel like I just bricked my Rasberry Pi.
I re-flashed my microSD with the latest software. Plugged it back into the pi, expecting it to be a fresh install where I need to enter my seed, apply channel backups and do a recovery.
Recovery just means the new node comes up and initiates channel closes with the peers. That is the recover, the recover is over. That is a lightning thing not an Umbrel thing.
However, I was confused to see that my nodes hostname, apps, and old password were still there.
I can not explain that except that info must be stored on the external disk where I store the chain.
It did boot into the web GUI and ssh access, but it’s stuck on “starting” now for about 12 hours.
I’m done. I am going to spin up a new node, shut everything down and get my funds back on-chain.
But I am traveling today and will be gone for 10 days. So 11 days of a dead lightning node.
I know there is some default amount of time I have to do the closures before losing my funds but not sure how long that is.
Super bummed up. Umbrel gave it a good go but the bottom line is that it’s way too buggy to be a lightning node with funds on it.
Sorry guys, I’m out.
1 Like
Hey @ClarionChain, you can always reflash the MicroSD card at anytime and not lose any data, this is a common troubleshooting step that resolves issues due to MicroSD card failures and corruptions. I usually recommend replacing the MicroSD card every 6 months - 1 year, have another one to swap in as they’re only about $10-20, or of course switch to a system that does not rely on a MicroSD card to avoid any interruptions.
Whenever you’re back we can get back into your dashboard, let me know if your screen says “Starting” or is accompanied with any other error. When you’re available I can assist you make sure funds are safe. The Lightning Node downtime you mentioned is > 2 weeks of downtime.
If it only says “Starting Umbrel,” check on another browser or your phone to see if you can login in now, or SSH and run: sudo reboot Then wait ~5-10 minutes for Pi 4 to boot up
Things that can cause this are non-official Pi 4 power supply, possibly installing a community app, or power interruptions from a failing MicroSD
Feel free to ping me to get back in your dashboard and happy to help you make sure all sats are secure
Hey @Billythesquid let me know the error you’re getting too and happy to help you get back into dashboard ASAP, can post your error here as screenshot or I can assist on Telegram as well to help accomplish whatever you’d like to do
Thank you.
What’s the default time before you lose your funds in lightning channels?
If a node goes offline and never comes back doesn’t the other peer get the funds after a certain amount of time?
Or is there way a mechanism without my node giving a update that returns the fees to both?
Error: System service failed is a read/write issue (or also in the case there’s already existing partitions in the drive during installation), it may be redundant but reflashing the MicroSD has been the most common fix
SSH and run sudo shutdown
then unplug power cable
Reflash MicroSD card and then plug in power last
It may be redundant but we can try a new MicroSD card if possible, this will likely get you back into your dashboard
If after multiple attempts this is not working we’d have to format the drive or use new hardware, starting with switching USB 3.0 ports, a new USB 3.0 cable, and isolate the issue
Your node is down but your channel is still on the lightning network, your peer can take manual intervention processes to force close and sweep their own funds back onchain, once your node is back online you can get your funds back - they would need to compromise your node and sweep your funds or for instance if you have two nodes online running the same seed phrase this broadcasts stale channel states and this could result in your funds getting swept due to the disincentives built into the lightning network (make sure you only have one seed phrase per one online node at a time) - so as long as you have your seed phrase you are fine until you’re back up, more information: https://thebitcoinmanual.com/articles/zombie-lightning-channels/
Here’s more information you may find helpful on how that process looks if you are dealing with zombie channels:
This is typically the process to bump/get funds out of waiting close channels:
https://docs.lightning.engineering/lightning-network-tools/lnd/unconfirmed-bitcoin-transactions
You can prepend all commands for lncli like so:
sudo ~/umbrel/scripts/app compose lightning exec lnd lncli <command>
You can use this command to see status of channels:
~/umbrel/scripts/app compose lightning exec lnd lncli pendingchannels
Any channels stuck in waiting close with a problematic peer you can attempt to force close:
~/umbrel/scripts/app compose lightning exec lnd lncli closechannel --force <channel point>
You can also see that process here in an easy guide: https://www.massmux.com/how-to-resolve-waiting-close-status-lightning-channel/
It can take anywhere from the next block to 2 weeks for one lightning node to recover their own liquidity from offline/zombie channels - if in the off chance we have zombie channels we can try to reach out to the peer directly to request a close on their side if you know who they are or we’d have to move to other manual intervention steps like Guggero’s chantools: https://github.com/lightninglabs/chantools
Here are also a lot of other helpful resources in this regard:
https://docs.lightning.engineering/lightning-network-tools/lnd/safety#zombie-channels
https://thebitcoinmanual.com/articles/zombie-lightning-channels/#:~:text=Zombie%20channels%20are%20lightning%20channels,didn’t%20close%20its%20channels.
https://github.com/guggero/chantools
https://github.com/guggero/chantools/blob/master/doc/zombierecovery.md#file-format
https://node-recovery.com/
Awesome. Thx.
Thanks, When trying to SSH, I get this after password entry
Connection closed by 2601:c6:d37e:ab60::f950 port 22
Here is Pi 4 troubleshooting to help resolve:
Make sure the device you’re connecting from is on the same local network and all VPNs are turned off, it must know it’s on the same local network, open a terminal window on Mac or command prompt or powershell on Windows then type like this: ssh -t umbrel@umbrel.local
Then if it lets you access it try to reboot with the command: sudo reboot
If it’s not accessible we’ll need to see if it’s detected on your network at all, you can check your router’s device list or use an app like Angry IP scanner to easily scan your network to see if your node’s IP shows up: https://angryip.org/
Your Umbrel’s IP should display next to the host name umbrel.local and you can see if you can reach Umbrel using the IP in your browser’s address bar instead or by SSHing like this:
ssh umbrel@YourNode'sIP
Check to see if there is a green flashing light on your Pi. There must be a green flashing light to indicate the MicroSD card is being read, if not we must reflash the MicroSD card, and then we can plug it back in and try once more.