Nothing but issues on my Raspberry Pi 4 since recent update

Ever since the update to 0.5.2 and subsequently 0.5.3 I have had nothing but problems on my Raspberry Pi 4. I mostly use Thunderhub to check forwards and open and rebalance channels, but since the update it has been extremely difficult to reach it. Usually I get a “cannot connect to node” or something similar when trying to login, but after a couple attempts I can manage to get through somehow.

However, today, I have had no consistent luck with ANY app (Bitcoin Node, RTL, Lightning Node, etc.). I have deleted and reinstalled most of them, restarted my pi, tried different browsers, etc. And when I was getting the “replace sd card” bug a few weeks ago I replaced the sd card. I cannot find anything online where anyone is having the same issue. Not sure where to turn.

Apologies if this has ever been answered previously or if I am missing anything – I’m at a very low level in terms of how everything works. Where should I begin?

There is a new user limitation preventing me from posting full troubleshooting logs here but here are some.

=====================
= Umbrel debug info =

Umbrel version

0.5.3

Flashed OS version

v0.5.1

Raspberry Pi Model

Revision : c03111
Serial : 10000000fd77dbcd
Model : Raspberry Pi 4 Model B Rev 1.1

Firmware

Dec 1 2021 15:01:54
Copyright © 2012 Broadcom
version 71bd3109023a0c8575585ba87cbb374d2eeb038f (clean) (release) (start)

Temperature

temp=52.1’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 3.8G 2.5G 142M 4.0M 1.1G 1.2G
Swap: 4.1G 17M 4.1G

total: 66.7%
bitcoin: 19.8%
lightning: 11.5%
bitfeed: 8.3%
thunderhub: 7.3%
core-lightning: 6.7%
ride-the-lightning: 5.9%
bluewallet: 4.4%
system: 2.8%

Memory monitor logs

2022-11-15 17:59:06 Warning memory usage at 91%
2022-11-15 18:00:11 Warning memory usage at 91%
2022-11-15 18:01:15 Warning memory usage at 91%
2022-11-15 18:02:19 Warning memory usage at 91%
2022-11-15 18:03:23 Warning memory usage at 91%
2022-11-15 18:04:25 Warning memory usage at 91%
2022-11-19 04:45:05 Memory monitor running!
2022-11-22 22:52:08 Memory monitor running!
2022-11-22 23:27:51 Memory monitor running!
2022-11-22 23:52:18 Memory monitor running!

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 3.2G 25G 12% /
/dev/sda1 916G 527G 343G 61% /home/umbrel/umbrel

Startup service logs

Nov 22 23:58:33 umbrel umbrel startup[1158]: Creating bluewallet_redis_1 … done
Nov 22 23:58:33 umbrel umbrel startup[1158]: Creating bluewallet_lndhub_1 …
Nov 22 23:58:36 umbrel umbrel startup[1158]: Creating lightning_app_proxy_1 … done
Nov 22 23:58:39 umbrel umbrel startup[1158]: Creating bitcoin_bitcoind_1 … done
Nov 22 23:58:39 umbrel umbrel startup[1158]: Creating bitcoin_server_1 …
Nov 22 23:58:40 umbrel umbrel startup[1158]: Creating bitcoin_tor_1 … done
Nov 22 23:58:40 umbrel umbrel startup[1158]: Creating thunderhub_app_proxy_1 … done
Nov 22 23:58:41 umbrel umbrel startup[1158]: Creating bluewallet_app_proxy_1 … done
Nov 22 23:58:42 umbrel umbrel startup[1158]: Creating thunderhub_tor_server_1 … done
Nov 22 23:58:42 umbrel umbrel startup[1158]: Creating bitfeed_app_proxy_1 … done
Nov 22 23:58:44 umbrel umbrel startup[1158]: Creating bitfeed_tor_server_1 … done
Nov 22 23:58:45 umbrel umbrel startup[1158]: Creating thunderhub_web_1 … done
Nov 22 23:58:46 umbrel umbrel startup[1158]: Creating bitcoin_tor_server_1 … done
Nov 22 23:58:49 umbrel umbrel startup[1158]: Creating lightning_tor_1 … done
Nov 22 23:58:51 umbrel umbrel startup[1158]: Creating bitfeed_api_1 … done
Nov 22 23:58:51 umbrel umbrel startup[1158]: Creating bitfeed_web_1 …
Nov 22 23:58:51 umbrel umbrel startup[1158]: Creating bluewallet_tor_server_1 … done
Nov 22 23:58:55 umbrel umbrel startup[1158]: Creating lightning_app_1 … done
Nov 22 23:58:57 umbrel umbrel startup[1158]: Creating lightning_tor_server_1 … done
Nov 22 23:59:00 umbrel umbrel startup[1158]: Creating bitcoin_server_1 … done
Nov 22 23:59:01 umbrel umbrel startup[1158]: Creating lightning_lnd_1 … done
Nov 22 23:59:07 umbrel umbrel startup[1158]: Creating bitcoin_app_proxy_1 … done
Nov 22 23:59:07 umbrel umbrel startup[1158]: Creating bluewallet_lndhub_1 … done
Nov 22 23:59:23 umbrel umbrel startup[1158]: Creating bitfeed_web_1 … done
Nov 22 23:59:23 umbrel umbrel startup[1158]: Umbrel is now accessible at
Nov 22 23:59:23 umbrel umbrel startup[1158]: http://umbrel.local
Nov 22 23:59:23 umbrel umbrel startup[1158]: http://192.168.0.2
Nov 22 23:59:23 umbrel systemd[1]: Started Umbrel Startup Service.
Nov 22 23:59:58 umbrel passwd[8859]: pam_unix(passwd:chauthtok): password changed for umbrel

External storage service logs

Nov 22 23:51:57 umbrel systemd[1]: Starting External Storage Mounter…
Nov 22 23:51:57 umbrel external storage mounter[479]: Running external storage mount script…
Nov 22 23:51:57 umbrel external storage mounter[479]: Found device “External USB3.0”
Nov 22 23:51:57 umbrel external storage mounter[479]: Blacklisting USB device IDs against UAS driver…
Nov 22 23:51:57 umbrel external storage mounter[479]: Rebinding USB drivers…
Nov 22 23:51:57 umbrel external storage mounter[479]: Checking USB devices are back…
Nov 22 23:51:57 umbrel external storage mounter[479]: Waiting for USB devices…
Nov 22 23:51:58 umbrel external storage mounter[479]: Waiting for USB devices…
Nov 22 23:51:59 umbrel external storage mounter[479]: Checking if the device is ext4…
Nov 22 23:51:59 umbrel external storage mounter[479]: Yes, it is ext4
Nov 22 23:51:59 umbrel external storage mounter[479]: Checking filesystem for corruption…
Nov 22 23:51:59 umbrel external storage mounter[479]: e2fsck 1.44.5 (15-Dec-2018)
Nov 22 23:52:00 umbrel external storage mounter[479]: umbrel: clean, 312043/61054976 files, 141493272/244190208 blocks
Nov 22 23:52:00 umbrel external storage mounter[479]: Mounting partition…
Nov 22 23:52:00 umbrel external storage mounter[479]: Checking if device contains an Umbrel install…
Nov 22 23:52:00 umbrel external storage mounter[479]: Yes, it contains an Umbrel install
Nov 22 23:52:00 umbrel external storage mounter[479]: Bind mounting external storage over local Umbrel installation…
Nov 22 23:52:00 umbrel external storage mounter[479]: Bind mounting external storage over local Docker data dir…
Nov 22 23:52:00 umbrel external storage mounter[479]: Bind mounting external storage to /swap
Nov 22 23:52:00 umbrel external storage mounter[479]: Bind mounting SD card root at /sd-card…
Nov 22 23:52:00 umbrel external storage mounter[479]: Checking Umbrel root is now on external storage…
Nov 22 23:52:01 umbrel external storage mounter[479]: Checking /var/lib/docker is now on external storage…
Nov 22 23:52:01 umbrel external storage mounter[479]: Checking /swap is now on external storage…
Nov 22 23:52:01 umbrel external storage mounter[479]: Setting up swapfile
Nov 22 23:52:08 umbrel external storage mounter[479]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
Nov 22 23:52:08 umbrel external storage mounter[479]: no label, UUID=c3e043fb-df6d-45fa-be89-c3993fbbc14d
Nov 22 23:52:08 umbrel external storage mounter[479]: Checking SD Card root is bind mounted at /sd-root…
Nov 22 23:52:08 umbrel external storage mounter[479]: Starting external drive mount monitor…
Nov 22 23:52:08 umbrel external storage mounter[479]: Mount script completed successfully!
Nov 22 23:52:08 umbrel systemd[1]: Started External Storage Mounter.

External storage SD card update service logs

– Logs begin at Tue 2022-11-22 23:51:52 UTC, end at Wed 2022-11-23 00:17:57 UTC. –
Nov 22 23:52:18 umbrel systemd[1]: Starting External Storage SDcard Updater…
Nov 22 23:52:18 umbrel external storage updater[1074]: Checking if SD card Umbrel is newer than external storage…
Nov 22 23:52:18 umbrel external storage updater[1074]: No, SD version is not newer, exiting.
Nov 22 23:52:18 umbrel systemd[1]: Started External Storage SDcard Updater.

Karen logs

Pulling web … extracting (69.8%)
Pulling web … extracting (71.2%)
Pulling web … extracting (72.5%)
Pulling web … extracting (73.9%)
Pulling web … extracting (75.9%)
Pulling web … extracting (77.3%)
Pulling web … extracting (77.9%)
Pulling web … extracting (81.3%)
Pulling web … extracting (82.7%)
Pulling web … extracting (83.4%)
Pulling web … extracting (84.0%)
Pulling web … extracting (84.7%)
Pulling web … extracting (85.4%)
Pulling web … extracting (86.1%)
Pulling web … extracting (86.7%)
Pulling web … extracting (88.1%)
Pulling web … extracting (89.5%)
Pulling web … extracting (90.1%)
Pulling web … extracting (91.5%)
Pulling web … extracting (92.2%)
Pulling web … extracting (92.8%)
Pulling web … extracting (93.5%)
Pulling web … extracting (94.2%)
Pulling web … extracting (95.6%)
Pulling web … extracting (96.2%)
Pulling web … extracting (97.6%)
Pulling web … extracting (98.3%)
Pulling web … extracting (98.9%)
Pulling web … extracting (99.6%)
Pulling web … extracting (100.0%)
Pulling web … pull complete
Pulling web … extracting (11.7%)
Pulling web … extracting (100.0%)
Pulling web … pull complete
Pulling web … digest: sha256:68f837be0acae9a19f…
Pulling web … status: downloaded newer image fo…
Pulling web … done
Starting app ride-the-lightning…
Creating ride-the-lightning_boltz_1 …
Creating ride-the-lightning_web_1 …
Creating ride-the-lightning_tor_server_1 …
Creating ride-the-lightning_app_proxy_1 …
Creating ride-the-lightning_tor_server_1 … done
Creating ride-the-lightning_web_1 … done
Creating ride-the-lightning_app_proxy_1 … done
Creating ride-the-lightning_boltz_1 … done
Saving app ride-the-lightning in DB…
Successfully installed app ride-the-lightning
Got signal: debug
karen is getting triggered!

As soon as I finished this post I was able to get on some apps – very slow, but I am on. Confirmed I am fully synched and running.

Definitely open to any best practices here – i don’t expect to stay connected forever.

Logs look OK. You seem to be running both Lightning and Core Lightning together but you only need one of those (just check which one is the active one on Dashboard page of Ride the Lightning). It may free up some CPU & memory and speed things up a bit.

Deleted core lightning, freed up space, pi is running cooler, other apps are running well. THANK YOU

1 Like