No problem logging in, but cannot start services

[amadeus]

====================== Umbrel debug info ======================Umbrel ve -...

[amadeus]

Some more context. The Umbrel was failing for some time, though it appeared to stabilize after multiple reflashings of SD card and permitted me to login/resync my node. Subsequently, there was a power outage, and reflashing the SD has not worked. Thinking the 2 year old card might be dying, I purchased a new one and did a fresh install, but the same behavior. I can login, but all the services are ‘starting’, and I’m unable to do the 0.5.4 install, even though that’s what was flashed on the new SD card. I SSH’d in to get a debug log which I posted in the OP. Would love to get this thing working…

[usernameisJim]

Only run this if you’re already on 0.5.3, and since we already reflashed and still shows starting this should manually update us to 0.5.4:

cd ~/umbrel && echo '{"state": "success", "progress": 100, "description": "Successfully installed Umbrel v0.5.3", "updateTo": "v0.5.4"}' | sudo tee statuses/update-status.json && sudo scripts/update/update --ota

Once complete you can wait ~5 minutes, if still unresponsive after a reboot on the Pi 4 we might just need to wait a little while longer and then re-open a new window for everything to finish loading

[amadeus]

Umbrel has updated to 0.5.4, but the services still aren’t starting:

=====================
= Umbrel debug info =

Umbrel version

0.5.4

Flashed OS version

v0.5.3

Raspberry Pi Model

Revision : d03114
Serial : 100000000a42cf81
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

Dec 1 2021 15:01:54
Copyright (c) 2012 Broadcom
version 71bd3109023a0c8575585ba87cbb374d2eeb038f (clean) (release) (start)

Temperature

temp=51.6’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 7.8G 467M 6.6G 24M 731M 7.2G
Swap: 4.1G 0B 4.1G

total: 6.0%
system: 6%

Memory monitor logs

2023-07-13 23:13:05 Memory monitor running!
2023-07-13 23:13:18 Memory monitor running!
2023-07-13 23:13:30 Memory monitor running!
2023-07-13 23:13:43 Memory monitor running!
2023-07-13 23:13:55 Memory monitor running!
2023-07-13 23:14:08 Memory monitor running!
2023-07-13 23:14:20 Memory monitor running!
2023-07-13 23:14:33 Memory monitor running!
2023-07-13 23:14:46 Memory monitor running!
2023-07-13 23:14:58 Memory monitor running!

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 30G 3.1G 25G 11% /
/dev/sda1 916G 620G 250G 72% /home/umbrel/umbrel

Startup service logs

Jul 13 23:14:45 umbrel umbrel startup[4008]: Starting Docker services…
Jul 13 23:14:47 umbrel umbrel startup[4008]: tor_server is up-to-date
Jul 13 23:14:47 umbrel umbrel startup[4008]: auth is up-to-date
Jul 13 23:14:47 umbrel umbrel startup[4008]: dashboard is up-to-date
Jul 13 23:14:47 umbrel umbrel startup[4008]: tor_proxy is up-to-date
Jul 13 23:14:47 umbrel umbrel startup[4008]: manager is up-to-date
Jul 13 23:14:47 umbrel umbrel startup[4008]: nginx is up-to-date
Jul 13 23:14:48 umbrel umbrel startup[4008]: Removing status server iptables entry…
Jul 13 23:14:48 umbrel umbrel startup[4008]: No existing iptables entry found.
Jul 13 23:14:48 umbrel umbrel startup[4008]: Starting installed apps…
Jul 13 23:14:48 umbrel umbrel startup[4008]: /home/umbrel/umbrel/scripts/start: line 169: /home/umbrel/umbrel/scripts/…/scripts/app: No such file or directory
Jul 13 23:14:48 umbrel systemd[1]: umbrel-startup.service: Control process exited, code=exited, status=127/n/a
Jul 13 23:14:48 umbrel systemd[1]: umbrel-startup.service: Failed with result ‘exit-code’.
Jul 13 23:14:48 umbrel systemd[1]: Failed to start Umbrel Startup Service.
Jul 13 23:14:58 umbrel systemd[1]: umbrel-startup.service: Service RestartSec=10s expired, scheduling restart.
Jul 13 23:14:58 umbrel systemd[1]: umbrel-startup.service: Scheduled restart job, restart counter is at 905.
Jul 13 23:14:58 umbrel systemd[1]: Stopped Umbrel Startup Service.
Jul 13 23:14:58 umbrel systemd[1]: Starting Umbrel Startup Service…
Jul 13 23:14:58 umbrel umbrel startup[4283]: ======================================
Jul 13 23:14:58 umbrel umbrel startup[4283]: ============= STARTING ===============
Jul 13 23:14:58 umbrel umbrel startup[4283]: ============== UMBREL ================
Jul 13 23:14:58 umbrel umbrel startup[4283]: ======================================
Jul 13 23:14:58 umbrel umbrel startup[4283]: Setting environment variables…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting karen…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting status monitors…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting memory monitor…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting backup monitor…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting decoy backup trigger…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting umbreld…
Jul 13 23:14:58 umbrel umbrel startup[4283]: Starting Docker services…

External storage service logs

Jul 13 20:05:34 umbrel external storage mounter[480]: Running external storage mount script…
Jul 13 20:05:34 umbrel external storage mounter[480]: Found device “WDC WDS1 00T2G0A-00JH”
Jul 13 20:05:34 umbrel external storage mounter[480]: Blacklisting USB device IDs against UAS driver…
Jul 13 20:05:34 umbrel external storage mounter[480]: Rebinding USB drivers…
Jul 13 20:05:35 umbrel external storage mounter[480]: Checking USB devices are back…
Jul 13 20:05:35 umbrel external storage mounter[480]: Waiting for USB devices…
Jul 13 20:05:36 umbrel external storage mounter[480]: Waiting for USB devices…
Jul 13 20:05:37 umbrel external storage mounter[480]: Waiting for USB devices…
Jul 13 20:05:38 umbrel external storage mounter[480]: Checking if the device is ext4…
Jul 13 20:05:38 umbrel external storage mounter[480]: Yes, it is ext4
Jul 13 20:05:38 umbrel external storage mounter[480]: Checking filesystem for corruption…
Jul 13 20:05:38 umbrel external storage mounter[480]: e2fsck 1.44.5 (15-Dec-2018)
Jul 13 20:05:38 umbrel external storage mounter[480]: umbrel: clean, 298991/61054976 files, 166595787/244190208 blocks
Jul 13 20:05:38 umbrel external storage mounter[480]: Mounting partition…
Jul 13 20:05:38 umbrel external storage mounter[480]: Checking if device contains an Umbrel install…
Jul 13 20:05:38 umbrel external storage mounter[480]: Yes, it contains an Umbrel install
Jul 13 20:05:38 umbrel external storage mounter[480]: Bind mounting external storage over local Umbrel installation…
Jul 13 20:05:38 umbrel external storage mounter[480]: Bind mounting external storage over local Docker data dir…
Jul 13 20:05:38 umbrel external storage mounter[480]: Bind mounting external storage to /swap
Jul 13 20:05:38 umbrel external storage mounter[480]: Bind mounting SD card root at /sd-card…
Jul 13 20:05:38 umbrel external storage mounter[480]: Checking Umbrel root is now on external storage…
Jul 13 20:05:39 umbrel external storage mounter[480]: Checking /var/lib/docker is now on external storage…
Jul 13 20:05:39 umbrel external storage mounter[480]: Checking /swap is now on external storage…
Jul 13 20:05:39 umbrel external storage mounter[480]: Setting up swapfile
Jul 13 20:05:41 umbrel external storage mounter[480]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
Jul 13 20:05:41 umbrel external storage mounter[480]: no label, UUID=3943d3cf-375a-419e-b547-76a07e2af5f7
Jul 13 20:05:41 umbrel external storage mounter[480]: Checking SD Card root is bind mounted at /sd-root…
Jul 13 20:05:41 umbrel external storage mounter[480]: Starting external drive mount monitor…
Jul 13 20:05:41 umbrel external storage mounter[480]: Mount script completed successfully!
Jul 13 20:05:41 umbrel systemd[1]: Started External Storage Mounter.

External storage SD card update service logs

– Logs begin at Thu 2023-07-13 20:05:29 UTC, end at Thu 2023-07-13 23:14:59 UTC. –
Jul 13 20:05:48 umbrel systemd[1]: Starting External Storage SDcard Updater…
Jul 13 20:05:48 umbrel external storage updater[1021]: Checking if SD card Umbrel is newer than external storage…
Jul 13 20:05:48 umbrel external storage updater[1021]: No, SD version is not newer, exiting.
Jul 13 20:05:48 umbrel systemd[1]: Started External Storage SDcard Updater.

Karen logs

karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events

Docker containers

NAMES STATUS
nginx Up 3 hours
auth Up 3 hours
tor_server Up 3 hours
tor_proxy Up 3 hours
manager Up 3 hours
dashboard Up 3 hours

Umbrel logs

Attaching to manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:49 GMT] “GET /v1/system/memory HTTP/1.0” 200 84 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:49 GMT] “GET /v1/system/storage HTTP/1.0” 200 92 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:49 GMT] “GET /v1/system/get-update HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:49 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:51 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:53 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:55 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:57 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:14:59 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Thu, 13 Jul 2023 23:15:01 GMT] “GET /v1/system/debug-result HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager

Tor Proxy logs

Attaching to tor_proxy
tor_proxy | Jul 13 20:06:17.000 [notice] Bootstrapped 57% (loading_descriptors): Loading relay descriptors
tor_proxy | Jul 13 20:06:17.000 [notice] Bootstrapped 64% (loading_descriptors): Loading relay descriptors
tor_proxy | Jul 13 20:06:18.000 [notice] Bootstrapped 69% (loading_descriptors): Loading relay descriptors
tor_proxy | Jul 13 20:06:18.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
tor_proxy | Jul 13 20:06:18.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
tor_proxy | Jul 13 20:06:18.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
tor_proxy | Jul 13 20:06:19.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
tor_proxy | Jul 13 20:06:19.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
tor_proxy | Jul 13 20:06:19.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
tor_proxy | Jul 13 20:06:20.000 [notice] Bootstrapped 100% (done): Done
/home/umbrel/umbrel/scripts/debug: line 139: ./scripts/app: No such file or directory

==== Result ====

The debug script did not automatically detect any issues with your Umbrel.
Connection to umbrel.local closed.

[usernameisJim]

So we’re on v0.5.4 but flashed version is still 0.5.3, if applications are taking a while to start on Pi 4 and haven’t become responsive after 5-15 minutes and opening a new window, I know it’s redundant but try reflashing or using a new MicroSD card, and then we can wait for it to then become responsive- sometimes on the Pi 4 it just take a little while longer for everything to start-

Here is a way as well to check if MicroSD card is failing, as a note it can still be an issue or corrupt even if it doesn’t seem like speed is completely degrading so worth to try another one so your apps start faster if there is a read/write issue:

You can SSH into your Umbrel and run a quick speed test on the SD card using the following commands:

1. Install hdparm:

sudo apt-get install hdparm

2. Run speedtest on the SD card:

sudo hdparm -tT /dev/mmcblk0

3. Run speedtest again to get the average speed:

sudo hdparm -tT /dev/mmcblk0

Then post results of the speed test here if you’d like. If the speeds are super slow, it’s probably a dead SD card, and just flashing a new one with umbrelOS will get you going without any data loss.

[amadeus]

Thanks, I’ll reflash 0.5.4 again. In the meantime, here are the speeds from hdparm on the new SD card I installed a couple days ago.

[amadeus]

Reflashed, same problem on login.

=====================
= Umbrel debug info =

Umbrel version

0.5.4

Flashed OS version

v0.5.4

Raspberry Pi Model

Revision : d03114
Serial : 100000000a42cf81
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

May 9 2023 12:16:34
Copyright (c) 2012 Broadcom
version 30aa0d70ab280427ba04ebc718c81d4350b9d394 (clean) (release) (start)

Temperature

temp=49.6’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 7.8G 374M 6.4G 8.0M 1.0G 7.3G
Swap: 4.1G 0B 4.1G

total: 4.8%
system: 4.8%

Memory monitor logs

2023-07-15 18:45:17 Memory monitor running!
2023-07-15 18:45:29 Memory monitor running!
2023-07-15 18:45:42 Memory monitor running!
2023-07-15 18:45:55 Memory monitor running!
2023-07-15 18:46:07 Memory monitor running!
2023-07-15 18:46:20 Memory monitor running!
2023-07-15 18:46:32 Memory monitor running!
2023-07-15 18:46:45 Memory monitor running!
2023-07-15 18:46:58 Memory monitor running!
2023-07-15 18:47:10 Memory monitor running!

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 30G 3.2G 25G 12% /
/dev/sda1 916G 620G 250G 72% /home/umbrel/umbrel

Startup service logs

Jul 15 18:47:00 umbrel systemd[1]: Failed to start Umbrel Startup Service.
Jul 15 18:47:10 umbrel systemd[1]: umbrel-startup.service: Service RestartSec=10s expired, scheduling restart.
Jul 15 18:47:10 umbrel systemd[1]: umbrel-startup.service: Scheduled restart job, restart counter is at 19.
Jul 15 18:47:10 umbrel systemd[1]: Stopped Umbrel Startup Service.
Jul 15 18:47:10 umbrel systemd[1]: Starting Umbrel Startup Service…
Jul 15 18:47:10 umbrel umbrel startup[10358]: ======================================
Jul 15 18:47:10 umbrel umbrel startup[10358]: ============= STARTING ===============
Jul 15 18:47:10 umbrel umbrel startup[10358]: ============== UMBREL ================
Jul 15 18:47:10 umbrel umbrel startup[10358]: ======================================
Jul 15 18:47:10 umbrel umbrel startup[10358]: Setting environment variables…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting karen…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting status monitors…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting memory monitor…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting backup monitor…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting decoy backup trigger…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting umbreld…
Jul 15 18:47:10 umbrel umbrel startup[10358]: Starting Docker services…
Jul 15 18:47:12 umbrel umbrel startup[10358]: auth is up-to-date
Jul 15 18:47:12 umbrel umbrel startup[10358]: tor_server is up-to-date
Jul 15 18:47:12 umbrel umbrel startup[10358]: dashboard is up-to-date
Jul 15 18:47:12 umbrel umbrel startup[10358]: tor_proxy is up-to-date
Jul 15 18:47:12 umbrel umbrel startup[10358]: manager is up-to-date
Jul 15 18:47:12 umbrel umbrel startup[10358]: nginx is up-to-date
Jul 15 18:47:12 umbrel umbrel startup[10358]: Removing status server iptables entry…
Jul 15 18:47:12 umbrel umbrel startup[10358]: No existing iptables entry found.
Jul 15 18:47:12 umbrel umbrel startup[10358]: Starting installed apps…
Jul 15 18:47:12 umbrel umbrel startup[10358]: /home/umbrel/umbrel/scripts/start: line 169: /home/umbrel/umbrel/scripts/…/scripts/app: No such file or directory
Jul 15 18:47:12 umbrel systemd[1]: umbrel-startup.service: Control process exited, code=exited, status=127/n/a
Jul 15 18:47:12 umbrel systemd[1]: umbrel-startup.service: Failed with result ‘exit-code’.
Jul 15 18:47:12 umbrel systemd[1]: Failed to start Umbrel Startup Service.

External storage service logs

Jun 22 16:44:12 umbrel external storage mounter[513]: e2fsck 1.44.5 (15-Dec-2018)
Jun 22 16:44:13 umbrel external storage mounter[513]: Superblock last mount time (Thu Jul 13 20:05:38 2023,
Jun 22 16:44:13 umbrel external storage mounter[513]: now = Thu Jun 22 16:44:12 2023) is in the future.
Jun 22 16:44:13 umbrel external storage mounter[513]: Fix? yes
Jun 22 16:44:13 umbrel external storage mounter[513]: Superblock last write time (Sat Jul 15 18:31:54 2023,
Jun 22 16:44:13 umbrel external storage mounter[513]: now = Thu Jun 22 16:44:12 2023) is in the future.
Jun 22 16:44:13 umbrel external storage mounter[513]: Fix? yes
Jun 22 16:44:13 umbrel external storage mounter[513]: Pass 1: Checking inodes, blocks, and sizes
Jun 22 16:44:20 umbrel external storage mounter[513]: Pass 2: Checking directory structure
Jun 22 16:44:25 umbrel external storage mounter[513]: Pass 3: Checking directory connectivity
Jun 22 16:44:26 umbrel external storage mounter[513]: Pass 4: Checking reference counts
Jun 22 16:44:30 umbrel external storage mounter[513]: Pass 5: Checking group summary information
Jun 22 16:44:30 umbrel external storage mounter[513]: umbrel: 299044/61054976 files (5.6% non-contiguous), 166605317/244190208 blocks
Jun 22 16:44:31 umbrel external storage mounter[513]: Mounting partition…
Jun 22 16:44:31 umbrel external storage mounter[513]: Checking if device contains an Umbrel install…
Jun 22 16:44:31 umbrel external storage mounter[513]: Yes, it contains an Umbrel install
Jun 22 16:44:31 umbrel external storage mounter[513]: Bind mounting external storage over local Umbrel installation…
Jun 22 16:44:31 umbrel external storage mounter[513]: Bind mounting external storage over local Docker data dir…
Jun 22 16:44:31 umbrel external storage mounter[513]: Bind mounting external storage to /swap
Jun 22 16:44:31 umbrel external storage mounter[513]: Bind mounting SD card root at /sd-card…
Jun 22 16:44:31 umbrel external storage mounter[513]: Checking Umbrel root is now on external storage…
Jul 15 18:42:54 umbrel external storage mounter[513]: Checking /var/lib/docker is now on external storage…
Jul 15 18:42:54 umbrel external storage mounter[513]: Checking /swap is now on external storage…
Jul 15 18:42:54 umbrel external storage mounter[513]: Setting up swapfile
Jul 15 18:42:54 umbrel external storage mounter[513]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
Jul 15 18:42:54 umbrel external storage mounter[513]: no label, UUID=b1694873-728b-49c4-b7a2-24dcf4f051e6
Jul 15 18:42:54 umbrel external storage mounter[513]: Checking SD Card root is bind mounted at /sd-root…
Jul 15 18:42:54 umbrel external storage mounter[513]: Starting external drive mount monitor…
Jul 15 18:42:54 umbrel external storage mounter[513]: Mount script completed successfully!
Jul 15 18:42:54 umbrel systemd[1]: Started External Storage Mounter.

External storage SD card update service logs

– Logs begin at Thu 2023-06-22 16:43:58 UTC, end at Sat 2023-07-15 18:47:21 UTC. –
Jul 15 18:43:00 umbrel systemd[1]: Starting External Storage SDcard Updater…
Jul 15 18:43:00 umbrel external storage updater[1185]: Checking if SD card Umbrel is newer than external storage…
Jul 15 18:43:00 umbrel external storage updater[1185]: No, SD version is not newer, exiting.
Jul 15 18:43:00 umbrel systemd[1]: Started External Storage SDcard Updater.

Karen logs

karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
Got signal:
karen is getting triggered!
Got signal:
karen is getting triggered!
./karen: line 75: /home/umbrel/umbrel/events/triggers/: Is a directory
./karen: line 75: /home/umbrel/umbrel/events/triggers/: Is a directory
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events

Docker containers

NAMES STATUS
nginx Up 4 minutes
manager Up 4 minutes
tor_server Up 4 minutes
tor_proxy Up 4 minutes
dashboard Up 4 minutes
auth Up 4 minutes

Umbrel logs

Attaching to manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:01 GMT] “GET /v1/system/get-update HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:16 GMT] “GET /v1/system/update-status HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/system/is-sd-card-failing HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/system/temperature HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/apps HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/system/info HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/apps?installed=1 HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/system/memory HTTP/1.0” 200 84 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:20 GMT] “GET /v1/system/storage HTTP/1.0” 200 92 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 15 Jul 2023 18:47:21 GMT] “GET /v1/system/get-update HTTP/1.0” 304 - “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36”
manager |
manager | umbrel-manager

Tor Proxy logs

Attaching to tor_proxy
tor_proxy | Jul 15 18:43:15.000 [notice] Bootstrapped 56% (loading_descriptors): Loading relay descriptors
tor_proxy | Jul 15 18:43:15.000 [notice] Bootstrapped 64% (loading_descriptors): Loading relay descriptors
tor_proxy | Jul 15 18:43:15.000 [notice] Bootstrapped 69% (loading_descriptors): Loading relay descriptors
tor_proxy | Jul 15 18:43:16.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
tor_proxy | Jul 15 18:43:16.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
tor_proxy | Jul 15 18:43:16.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
tor_proxy | Jul 15 18:43:16.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
tor_proxy | Jul 15 18:43:17.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
tor_proxy | Jul 15 18:43:17.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
tor_proxy | Jul 15 18:43:18.000 [notice] Bootstrapped 100% (done): Done
/home/umbrel/umbrel/scripts/debug: line 139: ./scripts/app: No such file or directory

==== Result ====

The debug script did not automatically detect any issues with your Umbrel.
Connection to umbrel.local closed.

[usernameisJim]

I’ve had some recent users also having issues on 2 year old Pi 4s, it is technically a board that MSRPs at $35 so at continuous usage for 2 years that is a pretty good lifespan, have your applications still been stuck on starting? If multiple MicroSD cards are not working we need to move on to further troubleshooting that include

Isolating the problem with the Pi 4: https://www.makeuseof.com/tag/raspberry-pi-wont-boot-fix/
more info: What Does the Green and Red Light Mean on Raspberry Pi? – RaspberryTips

I know it’s redundant but try to follow these steps when reflashing:

@usernameisJim

The most reliable fix for this issue if you are on the Raspberry Pi 4 platform is to reflash your MicroSD card. If you have another one you can swap in, please try another card as well

SSH and run this command to safely shutdown: sudo shutdown

Unplug power and reflash your MicroSD card, then when complete, plug in your power last and wait ~5 minutes

For reference, reflashing will not result in any data loss because all user data is stored on your external storage drive (SSD or HDD).

To reflash your MicroSD card, follow the steps on our website under “Install on a Rasberry Pi 4.”

Then we’d move on to the last resort options of formatting the drive and just doing a clean install

Then replacing the Pi 4 board if we keep getting failures as in that that Pi 4 troubleshooting article linked first above, to isolate the issue