Error: system service failed

Support and Troubleshooting
1

Hi guys,

Sorry to bother you with another startup question, but i’m receving the “Error: system service failed” error at boot as well.

My setup is a new raspberri pi 4, 8b, with a stock charger. large heatsink on top. 16gb memory and 1TB 970evoplus m2 samsung. This last ssd i purchased new today. A while ago i tried to set a node up with an older ssd i had but no luck either. I expected the ssd to be the issue, now with a new ssd same error.

I’ve been reading through the other posts but i havent been able to find anyting useful. I did however manage to obtain a debug log. Havent been able to figure out what http address to share so ill just post the whole thing:

I personally think that the ssd is still giving issues. But i would like an expert opinion… Any help would be greatly appreciated!

=====================
= Umbrel debug info =

Umbrel version

0.4.2

Flashed OS version

v0.4.2

Raspberry Pi Model

Revision : c03112
Serial : 10000000086b413f
Model : Raspberry Pi 4 Model B Rev 1.2

Firmware

Aug 31 2021 14:52:47
Copyright © 2012 Broadcom
version 67615e950e1e28b92dfae6303cf7a8b879a8908f (clean) (release) (start)

Temperature

temp=48.2’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 3.8G 136M 3.4G 8.0M 277M 3.6G
Swap: 0B 0B 0B

total: 3.6%
system: 3.6%
tor: 0%
lnd: 0%
electrs: 0%
bitcoin: 0%

Memory monitor logs

tail: cannot open ‘/home/umbrel/umbrel/scripts/…/logs/memory-monitor.log’ for reading: No such file or directory

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 15G 3.0G 11G 22% /
/dev/root 15G 3.0G 11G 22% /

Startup service logs

– Logs begin at Mon 2021-09-13 19:27:28 UTC, end at Mon 2021-09-13 19:48:50 UTC. –
Sep 13 19:27:42 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
Sep 13 19:27:42 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result ‘dependency’.

External storage service logs

– Logs begin at Mon 2021-09-13 19:27:28 UTC, end at Mon 2021-09-13 19:48:50 UTC. –
Sep 13 19:27:35 umbrel systemd[1]: Starting External Storage Mounter…
Sep 13 19:27:36 umbrel external storage mounter[501]: Running external storage mount script…
Sep 13 19:27:39 umbrel external storage mounter[501]: Found device “ASMedia USB 3.0 Device”
Sep 13 19:27:39 umbrel external storage mounter[501]: Blacklisting USB device IDs against UAS driver…
Sep 13 19:27:40 umbrel external storage mounter[501]: Rebinding USB drivers…
Sep 13 19:27:40 umbrel external storage mounter[501]: Checking USB devices are back…
Sep 13 19:27:40 umbrel external storage mounter[501]: Waiting for USB devices…
Sep 13 19:27:41 umbrel external storage mounter[501]: Waiting for USB devices…
Sep 13 19:27:42 umbrel external storage mounter[501]: Checking if the device is ext4…
Sep 13 19:27:42 umbrel external storage mounter[501]: No, it’s not ext4
Sep 13 19:27:42 umbrel external storage mounter[501]: Formatting device…
Sep 13 19:27:42 umbrel external storage mounter[501]: wipefs: error: /dev/sda: probing initialization failed: No medium found
Sep 13 19:27:42 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=1/FAILURE
Sep 13 19:27:42 umbrel systemd[1]: umbrel-external-storage.service: Failed with result ‘exit-code’.
Sep 13 19:27:42 umbrel systemd[1]: Failed to start External Storage Mounter.

External storage SD card update service logs

– Logs begin at Mon 2021-09-13 19:27:28 UTC, end at Mon 2021-09-13 19:48:50 UTC. –
Sep 13 19:27:42 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
Sep 13 19:27:42 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result ‘dependency’.

Karen logs

tail: cannot open ‘logs/karen.log’ for reading: No such file or directory

Docker containers

NAMES STATUS

Bitcoin Core logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

LND logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

Tor logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

==== Result ====

This script could not automatically detect an issue with your Umbrel.
Please share the following links and paste it in the Umbrel Telegram group (https://t.me/getumbrel) so we can help you with your problem.
tail: cannot open ‘/home/umbrel/umbrel/scripts/…/logs/memory-monitor.log’ for reading: No such file or directory
tail: cannot open ‘logs/karen.log’ for reading: No such file or directory
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements
ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

2

Some discrepancies here:
You said that you have a 8GB RAM memory RPi, but shows only 4GB RAM
“Mem: 3.8G 136M 3.4G 8.0M 277M 3.6G
Swap: 0B 0B 0B”

Then you said:
“16gb memory and 1TB 970evoplus m2 samsung”
And in logs shows that you are trying to use a 16GB memory USB stick?
The log is very clear about what is happening:
“Failed to start External Storage Mounter.”

3

sorry for the confusion. Trying to share as much info as possible to get some results. Apperantly i dont know which PI ive purchased.

that little memory card underneath the PI i used to flash the software on is the 16gb scandisk. The only storage device connected to the usb ports is a 1T samsung ssd…

But thanks, ill investigate some more!

4

Here you have a troubleshooting guide for Umbrel

5

A few days ago, my Umbrel was turning off randomly and restarting (and sometimes refusing to power on). Then I got the “error: system service failed” screen. I reflashed the SD card to 4.7, and still got the “error: system service failed” screen. I reflashed to 4.8, and am still stuck at the same screen. When I ssh in, it doesn’t recognize my custom password anymore, only the default one. Sparrow can’t connect to Umbrel anymore. I’ve tried the recommendations I could find, to no avail.

I am a noob using a Raspberry Pi 4 with a 32gb SD card. My Umbrel is connected through an ethernet cable to the router. I am using an SSD drive. My hardware was assembled by CryptoCloaks.

Thank you; here are the debug logs:

=====================

= Umbrel debug info =

Umbrel version

0.4.6

Flashed OS version

v0.4.8

Raspberry Pi Model

Revision : d03114
Serial : 10000000682ffc21
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

Oct 29 2021 10:47:33
Copyright © 2012 Broadcom
version b8a114e5a9877e91ca8f26d1a5ce904b2ad3cf13 (clean) (release) (start)

Temperature

temp=40.9’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 7.8G 398M 7.0G 8.0M 428M 7.3G
Swap: 4.1G 0B 4.1G

total: 5.1%
electrs: 4.1%
system: 1%
vaultwarden: 0%
tor: 0%
thunderhub: 0%
sphinx-relay: 0%
specter-desktop: 0%
ride-the-lightning: 0%
mempool: 0%
lnd: 0%
lightning-terminal: 0%
btc-rpc-explorer: 0%
btcpay-server: 0%
bluewallet: 0%
bitcoin: 0%

Memory monitor logs

2021-11-09 08:49:16 Memory monitor running!
2021-11-09 13:54:56 Memory monitor running!
2021-11-09 14:08:56 Memory monitor running!
2021-11-09 14:29:34 Memory monitor running!
2021-11-09 14:41:57 Memory monitor running!
2021-11-09 14:53:43 Memory monitor running!
2021-11-09 15:24:27 Memory monitor running!
2021-11-09 15:33:28 Memory monitor running!
2021-11-09 15:33:30 Memory monitor running!
2021-11-11 02:38:47 Memory monitor running!

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 3.0G 25G 11% /
/dev/sda1 916G 481G 389G 56% /home/umbrel/umbrel

Startup service logs

– Logs begin at Mon 2021-11-15 14:50:50 UTC, end at Mon 2021-11-15 17:21:56 UTC. –
Nov 15 14:51:22 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
Nov 15 14:51:22 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result ‘dependency’.

External storage service logs

– Logs begin at Mon 2021-11-15 14:50:50 UTC, end at Mon 2021-11-15 17:21:56 UTC. –
Nov 15 14:50:57 umbrel systemd[1]: Starting External Storage Mounter…
Nov 15 14:50:57 umbrel external storage mounter[516]: Running external storage mount script…
Nov 15 14:51:01 umbrel external storage mounter[516]: Found device “SanDisk SSD PLUS 1000GB”
Nov 15 14:51:01 umbrel external storage mounter[516]: Blacklisting USB device IDs against UAS driver…
Nov 15 14:51:01 umbrel external storage mounter[516]: Rebinding USB drivers…
Nov 15 14:51:02 umbrel external storage mounter[516]: Checking USB devices are back…
Nov 15 14:51:02 umbrel external storage mounter[516]: Waiting for USB devices…
Nov 15 14:51:03 umbrel external storage mounter[516]: Waiting for USB devices…
Nov 15 14:51:04 umbrel external storage mounter[516]: Checking if the device is ext4…
Nov 15 14:51:04 umbrel external storage mounter[516]: Yes, it is ext4
Nov 15 14:51:04 umbrel external storage mounter[516]: Checking if device contains an Umbrel install…
Nov 15 14:51:04 umbrel external storage mounter[516]: Yes, it contains an Umbrel install
Nov 15 14:51:04 umbrel external storage mounter[516]: Bind mounting external storage over local Umbrel installation…
Nov 15 14:51:04 umbrel external storage mounter[516]: Bind mounting external storage over local Docker data dir…
Nov 15 14:51:04 umbrel external storage mounter[516]: Bind mounting external storage to /swap
Nov 15 14:51:04 umbrel external storage mounter[516]: Bind mounting SD card root at /sd-card…
Nov 15 14:51:04 umbrel external storage mounter[516]: Checking Umbrel root is now on external storage…
Nov 15 14:51:07 umbrel external storage mounter[516]: Checking /var/lib/docker is now on external storage…
Nov 15 14:51:07 umbrel external storage mounter[516]: Checking /swap is now on external storage…
Nov 15 14:51:07 umbrel external storage mounter[516]: Setting up swapfile
Nov 15 14:51:08 umbrel external storage mounter[516]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
Nov 15 14:51:08 umbrel external storage mounter[516]: no label, UUID=2e50af29-d76d-442e-aabc-c4c771f3e7d2
Nov 15 14:51:08 umbrel external storage mounter[516]: Checking SD Card root is bind mounted at /sd-root…
Nov 15 14:51:08 umbrel external storage mounter[516]: Starting external drive mount monitor…
Nov 15 14:51:08 umbrel external storage mounter[516]: Mount script completed successfully!
Nov 15 14:51:08 umbrel systemd[1]: Started External Storage Mounter.

External storage SD card update service logs

– Logs begin at Mon 2021-11-15 14:50:50 UTC, end at Mon 2021-11-15 17:21:56 UTC. –
Nov 15 14:51:21 umbrel systemd[1]: Starting External Storage SDcard Updater…
Nov 15 14:51:21 umbrel external storage updater[1298]: Checking if SD card Umbrel is newer than external storage…
Nov 15 14:51:22 umbrel external storage updater[1298]: Yes, SD version is newer.
Nov 15 14:51:22 umbrel external storage updater[1298]: Checking if the external storage version “0.4.6” satisfies update requirement “>=0.2.1”…
Nov 15 14:51:22 umbrel external storage updater[1298]: Yes, it does, attempting an automatic update…
Nov 15 14:51:22 umbrel external storage updater[1298]: =======================================
Nov 15 14:51:22 umbrel external storage updater[1298]: =============== UPDATE ================
Nov 15 14:51:22 umbrel external storage updater[1298]: =======================================
Nov 15 14:51:22 umbrel external storage updater[1298]: ========== Stage: Download ============
Nov 15 14:51:22 umbrel external storage updater[1298]: =======================================
Nov 15 14:51:22 umbrel external storage updater[1298]: An update is already in progress. Exiting now.
Nov 15 14:51:22 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Nov 15 14:51:22 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Failed with result ‘exit-code’.
Nov 15 14:51:22 umbrel systemd[1]: Failed to start External Storage SDcard Updater.

Karen logs

Stopping middleware …
Stopping bitcoin …
Stopping nginx …
Stopping lnd …
Stopping manager …
Stopping electrs …
Stopping umbrel_app_3_tor_1 …
Stopping tor …
Stopping umbrel_app_tor_1 …
Stopping dashboard …
Stopping umbrel_app_2_tor_1 …
Stopping umbrel_app_tor_1 … done
Stopping middleware … done
Stopping umbrel_app_3_tor_1 … done
Stopping umbrel_app_2_tor_1 … done
Stopping electrs … done
Stopping lnd … done
Stopping bitcoin … done
Stopping nginx … done
Stopping manager … done
Stopping dashboard … done
Stopping tor … done
Removing middleware …
Removing neutrino-switcher …
Removing bitcoin …
Removing nginx …
Removing lnd …
Removing manager …
Removing electrs …
Removing umbrel_app_3_tor_1 …
Removing tor …
Removing umbrel_app_tor_1 …
Removing dashboard …
Removing umbrel_app_2_tor_1 …
Removing neutrino-switcher … done
Removing dashboard … done
Removing lnd … done
Removing umbrel_app_3_tor_1 … done
Removing bitcoin … done
Removing electrs … done
Removing middleware … done
Removing umbrel_app_2_tor_1 … done
Removing nginx … done
Removing tor … done
Removing umbrel_app_tor_1 … done
Removing manager … done
Removing network umbrel_main_network
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events

Docker containers

NAMES STATUS
electrs Restarting (1) 13 seconds ago

Umbrel logs

Attaching to

Bitcoin Core logs

Attaching to

LND logs

Attaching to

electrs logs

Attaching to electrs
electrs | [2021-11-15T17:19:33.519Z INFO electrs::chain] loading 708941 headers, tip=00000000000000000005e1c118d0b8a3e5917c693421d32442be8c63f1a81adc
electrs | [2021-11-15T17:19:37.444Z INFO electrs::chain] chain updated: tip=00000000000000000005e1c118d0b8a3e5917c693421d32442be8c63f1a81adc, height=708941
electrs | [2021-11-15T17:19:37.476Z INFO electrs::db] closing DB at /data/db/bitcoin
electrs | Error: electrs failed
electrs |
electrs | Caused by:
electrs | 0: failed to open bitcoind cookie file: /data/.bitcoin/.cookie
electrs | 1: No such file or directory (os error 2)
electrs | Starting electrs 0.9.1 on aarch64 linux with Config { network: Bitcoin, db_path: “/data/db/bitcoin”, daemon_dir: “/data/.bitcoin”, daemon_auth: CookieFile("/data/.bitcoin/.cookie"), daemon_rpc_addr: V4(10.21.21.8:8332), daemon_p2p_addr: V4(10.21.21.8:8333), electrum_rpc_addr: V4(0.0.0.0:50001), monitoring_addr: V4(127.0.0.1:4224), wait_duration: 10s, jsonrpc_timeout: 15s, index_batch_size: 10, index_lookup_limit: Some(200), reindex_last_blocks: 0, auto_reindex: true, ignore_mempool: false, sync_once: false, disable_electrum_rpc: false, server_banner: “Umbrel v0.4.6”, args: [] }
electrs | [2021-11-15T17:20:38.410Z INFO electrs::metrics::metrics_impl] serving Prometheus metrics on 127.0.0.1:4224
electrs | [2021-11-15T17:20:38.459Z INFO electrs::db] “/data/db/bitcoin”: 134 SST files, 32.606096919 GB, 4.035228625 Grows
electrs | [2021-11-15T17:20:42.654Z INFO electrs::chain] loading 708941 headers, tip=00000000000000000005e1c118d0b8a3e5917c693421d32442be8c63f1a81adc
electrs | [2021-11-15T17:20:46.589Z INFO electrs::chain] chain updated: tip=00000000000000000005e1c118d0b8a3e5917c693421d32442be8c63f1a81adc, height=708941
electrs | [2021-11-15T17:20:46.620Z INFO electrs::db] closing DB at /data/db/bitcoin
electrs | Error: electrs failed
electrs |
electrs | Caused by:
electrs | 0: failed to open bitcoind cookie file: /data/.bitcoin/.cookie
electrs | 1: No such file or directory (os error 2)
electrs | Starting electrs 0.9.1 on aarch64 linux with Config { network: Bitcoin, db_path: “/data/db/bitcoin”, daemon_dir: “/data/.bitcoin”, daemon_auth: CookieFile("/data/.bitcoin/.cookie"), daemon_rpc_addr: V4(10.21.21.8:8332), daemon_p2p_addr: V4(10.21.21.8:8333), electrum_rpc_addr: V4(0.0.0.0:50001), monitoring_addr: V4(127.0.0.1:4224), wait_duration: 10s, jsonrpc_timeout: 15s, index_batch_size: 10, index_lookup_limit: Some(200), reindex_last_blocks: 0, auto_reindex: true, ignore_mempool: false, sync_once: false, disable_electrum_rpc: false, server_banner: “Umbrel v0.4.6”, args: [] }
electrs | [2021-11-15T17:21:47.548Z INFO electrs::metrics::metrics_impl] serving Prometheus metrics on 127.0.0.1:4224
electrs | [2021-11-15T17:21:47.598Z INFO electrs::db] “/data/db/bitcoin”: 135 SST files, 32.606097723 GB, 4.035228626 Grows
electrs | [2021-11-15T17:21:51.846Z INFO electrs::chain] loading 708941 headers, tip=00000000000000000005e1c118d0b8a3e5917c693421d32442be8c63f1a81adc
electrs | [2021-11-15T17:21:55.995Z INFO electrs::chain] chain updated: tip=00000000000000000005e1c118d0b8a3e5917c693421d32442be8c63f1a81adc, height=708941
electrs | [2021-11-15T17:21:56.041Z INFO electrs::db] closing DB at /data/db/bitcoin
electrs | Error: electrs failed
electrs |
electrs | Caused by:
electrs | 0: failed to open bitcoind cookie file: /data/.bitcoin/.cookie
electrs | 1: No such file or directory (os error 2)

Tor logs

Attaching to tor, umbrel_app_tor_1, umbrel_app_2_tor_1
tor | Nov 13 00:14:07.000 [notice] Bootstrapped 66% (loading_descriptors): Loading relay descriptors
tor | Nov 13 00:14:07.000 [notice] Bootstrapped 72% (loading_descriptors): Loading relay descriptors
tor | Nov 13 00:14:07.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
tor | Nov 13 00:14:07.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
tor | Nov 13 00:14:07.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
tor | Nov 13 00:14:08.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
tor | Nov 13 00:14:09.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
tor | Nov 13 00:14:09.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
tor | Nov 13 00:14:10.000 [notice] Bootstrapped 100% (done): Done
tor | Nov 13 00:15:51.000 [notice] Catching signal TERM, exiting cleanly.
app_tor_1 | Nov 13 00:14:07.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
app_tor_1 | Nov 13 00:14:07.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
app_tor_1 | Nov 13 00:14:08.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
app_tor_1 | Nov 13 00:14:08.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
app_tor_1 | Nov 13 00:14:08.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
app_tor_1 | Nov 13 00:14:09.000 [notice] Bootstrapped 100% (done): Done
app_tor_1 | Nov 13 00:14:24.000 [notice] Your network connection speed appears to have changed. Resetting timeout to 60s after 18 timeouts and 100 buildtimes.
app_tor_1 | Nov 13 00:14:25.000 [notice] Guard dragonhoard ($96B70F86B45623B4EC71E8E953DCD61A05D5BAAC) is failing more circuits than usual. Most likely this means the Tor network is overloaded. Success counts are 86/151. Use counts are 57/57. 86 circuits completed, 0 were unusable, 0 collapsed, and 0 timed out. For reference, your timeout cutoff is 60 seconds.
app_tor_1 | Nov 13 00:14:25.000 [warn] Guard dragonhoard ($96B70F86B45623B4EC71E8E953DCD61A05D5BAAC) is failing a very large amount of circuits. Most likely this means the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Success counts are 86/173. Use counts are 57/57. 86 circuits completed, 0 were unusable, 0 collapsed, and 0 timed out. For reference, your timeout cutoff is 60 seconds.
app_tor_1 | Nov 13 00:15:51.000 [notice] Catching signal TERM, exiting cleanly.
app_2_tor_1 | Nov 13 00:14:08.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
app_2_tor_1 | Nov 13 00:14:08.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
app_2_tor_1 | Nov 13 00:14:09.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
app_2_tor_1 | Nov 13 00:14:09.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
app_2_tor_1 | Nov 13 00:14:09.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
app_2_tor_1 | Nov 13 00:14:10.000 [notice] Bootstrapped 100% (done): Done
app_2_tor_1 | Nov 13 00:14:26.000 [notice] Guard torhammer ($DA84D41783BBB6058CB9DF8C90697E8D5EA647C3) is failing more circuits than usual. Most likely this means the Tor network is overloaded. Success counts are 95/151. Use counts are 58/58. 95 circuits completed, 0 were unusable, 0 collapsed, and 0 timed out. For reference, your timeout cutoff is 60 seconds.
app_2_tor_1 | Nov 13 00:14:27.000 [warn] Guard torhammer ($DA84D41783BBB6058CB9DF8C90697E8D5EA647C3) is failing a very large amount of circuits. Most likely this means the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Success counts are 96/193. Use counts are 59/59. 96 circuits completed, 0 were unusable, 0 collapsed, and 0 timed out. For reference, your timeout cutoff is 60 seconds.
app_2_tor_1 | Nov 13 00:14:27.000 [notice] Your network connection speed appears to have changed. Resetting timeout to 60s after 18 timeouts and 122 buildtimes.
app_2_tor_1 | Nov 13 00:15:51.000 [notice] Catching signal TERM, exiting cleanly.

App logs

bluewallet

Attaching to

btc-rpc-explorer

Attaching to

btcpay-server

Attaching to

lightning-terminal

Attaching to

mempool

Attaching to

ride-the-lightning

Attaching to

specter-desktop

Attaching to

sphinx-relay

Attaching to

thunderhub

Attaching to

vaultwarden

Attaching to

==== Result ====

The debug script did not automatically detect any issues with your Umbrel.

6

To try, because definitely is a hardware failure

  • change power supply
  • change SSD case
  • change USB cable
  • change mSD card and re-flash it with latest UmbrelOS
7

Ok thanks. I had already got a new approved Pi power cable and changed and re-flashed the SD card with the latest OS. I’ll order another USB cable, and if that doesn’t work, I’ll try to change the SSD case.

8

I just noticed that my SSD is glued to the CryptoCloaks shell without a case. The SATA to USB cable plugs directly into it. I ordered another cable to see if it helps. I also ordered yet another mSD card just in case. I hope one of those work, because I don’t know what else to try. When I plugged my SSD into my computer, it recognized it fine, so the cable seems to be working. Thanks again.

9

I was able to get in and start syncing with this command:

ssh -t umbrel@umbrel.local "rm -f ~/umbrel/statuses/update-in-progress && sudo reboot"

Thanks

2 Likes
11

It worked also for me! Many thanks buddy

12

@DarthCoin FYI, when the systemd service umbrel external storage mounter doesn’t fail, but the umbrel external storage updater fails, the issue is likely with the update process. This is confirmed in the logs by:

The issue here is that a update failed but the lock file didn’t get removed, so the script detects that an update is in progress when it is not the case. A simple fix can be, as @btpl21found, removing the update lock file and reboot the device:

ssh -t umbrel@umbrel.local "rm -f ~/umbrel/statuses/update-in-progress && sudo reboot"
2 Likes
13

Thanks for pointing in the right direction. Noted. Will add this in troubleshooting manual

2 Likes
14

Can you please help me?

I have the same issue " Error: System service failed "

I have tried all the things in multi forms that you have done to get people back up and running but none works for me.

I am not the best with SSH so I will need some help using terminal on a Mac.

15

Also, sorry new to the form

16

=====================
= Umbrel debug info =

Umbrel version

0.4.8

Flashed OS version

v0.4.8

Raspberry Pi Model

Revision : c03114
Serial : 10000000f6ce892c
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

Oct 29 2021 10:47:33
Copyright © 2012 Broadcom
version b8a114e5a9877e91ca8f26d1a5ce904b2ad3cf13 (clean) (release) (start)

Temperature

temp=39.9’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 3.8G 124M 3.5G 8.0M 209M 3.6G
Swap: 0B 0B 0B

total: 3.3%
system: 3.3%
tor: 0%
lnd: 0%
electrs: 0%
bitcoin: 0%

Memory monitor logs

tail: cannot open ‘/home/umbrel/umbrel/scripts/…/logs/memory-monitor.log’ for reading: No such file or directory

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 30G 3.1G 25G 11% /
/dev/root 30G 3.1G 25G 11% /

Startup service logs

– Logs begin at Fri 2021-11-26 17:36:03 UTC, end at Fri 2021-11-26 17:41:12 UTC. –
Nov 26 17:36:11 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
Nov 26 17:36:11 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result ‘dependency’.

External storage service logs

– Logs begin at Fri 2021-11-26 17:36:03 UTC, end at Fri 2021-11-26 17:41:12 UTC. –
Nov 26 17:36:08 umbrel systemd[1]: Starting External Storage Mounter…
Nov 26 17:36:08 umbrel external storage mounter[479]: Running external storage mount script…
Nov 26 17:36:08 umbrel external storage mounter[479]: Found device “CT1000MX 500SSD1”
Nov 26 17:36:08 umbrel external storage mounter[479]: Blacklisting USB device IDs against UAS driver…
Nov 26 17:36:08 umbrel external storage mounter[479]: Rebinding USB drivers…
Nov 26 17:36:08 umbrel external storage mounter[479]: Checking USB devices are back…
Nov 26 17:36:08 umbrel external storage mounter[479]: Waiting for USB devices…
Nov 26 17:36:09 umbrel external storage mounter[479]: Waiting for USB devices…
Nov 26 17:36:10 umbrel external storage mounter[479]: Checking if the device is ext4…
Nov 26 17:36:11 umbrel external storage mounter[479]: Yes, it is ext4
Nov 26 17:36:11 umbrel external storage mounter[479]: mount: /mnt/data: can’t read superblock on /dev/sda1.
Nov 26 17:36:11 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=32/n/a
Nov 26 17:36:11 umbrel systemd[1]: umbrel-external-storage.service: Failed with result ‘exit-code’.
Nov 26 17:36:11 umbrel systemd[1]: Failed to start External Storage Mounter.

External storage SD card update service logs

– Logs begin at Fri 2021-11-26 17:36:03 UTC, end at Fri 2021-11-26 17:41:12 UTC. –
Nov 26 17:36:11 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
Nov 26 17:36:11 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result ‘dependency’.

Karen logs

tail: cannot open ‘logs/karen.log’ for reading: No such file or directory

Docker containers

NAMES STATUS

Umbrel logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

Bitcoin Core logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

LND logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

electrs logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

Tor logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor.ports contains an invalid type, it should be a number, or an object
services.bitcoin.ports contains an invalid type, it should be a number, or an object
services.electrs.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.lnd.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object
services.lnd.ports value [‘9735:9735’, ‘:’, ‘:’] has non-unique elements

==== Result ====

The debug script did not automatically detect any issues with your Umbrel.

17

It says right there you issue
You have problems with your external drive: corruption of data, bad connection, low voltage for the drive etc.

18

Thanks you. Whats the first steps to take? Do I need to reformat? Power supply tests with correct voltage and I change the USB 3 to USB c to a new cable that I tested and works. I tried looking for a form about replacing hard drive and couldn’t find any

19

I have the exactly same problem.
an accidental power shut down, no come back. (error:system service failed)
reflashed SD card. still not working.
can connect to SSH, not with my password but with the default one.
tried the command but, “remote side unexpectedly closed network connection”

It seems the node can’t read from the external drive.
it is so much frustrating…anyone help?

20

yes a re-format could fix the issue. Remember it should be in ext4 not ntfs. And remember to have the channels.backup file (in case you had LN channels open)

21

What command do you use to get a log ?