Error: System service failed (not ext4)

At first I managed to connect to umbrel.local from the web UI. I created an account and installed the Bitcoin Node app. The blockchain syncing stopped at 4% and I shut down my raspberry. After starting it again, every attempt to connect to umbrel.local have failed. I can still ssh to umbrel though.

I am a noob with basically 0 technical background, but according to the debug log it seems that there is an issue with the format of my SSD (it says it is not ext4). I wiped out the disk using AOMEI partition assistant but this did not resolve the problem.

My HW setup:

Device: Raspberry Pi 4 Model B 8 GB RAM
Case: Argon One M.2: Aluminum Case
Storage drive: Samsung 980 PRO 1TB
Micro SD: SanDisk micro SDHC 32 GB Extreme Pro A1 UHS-I (V30)
Power supply: Raspberry Pi 4 USB-C official power supply

=====================
= Umbrel debug info =
=====================

Umbrel version
--------------
0.5.4

Flashed OS version
-----------------
v0.5.4

Raspberry Pi Model
------------------
Revision        : d03114
Serial          : 10000000fb30698e
Model           : Raspberry Pi 4 Model B Rev 1.4

Firmware
--------
May  9 2023 12:16:34
Copyright (c) 2012 Broadcom
version 30aa0d70ab280427ba04ebc718c81d4350b9d394 (clean) (release) (start)

Temperature
-----------
temp=36.0'C

Throttling
----------
throttled=0x0

Memory usage
------------
              total        used        free      shared  buff/cache   available
Mem:           7.8G        208M        7.0G        8.0M        633M        7.5G
Swap:            0B          0B          0B

total: 2.7%
system: 2.7%
bitcoin: 0%

Memory monitor logs
-------------------
2023-09-04 12:11:16 Memory monitor running!

Filesystem information
----------------------
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        30G   28G  262M 100% /
/dev/root        30G   28G  262M 100% /

Startup service logs
--------------------
-- Logs begin at Wed 2023-09-06 09:51:30 UTC, end at Thu 2023-10-05 06:48:17 UTC. --
Sep 06 09:51:38 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
Sep 06 09:51:38 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result 'dependency'.

External storage service logs
-----------------------------
-- Logs begin at Wed 2023-09-06 09:51:30 UTC, end at Thu 2023-10-05 06:48:17 UTC. --
Sep 06 09:51:35 umbrel systemd[1]: Starting External Storage Mounter...
Sep 06 09:51:35 umbrel external storage mounter[486]: Running external storage mount script...
Sep 06 09:51:35 umbrel external storage mounter[486]: Found device "Argon Forty"
Sep 06 09:51:35 umbrel external storage mounter[486]: Blacklisting USB device IDs against UAS driver...
Sep 06 09:51:35 umbrel external storage mounter[486]: Rebinding USB drivers...
Sep 06 09:51:35 umbrel external storage mounter[486]: Checking USB devices are back...
Sep 06 09:51:35 umbrel external storage mounter[486]: Waiting for USB devices...
Sep 06 09:51:36 umbrel external storage mounter[486]: Waiting for USB devices...
Sep 06 09:51:37 umbrel external storage mounter[486]: Waiting for USB devices...
Sep 06 09:51:38 umbrel external storage mounter[486]: Checking if the device is ext4...
Sep 06 09:51:38 umbrel external storage mounter[486]: No, it's not ext4
Sep 06 09:51:38 umbrel external storage mounter[486]: Formatting device...
Sep 06 09:51:38 umbrel external storage mounter[486]: wipefs: error: /dev/sda: probing initialization failed: No medium found
Sep 06 09:51:38 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=1/FAILURE
Sep 06 09:51:38 umbrel systemd[1]: umbrel-external-storage.service: Failed with result 'exit-code'.
Sep 06 09:51:38 umbrel systemd[1]: Failed to start External Storage Mounter.

External storage SD card update service logs
--------------------------------------------
-- Logs begin at Wed 2023-09-06 09:51:30 UTC, end at Thu 2023-10-05 06:48:17 UTC. --
Sep 06 09:51:38 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
Sep 06 09:51:38 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result 'dependency'.

Karen logs
----------

Pulling server      ... extracting (91.7%)
Pulling server      ... extracting (95.7%)
Pulling server      ... extracting (99.8%)
Pulling server      ... extracting (100.0%)
Pulling server      ... pull complete
Pulling server      ... extracting (100.0%)
Pulling server      ... extracting (100.0%)
Pulling server      ... pull complete
Pulling server      ... digest: sha256:edc0567efeadf30128...
Pulling server      ... status: downloaded newer image fo...
Pulling server      ... done
Starting app bitcoin...
Executing hook: /home/umbrel/umbrel/app-data/bitcoin/hooks/pre-start
Creating bitcoin_bitcoind_1 ...
Creating bitcoin_bitcoind_1 ... done
Creating bitcoin_tor_1 ...
Creating bitcoin_tor_1 ... done
App: bitcoin - Generating Tor Hidden Service...
App: bitcoin - Hidden service file created successfully!
bitcoin_tor_1 is up-to-date
bitcoin_bitcoind_1 is up-to-date
Creating bitcoin_i2pd_daemon_1 ...
Creating bitcoin_app_proxy_1   ...
Creating bitcoin_server_1      ...
Creating bitcoin_i2pd_daemon_1 ... done
Creating bitcoin_server_1      ... done
Creating bitcoin_app_proxy_1   ... done
Saving app bitcoin in DB...
Successfully installed app bitcoin
Got signal: backup
karen is getting triggered!
Deriving keys...
Creating backup...
Adding random padding...
1+0 records in
1+0 records out
9309 bytes (9.3 kB, 9.1 KiB) copied, 0.000385572 s, 24.1 MB/s
Creating encrypted tarball...
backup/
backup/.padding
gpg: directory '/root/.gnupg' created
gpg: keybox '/root/.gnupg/pubring.kbx' created
Uploading backup...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  9998  100   146  100  9852     37   2545  0:00:03  0:00:03 --:--:--  2582
{"message":"Successfully uploaded backup 1693850913125.tar.gz.pgp for backup ID c3b5d856934f74e7dcae799f287deb99db5304be23a96bf250427a7016037b10"}
=============================
====== Backup success =======
=============================

Docker containers
-----------------
NAMES     STATUS

Umbrel logs
-----------

Attaching to

Tor Proxy logs
--------

Attaching to

App logs
--------

bitcoin

Attaching to
================
==== Result ====
================
==== END =====

Hi @peterko

Unfortunately this is likely an issue related to the NVME drive. It is a common issue in the self-hosting world that the Raspberry Pi 4 and it’s power adapter do not provide enough power to allow for stable NVME use.
Do you have a spare SSD/HDD drive you could try?

Thank you for the answer.

I do not have spare SSD at hand right now.

  1. Is replacing it the only option I can try?
  2. Why did this problem occurred just lately? (at first I did not get the error warning)
  3. Is there tested and recommended SSD product I should use?
  4. In case there is an issue with power supply stability, could some UPS device help?

Sure thing- this is what we recommend on umbrel.com:
Drive & Case

The Raspberry Pi is only able to power the NVME drive intermittently, which is why it likely worked for a short while.
As per your 4th question, you can also try a seperate powered USB hub, however I haven’t tried this myself, so I can’t attest to it’s relability.