Error: System service failed ("*** Disk space is too low!" and "No space left on device"

Hello,

First, I LOVE my Umbrels…but have an issue with one of the 3 Umbrel nodes I have. It won’t start up (since June 2022), just getting to it now. Seems to be a disk space issue, as I see plenty of “*** Disk space is too low!” and “No space left on device” messages. Storage device is the recommended 1TB SSD ( Crucial MX500 1TB 3D NAND SATA 2.5 Inch Internal SSD)

Not sure how to resolve it. At the time, I remember installing all the available apps on it (successfully), backing up the channel file (successfully) and then trying to upgrade to the latest OS version. It never successfully restarted and gave the “Error: System service failed” message with the orange Umbrel logo, which is where I am at now.

Is there any way of freeing up disk space on the drive via SSH, maybe deleting some of the apps? Would that help?

Thanks for any assistance.

Some System Info:

Umbrel version

0.4.17
Flashed OS version

v0.4.14
Raspberry Pi Model

Revision : d03114
Serial : 10000000841ecb2b
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

Dec 1 2021 15:01:54
Copyright © 2012 Broadcom
version 71bd3109023a0c8575585ba87cbb374d2eeb038f (clean) (release) (start)

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 7.8G 109M 7.3G 89M 381M 7.5G
Swap: 4.1G 0B 4.1G

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 29G 0 100% /
/dev/sda1 916G 916G 0 100% /home/umbrel/umbrel

umbrel@umbrel:~ $ tail -f ~/umbrel/bitcoin/debug.log
2022-06-03T01:30:11Z *** Disk space is too low!
2022-06-03T01:30:11Z Error: Disk space is too low!
2022-06-03T01:30:11Z FlushStateToDisk: write coins cache to disk (2255922 coins, 326482kB) completed (0.00s)
2022-06-03T01:30:11Z ForceFlushStateToDisk: failed to flush state (Disk space is too low!)
2022-06-03T01:30:12Z FlushStateToDisk: write coins cache to disk (2255922 coins, 326482kB) started
2022-06-03T01:30:12Z *** Disk space is too low!
2022-06-03T01:30:12Z Error: Disk space is too low!
2022-06-03T01:30:12Z FlushStateToDisk: write coins cache to disk (2255922 coins, 326482kB) completed (0.00s)
2022-06-03T01:30:12Z ForceFlushStateToDisk: failed to flush state (Disk space is too low!)
2022-06-03T01:30:31Z Shutdown: done

umbrel@umbrel:~ $ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 30361180 30344796 0 100% /
devtmpfs 3834564 0 3834564 0% /dev
tmpfs 4000260 0 4000260 0% /dev/shm
tmpfs 4000260 366844 3633416 10% /run
tmpfs 5120 4 5116 1% /run/lock
tmpfs 4000260 0 4000260 0% /sys/fs/cgroup
/dev/mmcblk0p1 258095 31056 227040 13% /boot
/dev/sda1 960379920 960363540 0 100% /mnt/data
tmpfs 800052 0 800052 0% /run/user/1000

One solution… get a bigger SSD. Then clone the old SSD to the new larger SSD. There is a guide here