1TB SSD is full?

Hey all, has anyone had any issues with SSD space after upgrading to Umbrel 0.4.1?

I have Umbrel running on Pi, 1TB hard drive, but Umbrel says the system is using 933GB, so I get “hard disk full” errors. Does anyone have any advice on how to remedy? Thanks!

Some log file info:

= Umbrel debug info =

Umbrel version

0.4.1

Flashed OS version

v0.3.12

Raspberry Pi Model

Revision : d03114
Serial : 10000000550f4c51
Model : Raspberry Pi 4 Model B Rev 1.4

Firmware

May 27 2021 14:02:15
Copyright © 2012 Broadcom
version 7d9a298cda813f747b51fe17e1e417e7bf5ca94d (clean) (release) (start)

Temperature

temp=44.3’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 7.8G 1.0G 5.6G 9.0M 1.1G 6.7G
Swap: 4.1G 0B 4.1G

total: 13.2%
bitcoin: 4.4%
lnd: 3%
thunderhub: 2.3%
mempool: 2%
system: 0.7%
tor: 0.4%
lightning-terminal: 0.4%
electrs: 0%

Memory monitor logs

Memory monitor is already running
2021-07-07 13:29:54 Memory monitor running!
2021-07-12 20:09:13 Memory monitor running!
2021-07-22 13:06:33 Memory monitor running!
2021-07-28 21:12:04 Memory monitor running!
2021-07-29 22:34:12 Memory monitor running!
2021-07-31 23:26:58 Memory monitor running!
3179 ? S 1:05 bash ./scripts/memory-monitor
Memory monitor is already running
2021-08-22 07:51:13 Memory monitor running!

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 2.9G 25G 11% /
/dev/sda1 916G 870G 43M 100% /home/umbrel/umbrel

Startup service logs

Aug 22 07:51:24 umbrel umbrel startup[1034]: Creating nginx … done
Aug 22 07:51:24 umbrel umbrel startup[1034]: Creating bitcoin …
Aug 22 07:51:38 umbrel umbrel startup[1034]: Creating lnd … done
Aug 22 07:51:40 umbrel umbrel startup[1034]: Creating bitcoin … done
Aug 22 07:51:40 umbrel umbrel startup[1034]: Creating middleware …
Aug 22 07:51:40 umbrel umbrel startup[1034]: Creating neutrino-switcher …
Aug 22 07:51:43 umbrel umbrel startup[1034]: Creating middleware … done
Aug 22 07:51:43 umbrel umbrel startup[1034]: Creating neutrino-switcher … done
Aug 22 07:51:43 umbrel umbrel startup[1034]: Removing status server iptables entry…
Aug 22 07:51:43 umbrel umbrel startup[1034]: Removed existing iptables entry.
Aug 22 07:51:43 umbrel umbrel startup[1034]: Starting installed apps…
Aug 22 07:51:43 umbrel umbrel startup[1034]: Starting app mempool…
Aug 22 07:51:43 umbrel umbrel startup[1034]: Starting app thunderhub…
Aug 22 07:51:43 umbrel umbrel startup[1034]: Starting app lightning-terminal…
Aug 22 07:51:48 umbrel umbrel startup[1034]: Creating mempool_web_1 …
Aug 22 07:51:48 umbrel umbrel startup[1034]: Creating mempool_api_1 …
Aug 22 07:51:48 umbrel umbrel startup[1034]: Creating mempool_mariadb_1 …
Aug 22 07:51:48 umbrel umbrel startup[1034]: Creating lightning-terminal_web_1 …
Aug 22 07:51:49 umbrel umbrel startup[1034]: Creating thunderhub_web_1 …
Aug 22 07:51:54 umbrel umbrel startup[1034]: Creating mempool_api_1 … done
Aug 22 07:51:55 umbrel umbrel startup[1034]: Creating mempool_web_1 … done
Aug 22 07:51:55 umbrel umbrel startup[1034]: Creating thunderhub_web_1 … done
Aug 22 07:51:55 umbrel umbrel startup[1034]: Creating mempool_mariadb_1 … done
Aug 22 07:51:55 umbrel umbrel startup[1034]: Creating lightning-terminal_web_1 … done
Aug 22 07:51:56 umbrel umbrel startup[1034]: Umbrel is now accessible at
Aug 22 07:51:56 umbrel umbrel startup[1034]: http://umbrel.local
Aug 22 07:51:56 umbrel umbrel startup[1034]: http://192.168.0.101
Aug 22 07:51:56 umbrel systemd[1]: Started Umbrel Startup Service.
Aug 22 07:53:04 umbrel passwd[4950]: pam_unix(passwd:chauthtok): password changed for umbrel

External storage service logs

– Logs begin at Sun 2021-08-22 07:50:51 UTC, end at Sun 2021-08-22 08:27:17 UTC. –
Aug 22 07:50:55 umbrel systemd[1]: Starting External Storage Mounter…
Aug 22 07:50:55 umbrel external storage mounter[505]: Running external storage mount script…
Aug 22 07:50:55 umbrel external storage mounter[505]: Found device “CT1000MX 500SSD1”
Aug 22 07:50:55 umbrel external storage mounter[505]: Blacklisting USB device IDs against UAS driver…
Aug 22 07:50:56 umbrel external storage mounter[505]: Rebinding USB drivers…
Aug 22 07:50:56 umbrel external storage mounter[505]: Checking USB devices are back…
Aug 22 07:50:56 umbrel external storage mounter[505]: Waiting for USB devices…
Aug 22 07:50:57 umbrel external storage mounter[505]: Waiting for USB devices…
Aug 22 07:50:58 umbrel external storage mounter[505]: Waiting for USB devices…
Aug 22 07:50:59 umbrel external storage mounter[505]: Checking if the device is ext4…
Aug 22 07:50:59 umbrel external storage mounter[505]: Yes, it is ext4
Aug 22 07:50:59 umbrel external storage mounter[505]: Checking if device contains an Umbrel install…
Aug 22 07:50:59 umbrel external storage mounter[505]: Yes, it contains an Umbrel install
Aug 22 07:50:59 umbrel external storage mounter[505]: Bind mounting external storage over local Umbrel installation…
Aug 22 07:50:59 umbrel external storage mounter[505]: Bind mounting external storage over local Docker data dir…
Aug 22 07:50:59 umbrel external storage mounter[505]: Bind mounting external storage to /swap
Aug 22 07:50:59 umbrel external storage mounter[505]: Bind mounting SD card root at /sd-card…
Aug 22 07:50:59 umbrel external storage mounter[505]: Checking Umbrel root is now on external storage…
Aug 22 07:51:00 umbrel external storage mounter[505]: Checking /var/lib/docker is now on external storage…
Aug 22 07:51:00 umbrel external storage mounter[505]: Checking /swap is now on external storage…
Aug 22 07:51:00 umbrel external storage mounter[505]: Setting up swapfile
Aug 22 07:51:03 umbrel external storage mounter[505]: Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
Aug 22 07:51:03 umbrel external storage mounter[505]: no label, UUID=3386e4b5-e2a2-49b3-a218-1aa7056b519a
Aug 22 07:51:03 umbrel external storage mounter[505]: Checking SD Card root is bind mounted at /sd-root…
Aug 22 07:51:03 umbrel external storage mounter[505]: Starting external drive mount monitor…
Aug 22 07:51:03 umbrel external storage mounter[505]: Mount script completed successfully!
Aug 22 07:51:03 umbrel systemd[1]: Started External Storage Mounter.

External storage SD card update service logs

– Logs begin at Sun 2021-08-22 07:50:51 UTC, end at Sun 2021-08-22 08:27:17 UTC. –
Aug 22 07:51:13 umbrel systemd[1]: Starting External Storage SDcard Updater…
Aug 22 07:51:13 umbrel external storage updater[957]: Checking if SD card Umbrel is newer than external storage…
Aug 22 07:51:13 umbrel external storage updater[957]: No, SD version is not newer, exiting.
Aug 22 07:51:13 umbrel systemd[1]: Started External Storage SDcard Updater.

Should be able to get to where its being used up using these commands :+1:

How to Check Disk Space in Linux {df and du Commands} (phoenixnap.com)

Managed to solve it - posting here for future ref if anyone has the same issue!

This is a known bug with electrs. The only way I know to fix this is resetting electrs, but please note that this means some apps won’t work and some wallets won’t connect during the ~8h indexing time.

sudo systemctl stop umbrel-startup && sudo rm -rf ~/umbrel/electrs/db/mainnet && sudo systemctl start umbrel-startup

This command via SSH will reset electrs

You can try rebooting and waiting a few hours, maybe it’ll compact the DB automatically

2 Likes

My hard drive is also full. How can I tell if the the issue I’m having is due to this bug?

I ran the command:

sudo systemctl stop umbrel-startup && sudo rm -rf ~/umbrel/electrs/db/mainnet && sudo systemctl start umbrel-startup

I got this as an error:

Job for umbrel-startup.service failed because the control process exited with error code.

See “systemctl status umbrel-startup.service” and “journalctl -xe” for details.


however, there is now 100 more GB available on my drive