"Error: System Service Failed"

[mbncsa]

Hi , I tried to revive an old thread to this a week ago, sadly without success. So I repost this as a new thread (hope that’s ok):

I noticed my note not reacting (snowflake log showed last traffic about two weeks ago). When I logged in, the Blockchain wasn’t syncing. I ran the suggested updates and restarted, after that, I got this error message. The node ran fine until then. When I logged in, the only thing was that it told me it was running hot currently.
I made a debugging log per the guide, which can be found here.

I successfully recovered the channel backup and the backup ID via SSH but I am lost on how to idetify the cause of the error and how to fix it. Any help would be greatly appreciated!

[mbncsa]

Hi, sadly I still don’t know if my node can be saved. Enclosed you find a selection from the lof I posted in the original message. Various start-up processes fail, but I do not know what I can do about it. Help would be greatly apprciated!

Jun 18 18:52:03 umbrel umbrel startup[19275]: ERROR: for manager  Cannot start service manager: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/a3d38f7607129100c758ab445e615437886dee15f2adaf5a47f5b6be6f11b40d: no space left on device: unknown
Jun 18 18:52:03 umbrel umbrel startup[19275]: Starting tor_server ... error
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for tor_server  Cannot start service tor_server: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/00c759772eaa7d57e0ecd11678aeeb468092d0b24b5add832d2d61ddda4782d6: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: Starting dashboard  ... error
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for dashboard  Cannot start service dashboard: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/4965fc749a17fba5861a97aedfb0017dd82da584be38cc9f225fff825f35fc17: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: Starting auth       ... error
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for auth  Cannot start service auth: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/3868e04c595ed7132afac5682ae8d56e9c7b5ff736cf54d2272e399c41775e30: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: Starting tor_proxy  ... error
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for tor_proxy  Cannot start service tor_proxy: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/294578eb81b52949fcadb145b41093703c729195abdde9c459a0a97b05a391ff: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for manager  Cannot start service manager: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/a3d38f7607129100c758ab445e615437886dee15f2adaf5a47f5b6be6f11b40d: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for tor_server  Cannot start service tor_server: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/00c759772eaa7d57e0ecd11678aeeb468092d0b24b5add832d2d61ddda4782d6: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for dashboard  Cannot start service dashboard: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/4965fc749a17fba5861a97aedfb0017dd82da584be38cc9f225fff825f35fc17: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for auth  Cannot start service auth: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/3868e04c595ed7132afac5682ae8d56e9c7b5ff736cf54d2272e399c41775e30: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: ERROR: for tor_proxy  Cannot start service tor_proxy: failed to start shim: mkdir /var/lib/containerd/io.containerd.runtime.v2.task/moby/294578eb81b52949fcadb145b41093703c729195abdde9c459a0a97b05a391ff: no space left on device: unknown
Jun 18 18:52:04 umbrel umbrel startup[19275]: Encountered errors while bringing up the project.

[smolgrrr]

Looks like your node is having trouble with the update process- try running this and then give your node some time to start syncing again.

rm -f ~/umbrel/statuses/update-in-progress && sudo reboot

Let me know how that goes!

[smolgrrr]

also, this guide here.

@usernameisJim

As a disclaimer please always make sure you have secured your 24 word seed phrase before making any drastic system changes, primarily if you skip the step to reflash your MicroSD card (where you don’t lose any data) and instead skip to reformatting your storage drive then you would lose all your data (all data is stored on the storage drive and will result in a clean install)

This error is the result of several issues usually occurring on Raspberry Pi 4 devices and related to the MicroSD card and/or a read/write issue with the installed storage drive.

This is for when you receive this screen specifically:

It can be caused by a failing MicroSD or a storage device that has not been formatted correctly in ext4, has become corrupted, or simply had any data on it prior to attempting a new installation.

The most reliable fix for this issue if you are on the Raspberry Pi 4 platform is to reflash your MicroSD card. If you have another one you can swap in, please try another card as well

SSH and run this command to safely shutdown: sudo shutdown

Unplug power and reflash your MicroSD card, then when complete, plug in your power last and wait ~5 minutes

For reference, reflashing will not result in any data loss because all user data is stored on your external storage drive (SSD or HDD).

To reflash your MicroSD card, follow the steps on our website under “Install on a Rasberry Pi 4.”

If it is your first time setting up Umbrel, and you don’t have any data on it yet please try reformatting your storage device again and make sure all partitions are completely clear- this is also a common cause if you already had a prior installation or any data on your drive and are trying to setup like new once more. You can delete all partitions and format in any default format file type, upon a successful Umbrel installation- during the Umbrel setup it will then automatically format again in the correct ext4.

For Windows you can use a program called AOMEI partition assistant to detect and manage ext4 disks and format them. On Mac there is a program suite from Paragon software you can also use to manage ext4 disks, to avoid the hassle of manually mounting and wiping an ext4 drive. This process is detailed in another guide here.

If after a reflash or this problem persists on a different Linux installation please reach out to us so we can track and resolve these issues for you to avoid having to delete and reinstall. You can always connect directly with our team using the chat icon located at the bottom right of Umbrel.com or make a post in our Help & Support Telegram Channel

When reaching out let us know when this issue began and provide us your debug logs for review so we can investigate and get you back up and running.

Following the instructions in How to SSH into Umbrel on a Raspberry Pi and Generating Troubleshooting Logs

[RyanStackSats]

Just setting up my raspberry pi and node for the first time, having the same “system service failure”, I’ve tried reflashing the SD, and obviously restarting the machine. Short of buying new cords, and hardware, what else can I do? Its all official parts, it came in a kit, i’m using a SanDisk 2TB pro SSD. I’m new to this, so forgive me if i overlooked something obvious, but can anybody help?

[smolgrrr]

Similar to this?
Another thing worth trying first is reformatting the SSD
How to easily reformat an ext4 / Umbrel drive