Reflash not helping

Rasp Pi 4
SSD
Ethernet

I’ve been having issues with my router and it shut down unexpectedly, which affected the Umbrel and it shut down.
I reflashed the latest version of umbrel (v0.4.15) - I have done this several times in the past and it always worked fine with Umbrel back up and running very quickly. However this time it didn’t work and I just keep getting an error message.

I decided to reformat the SD card and try again but I get the same result.

Hey @Catoshi, am not really technical… but hopefully this post will help you

Awesome, thank you so much N080dy :cowboy_hat_face:

1 Like

I bought a brand new SD card, reflashed it but no response when I fired up the RaspPi.
I have no idea how to ssh, I tried following the commands in a guide I found (Darthcoin)
ssh umbrel@umbrel.local
on my Mac terminal app but it didn’t seem to do anything.
Is my rasp Pi fried? Any ideas?

So you could login through SSH or not?
Maybe instead of umbrel.local use node IP

I could not get in through SSH.
Using IP helped bring up the Error: System service failed message, which I hadn’t been getting through umbrel.local
At least I can shut it down correctly from there.
Please talk me through any steps I could take with SSH. I know nothing about coding, at the moment my terminal screen (mac) shows ~ %

SSH is just a terminal where you can interact with your node through command lines. That’s all.
So once you open a terminal on your computer and enter:

ssh -t umbrel@umbrel.local

Instead of umbrel.local also you can use the local IP of your node.

The password is your personal Umbrel login password. But you have to type it in blank (you will not see what you type) and press ENTER.

After that you start using the debug commands described in the troubleshooting manual for what you need.

1 Like

thanks a lot. I was able to SSH in and I ran the debug command. However it’s too many characters to post here.
I have posted the start of it below

=====================
= Umbrel debug info =

Umbrel version

0.4.15

Flashed OS version

v0.4.15

Raspberry Pi Model

Revision : c03112
Serial : 100000002265081e
Model : Raspberry Pi 4 Model B Rev 1.2

Firmware

Dec 1 2021 15:01:54
Copyright © 2012 Broadcom
version 71bd3109023a0c8575585ba87cbb374d2eeb038f (clean) (release) (start)

Temperature

temp=33.6’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 3.8G 98M 3.5G 8.0M 202M 3.6G
Swap: 0B 0B 0B

total: 2.6%
system: 2.6%
tor: 0%
thunderhub: 0%
specter-desktop: 0%
simple-torrent: 0%
ride-the-lightning: 0%
mempool: 0%
lnd: 0%
electrs: 0%
btc-rpc-explorer: 0%
bluewallet: 0%
bitfeed: 0%
bitcoin: 0%

Memory monitor logs

2022-03-12 22:02:32 Memory monitor running!
2022-03-12 22:02:44 Memory monitor running!
2022-03-12 22:02:56 Memory monitor running!
2022-03-12 22:03:08 Memory monitor running!
2022-03-12 22:03:25 Memory monitor running!
2022-03-12 22:03:37 Memory monitor running!
2022-03-12 22:03:49 Memory monitor running!
2022-03-12 22:04:01 Memory monitor running!
2022-03-12 22:04:13 Memory monitor running!
2022-03-
Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 29G 3.1G 25G 12% /
/dev/sda1 916G 916G 0 100% /home/umbrel/umbrel

Startup service logs

– Logs begin at Tue 2022-03-15 08:53:15 UTC, end at Tue 2022-03-15 21:35:30 UTC. –
Mar 15 08:53:25 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
Mar 15 08:53:25 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result ‘dependency’.

External storage service logs

– Logs begin at Tue 2022-03-15 08:53:15 UTC, end at Tue 2022-03-15 21:35:30 UTC. –
Mar 15 08:53:19 umbrel systemd[1]: Starting External Storage Mounter…
Mar 15 08:53:19 umbrel external storage mounter[482]: Running external storage mount script…
Mar 15 08:53:19 umbrel external storage mounter[482]: Found device “JMicron Tech”
Mar 15 08:53:19 umbrel external storage mounter[482]: Blacklisting USB device IDs against UAS driver…
Mar 15 08:53:19 umbrel external storage mounter[482]: Rebinding USB drivers…
Mar 15 08:53:19 umbrel external storage mounter[482]: Checking USB devices are back…
Mar 15 08:53:19 umbrel external storage mounter[482]: Waiting for USB devices…
Mar 15 08:53:20 umbrel external storage mounter[482]: Waiting for USB devices…
Mar 15 08:53:21 umbrel external storage mounter[482]: Checking if the device is ext4…
Mar 15 08:53:21 umbrel external storage mounter[482]: Yes, it is ext4
Mar 15 08:53:22 umbrel external storage mounter[482]: Checking if device contains an Umbrel install…
Mar 15 08:53:22 umbrel external storage mounter[482]: Yes, it contains an Umbrel install
Mar 15 08:53:22 umbrel external storage mounter[482]: Bind mounting external storage over local Umbrel installation…
Mar 15 08:53:22 umbrel external storage mounter[482]: Bind mounting external storage over local Docker data dir…
Mar 15 08:53:22 umbrel external storage mounter[482]: Bind mounting external storage to /swap
Mar 15 08:53:22 umbrel external storage mounter[482]: Bind mounting SD card root at /sd-card…
Mar 15 08:53:22 umbrel external storage mounter[482]: Checking Umbrel root is now on external storage…
Mar 15 08:53:23 umbrel external storage mounter[482]: Checking /var/lib/docker is now on external storage…
Mar 15 08:53:23 umbrel external storage mounter[482]: Checking /swap is now on external storage…
Mar 15 08:53:23 umbrel external storage mounter[482]: Setting up swapfile
Mar 15 08:53:25 umbrel external storage mounter[482]: fallocate: fallocate failed: No space left on device
Mar 15 08:53:25 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 08:53:25 umbrel systemd[1]: umbrel-external-storage.service: Failed with result ‘exit-code’.
Mar 15 08:53:25 umbrel systemd[1]: Failed to start External Storage Mounter.

External storage SD card update service logs

– Logs begin at Tue 2022-03-15 08:53:15 UTC, end at Tue 2022-03-15 21:35:30 UTC. –
Mar 15 08:53:25 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
Mar 15 08:53:25 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result ‘dependency’.

Karen logs

Creating encrypted tarball…
backup/
backup/channel.backup
backup/.padding
Uploading backup…
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3254 100 146 100 3108 72 1539 0:00:02 0:00:02 --:–:-- 1611
{“message”:“Successfully uploaded backup 1647035124217.tar.gz.pgp for backup ID 2e727cc2eeef6b65732b8e2c45963741ec9989318dac3975495cd9742f9d8716”}

====== Backup success =======

Got signal: backup
karen is getting triggered!
Deriving keys…
Creating backup…
Adding random padding…
1+0 records in
1+0 records out
6696 bytes (6.7 kB, 6.5 KiB) copied, 0.000389496 s, 17.2 MB/s
Creating encrypted tarball…
backup/
backup/channel.backup
backup/.padding
Uploading backup…
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9550 100 146 100 9404 85 5531 0:00:01 0:00:01 --:–:-- 5617
{“message”:“Successfully uploaded backup 1647074542947.tar.gz.pgp for backup ID 2e727cc2eeef6b65732b8e2c45963741ec9989318dac3975495cd9742f9d8716”}

====== Backup success =======

karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/umbrel/events
karen is running in /home/umbrel/
Docker containers

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

There are a lot of lines like the following :cowboy_hat_face:
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/lib/python3.7/dist-packages/docker/api/client.py”, line 214, in _retrieve_server_version
return self.version(api_version=False)[“ApiVersion”]
File “/usr/local/lib/python3.7/dist-packages/docker/api/daemon.py”, line 181, in version
return self._result(self._get(url), json=True)
File “/usr/local/lib/python3.7/dist-packages/docker/utils/decorators.py”, line 46, in inner
return f(self, *args, **kwargs)
File “/usr/local/lib/python3.7/dist-packages/docker/api/client.py”, line 237, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File “/usr/lib/python3/dist-packages/requests/sessions.py”, line 546, in get
return self.request(‘GET’, url, **kwargs)
File “/usr/lib/python3/dist-packages/requests/sessions.py”, line 533, in request
resp = self.send(prep, **send_kwargs)
File “/usr/lib/python3/dist-packages/requests/sessions.py”, line 646, in send
r = adapter.send(request, **kwargs)
File “/usr/lib/python3/dist-packages/requests/adapters.py”, line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: (‘Connection aborted.’, ConnectionRefusedError(111, ‘Connection refused’))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/bin/docker-compose”, line 10, in
sys.exit(main())

But these windows only allow 30000 characters or something, this script was over 80000

Paste all the log into 0bin.net and post here the link to that bin file.

Cool

https://0bin.net/paste/x1Ev9qpL#eWbigUxZM+sTl6SDqqX1336BH+prB-9dr4mRcyuABdI

OK seems that some docker packages are missing.
Reflash the mSD card with the latest version 0.4.16 or even change that mSD card, maybe is damaged.

OK, will do. The SD card was new before last reflash 0.4.15 two days ago
Will try again with v.16
Thanks

Unfortunately reflashing 0.4.16 didn’t work, here’s the latest debug script:
https://0bin.net/paste/TFqzKzMB#o4WZQmEcff+qNIrHn6th2kS4YDhUGqY5HUhUXpMSVnT

Mar 15 22:39:17 umbrel external storage mounter[481]: fallocate: fallocate failed: No space left on device
Mar 15 22:39:17 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 22:39:17 umbrel systemd[1]: umbrel-external-storage.service: Failed with result ‘exit-code’.
Mar 15 22:39:17 umbrel systemd[1]: Failed to start External Storage Mounter.

External storage SD card update service logs

– Logs begin at Tue 2022-03-15 22:39:07 UTC, end at Tue 2022-03-15 22:40:59 UTC. –
Mar 15 22:39:17 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
Mar 15 22:39:17 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result ‘dependency’.

I think the answer is there: something wrong with that mSD card.

Logs simply say many times SSD disk is full, but people don´t care logs… :man_shrugging:

so a 64GB mSD should solve the problem?

Look’s like the current one is corrupted or full or for whatever reason is not working as it should. If you have another (32GB is enough, 64GB maybe better) one try with a fresh umbrel install and follow the steps to restore your node.

2 Likes

It’s always good to have a spare mSD ready to replace. Those little fuckers broke very easy.

1 Like

I can´t stop laughing. People don´t know what is a SSD… :laughing:

1 Like