Error : External Drive disconnected or Service System Failed

Support and Troubleshooting
1

Hi everyone,

Before to explain my error I will give some contexte :

Hardware :

  • Node : RaspberryPi 4B 4Go
  • Power Supply : The official Raspberry pi 4
  • SD card : SanDisk Extreme 64 Go
  • Box : Argon
  • SSD : Samsung 970 EVO 2To
  • SSD Box : Orico

Everything is brand new, and i don’t understand why it is not working,

The node start properly I can go on the WebGUI with no problem and then suddenly it and say that thing :

  • Error : External Drive disconnected
    or
  • Service System Failed

I’ve tried to re flashed multiple time It didn’t work, and i even do this :
sed -i ‘s/ blacklist_uas/ #blacklist_uas/g’ /home/umbrel/umbrel/scripts/umbrel-os/external-storage/mount && sudo reboot

So what’s should be my next move ?

In advance thank you for your time

2

Hi @Lumbia

Sorry to hear you’re having troubles here.
Can you please do the following for me- SSH back into your Umbrel instance and run this script.

sudo ~/umbrel/scripts/debug

You can paste the out here (with https://pastebin.com/ if’d you like), so we can help troubleshoot this.

3

if the logs do not show anything keep in mind that sometimes these drives are just bad. Could be a number of things like cable, drive, ssd adapter.
The good thing is you only just now started so you won’t lose anything much

That being said I am jumping to conclusions and we have not seen any logs yet.

4

Well now I’m not even able to connect myself by ssh, I’ll reflashed again the SD card, when I’ll encounter the bug again I’ll show you some logs

5

umbrel@umbrel:~ $ sudo ~/umbrel/scripts/debug

= Umbrel debug info =

Umbrel version

0.5.4

Flashed OS version

v0.5.4

Raspberry Pi Model

Revision : c03111
Serial : 10000000bf849b6a
Model : Raspberry Pi 4 Model B Rev 1.1

Firmware

May 9 2023 12:16:34
Copyright (c) 2012 Broadcom
version 30aa0d70ab280427ba04ebc718c81d4350b9d394 (clean) (release) (start)

Temperature

temp=43.8’C

Throttling

throttled=0x0

Memory usage

          total        used        free      shared  buff/cache   available

Mem: 3.8G 129M 3.0G 16M 696M 3.6G
Swap: 0B 0B 0B

total: 3.4%
system: 3.4%

Memory monitor logs

tail: cannot open ‘/home/umbrel/umbrel/scripts/…/logs/memory-monitor.log’ for reading: No such file or directory

Filesystem information

Filesystem Size Used Avail Use% Mounted on
/dev/root 59G 3.1G 53G 6% /
/dev/root 59G 3.1G 53G 6% /

Startup service logs

– Logs begin at Thu 2023-06-22 16:43:58 UTC, end at Mon 2023-09-11 09:03:39 UTC. –
Jun 22 16:44:16 umbrel systemd[1]: Dependency failed for Umbrel Startup Service.
Jun 22 16:44:16 umbrel systemd[1]: umbrel-startup.service: Job umbrel-startup.service/start failed with result ‘dependency’.

External storage service logs

– Logs begin at Thu 2023-06-22 16:43:58 UTC, end at Mon 2023-09-11 09:03:39 UTC. –
Jun 22 16:44:04 umbrel systemd[1]: Starting External Storage Mounter…
Jun 22 16:44:04 umbrel external storage mounter[504]: Running external storage mount script…
Jun 22 16:44:07 umbrel external storage mounter[504]: Found device “Realtek RTL9210 NVME”
Jun 22 16:44:07 umbrel external storage mounter[504]: Blacklisting USB device IDs against UAS driver…
Jun 22 16:44:07 umbrel external storage mounter[504]: Rebinding USB drivers…
Jun 22 16:44:07 umbrel external storage mounter[504]: Checking USB devices are back…
Jun 22 16:44:07 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:08 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:09 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:10 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:11 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:12 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:13 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:14 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:15 umbrel external storage mounter[504]: Waiting for USB devices…
Jun 22 16:44:16 umbrel external storage mounter[504]: USB devices weren’t registered after 10 tries…
Jun 22 16:44:16 umbrel external storage mounter[504]: Exiting mount script without doing anything
Jun 22 16:44:16 umbrel systemd[1]: umbrel-external-storage.service: Main process exited, code=exited, status=1/FAILURE
Jun 22 16:44:16 umbrel systemd[1]: umbrel-external-storage.service: Failed with result ‘exit-code’.
Jun 22 16:44:16 umbrel systemd[1]: Failed to start External Storage Mounter.

External storage SD card update service logs

– Logs begin at Thu 2023-06-22 16:43:58 UTC, end at Mon 2023-09-11 09:03:39 UTC. –
Jun 22 16:44:16 umbrel systemd[1]: Dependency failed for External Storage SDcard Updater.
Jun 22 16:44:16 umbrel systemd[1]: umbrel-external-storage-sdcard-update.service: Job umbrel-external-storage-sdcard-update.service/start failed with result ‘dependency’.

Karen logs

tail: cannot open ‘logs/karen.log’ for reading: No such file or directory

Docker containers

NAMES STATUS

Umbrel logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.auth.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object

Tor Proxy logs

ERROR: The Compose file ‘./docker-compose.yml’ is invalid because:
networks.default.ipam.config.subnet is invalid: should use the CIDR format
services.tor_proxy.ports contains an invalid type, it should be a number, or an object
services.auth.ports contains an invalid type, it should be a number, or an object
services.nginx.ports contains an invalid type, it should be a number, or an object

==== Result ====

==== END =====

6

I changed the ssd adapter twice, I changed the SD card three times, and I still don’t changed the ssd …

7

Well since I install again the OS it is now not mounted my SSD

8

I’ve follow again this command :
sed -i ‘s/ blacklist_uas/ #blacklist_uas/g’ /home/umbrel/umbrel/scripts/umbrel-os/external-storage/mount && sudo reboot

Now the disk is mounted buy I will see if it is down again after a few minutes

9

And now i got this :

umbrel@umbrel:~/umbrel $ ls
ls: reading directory ‘.’: Input/output error
umbrel@umbrel:~/umbrel $ pwd
/home/umbrel/umbrel

10

these are the log when the problem appear :

Sep 11 10:16:58 umbrel kernel: [ 876.680791] usb 2-1: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
Sep 11 10:17:01 umbrel kernel: [ 880.093791] usb 2-1: Enable of device-initiated U1 failed.
Sep 11 10:17:01 umbrel kernel: [ 880.095123] usb 2-1: Enable of device-initiated U2 failed.
Sep 11 10:17:01 umbrel kernel: [ 880.097895] scsi host0: uas_eh_device_reset_handler success
Sep 11 10:19:00 umbrel kernel: [ 999.160891] sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD
Sep 11 10:19:00 umbrel kernel: [ 999.160906] sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 74 45 59 f8 00 00 b0 00
Sep 11 10:19:20 umbrel kernel: [ 1018.873131] sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD
Sep 11 10:19:20 umbrel kernel: [ 1018.873146] sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x2a 2a 00 05 c4 21 50 00 00 10 00
Sep 11 10:19:20 umbrel kernel: [ 1018.901144] sd 0:0:0:0: [sda] tag#16 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD IN
Sep 11 10:19:20 umbrel kernel: [ 1018.901163] sd 0:0:0:0: [sda] tag#16 CDB: opcode=0x28 28 00 03 44 ca e8 00 00 08 00
Sep 11 10:19:20 umbrel kernel: [ 1018.917168] scsi host0: uas_eh_device_reset_handler start
Sep 11 10:19:20 umbrel kernel: [ 1019.050592] usb 2-1: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
Sep 11 10:19:25 umbrel kernel: [ 1024.237788] xhci_hcd 0000:01:00.0: Trying to add endpoint 0x83 without dropping it.
Sep 11 10:19:25 umbrel kernel: [ 1024.245355] scsi host0: uas_eh_device_reset_handler FAILED err -19
Sep 11 10:19:25 umbrel kernel: [ 1024.245384] sd 0:0:0:0: Device offlined - not ready after error recovery
Sep 11 10:19:25 umbrel kernel: [ 1024.245399] sd 0:0:0:0: Device offlined - not ready after error recovery
Sep 11 10:19:25 umbrel kernel: [ 1024.245412] sd 0:0:0:0: Device offlined - not ready after error recovery
Sep 11 10:19:25 umbrel kernel: [ 1024.245459] sd 0:0:0:0: [sda] tag#16 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x06 cmd_age=35s
Sep 11 10:19:25 umbrel kernel: [ 1024.245478] sd 0:0:0:0: [sda] tag#16 CDB: opcode=0x28 28 00 03 44 ca e8 00 00 08 00
Sep 11 10:19:25 umbrel kernel: [ 1024.245548] sd 0:0:0:0: [sda] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x06 cmd_age=36s
Sep 11 10:19:25 umbrel kernel: [ 1024.245564] sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x2a 2a 00 05 c4 21 50 00 00 10 00
Sep 11 10:19:25 umbrel kernel: [ 1024.245603] EXT4-fs warning (device sda1): ext4_end_bio:349: I/O error 10 writing to inode 3014995 starting block 12092460)
Sep 11 10:19:25 umbrel kernel: [ 1024.245651] sd 0:0:0:0: [sda] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x06 cmd_age=55s
Sep 11 10:19:25 umbrel kernel: [ 1024.245675] sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 74 45 59 f8 00 00 b0 00
Sep 11 10:19:25 umbrel kernel: [ 1024.246628] EXT4-fs warning (device sda1): ext4_end_bio:349: I/O error 10 writing to inode 791805 starting block 3196150)
Sep 11 10:19:25 umbrel kernel: [ 1024.250048] usb 2-1: USB disconnect, device number 2
Sep 11 10:19:26 umbrel kernel: [ 1024.746989] sd 0:0:0:0: [sda] Synchronizing SCSI cache
Sep 11 10:19:26 umbrel kernel: [ 1024.773099] br-b0181b6c38d3: port 10(veth7cd426a) entered disabled state
Sep 11 10:19:26 umbrel kernel: [ 1024.773647] vethbe4f68c: renamed from eth0
Sep 11 10:19:26 umbrel kernel: [ 1024.834437] veth2b755f8: renamed from eth0
Sep 11 10:19:26 umbrel kernel: [ 1024.855115] br-b0181b6c38d3: port 7(veth197390a) entered disabled state
Sep 11 10:19:26 umbrel kernel: [ 1024.899765] veth90d70ca: renamed from eth0
Sep 11 10:19:26 umbrel kernel: [ 1024.913951] br-b0181b6c38d3: port 4(veth2d15d73) entered disabled state
Sep 11 10:19:26 umbrel kernel: [ 1024.945875] br-b0181b6c38d3: port 6(vethbcc0f5f) entered disabled state
Sep 11 10:19:26 umbrel kernel: [ 1024.946133] veth7eb9212: renamed from eth0
Sep 11 10:19:26 umbrel kernel: [ 1024.985307] sd 0:0:0:0: [sda] Synchronize Cache(10) failed: Result: hostbyte=0x07 driverbyte=0x00
Sep 11 10:19:26 umbrel kernel: [ 1025.221846] usb 2-1: new SuperSpeed Gen 1 USB device number 3 using xhci_hcd
Sep 11 10:19:26 umbrel kernel: [ 1025.253244] usb 2-1: New USB device found, idVendor=0bda, idProduct=9210, bcdDevice=20.01
Sep 11 10:19:26 umbrel kernel: [ 1025.253258] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Sep 11 10:19:26 umbrel kernel: [ 1025.253263] usb 2-1: Product: RTL9210
Sep 11 10:19:26 umbrel kernel: [ 1025.253268] usb 2-1: Manufacturer: Realtek
Sep 11 10:19:26 umbrel kernel: [ 1025.253273] usb 2-1: SerialNumber: 012345678904
Sep 11 10:19:26 umbrel kernel: [ 1025.256516] usb 2-1: Enable of device-initiated U1 failed.
Sep 11 10:19:26 umbrel kernel: [ 1025.257775] usb 2-1: Enable of device-initiated U2 failed.
Sep 11 10:19:27 umbrel kernel: [ 1025.408929] br-b0181b6c38d3: port 8(veth5349cc4) entered disabled state
Sep 11 10:19:27 umbrel kernel: [ 1025.409683] veth782d128: renamed from eth0
Sep 11 10:19:27 umbrel kernel: [ 1025.462008] br-b0181b6c38d3: port 9(veth2e0ddf6) entered disabled state
Sep 11 10:19:27 umbrel kernel: [ 1025.463087] veth6c6bc75: renamed from eth0
Sep 11 10:19:27 umbrel kernel: [ 1025.481696] veth853a66f: renamed from eth0
Sep 11 10:19:27 umbrel kernel: [ 1025.505733] br-b0181b6c38d3: port 3(vethba57077) entered disabled state
Sep 11 10:19:27 umbrel kernel: [ 1025.586251] br-b0181b6c38d3: port 2(veth96ca928) entered disabled state
Sep 11 10:19:27 umbrel kernel: [ 1025.586633] vetheb74ecb: renamed from eth0
Sep 11 10:19:27 umbrel kernel: [ 1025.676352] br-b0181b6c38d3: port 5(veth08dc92a) entered disabled state
Sep 11 10:19:27 umbrel kernel: [ 1025.676593] br-b0181b6c38d3: port 1(veth6552a35) entered disabled state
Sep 11 10:19:27 umbrel kernel: [ 1025.677722] veth7d0745f: renamed from eth0
Sep 11 10:19:27 umbrel kernel: [ 1025.729846] vethf773ec7: renamed from eth0
Sep 11 10:19:29 umbrel kernel: [ 1028.279667] usb 2-1: Enable of device-initiated U1 failed.
Sep 11 10:19:29 umbrel kernel: [ 1028.280879] usb 2-1: Enable of device-initiated U2 failed.
Sep 11 10:19:29 umbrel kernel: [ 1028.284220] scsi host1: uas
Sep 11 10:19:29 umbrel kernel: [ 1028.288126] scsi 1:0:0:0: Direct-Access Realtek RTL9210 NVME 1.00 PQ: 0 ANSI: 6
Sep 11 10:19:29 umbrel kernel: [ 1028.295168] sd 1:0:0:0: Attached scsi generic sg0 type 0
Sep 11 10:19:29 umbrel kernel: [ 1028.307668] sd 1:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
Sep 11 10:19:29 umbrel kernel: [ 1028.309402] sd 1:0:0:0: [sdb] Write Protect is off
Sep 11 10:19:29 umbrel kernel: [ 1028.312721] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn’t support DPO or FUA
Sep 11 10:19:29 umbrel kernel: [ 1028.315648] sd 1:0:0:0: [sdb] Optimal transfer size 33553920 bytes
Sep 11 10:19:30 umbrel kernel: [ 1028.630545] sdb: sdb1
Sep 11 10:19:30 umbrel kernel: [ 1028.652933] sd 1:0:0:0: [sdb] Attached SCSI disk
Sep 11 10:19:31 umbrel kernel: [ 1029.812349] EXT4-fs error: 140 callbacks suppressed
Sep 11 10:19:38 umbrel kernel: [ 1036.526882] TCP: request_sock_TCP: Possible SYN flooding on port 8000. Sending cookies. Check SNMP counters.
Sep 11 10:19:41 umbrel kernel: [ 1039.856234] EXT4-fs error: 3 callbacks suppressed

11

kernel:[ 2184.760277] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 81792968, error -30)

Message from syslogd@umbrel at Sep 11 11:25:02 …
kernel:[ 2184.760359] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 79170369, error -30)

Message from syslogd@umbrel at Sep 11 11:25:02 …
kernel:[ 2184.760412] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 79037844, error -30)

Message from syslogd@umbrel at Sep 11 11:25:02 …
kernel:[ 2184.760468] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 79301914, error -30)

Message from syslogd@umbrel at Sep 11 11:25:02 …
kernel:[ 2184.760518] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 81791891, error -30)

Message from syslogd@umbrel at Sep 11 11:25:02 …
kernel:[ 2184.760565] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 13501425, error -30)

Message from syslogd@umbrel at Sep 11 11:25:02 …
kernel:[ 2184.760612] EXT4-fs (sda1): failed to convert unwritten extents to written extents – potential data loss! (inode 13501495, error -30)

12

Yeah unfortunately this is likely an issue with the drive.
We recommend SATA SSDs partly due to their lower power requirements. As I understand, your current drive sits on 6W at average, which may drop out occasionally given the power supply limitations of the Pi.

You may want to check you’re using the USB3.0 ports (blue) too. :pray: Some people also recommend powered USB hubs to make these work with the Pi when they’re playing up, but I’ve never tried it myself so can’t speak to it.

13

Check the RPi requirements to see which type of casing is compatible.
You can usually check by plugging in a USB2 port and if the disk works, then the USB3 port is incompatible with the enclosure.