How to fix persistent critical EXT4-fs error

Support and Troubleshooting
1

Was working fine until about a week ago after that it’s started crashing and overheating.

I have appropriate cooling mechanisms setup including this heatsink case

This fan which is just a regular foldable handled fan I got a few years ago as additional cooling for the raspberry pi 4 umbrel setup. Additional to this it’s located near a window away from direct sunlight.

umbrel login:
umbrel login:
umbrel login:
umbrel login: [ 215.155332] EXT4-fs error (device nvme0n1p7): ext4_check_bdev_write_error:224: comm kworker/u10:0: Error while async write back metadata
[ 215.155493] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #89645629: comm node: reading directory lblock 0
[ 215.156344] EXT4-fs error (device nvme0n1p7): ext4_check_bdev_write_error:224: comm kworker/u10:3: Error while async write back metadata
[ 215.156823] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #89664893: comm runc:[2:INIT]: reading directory lblock 0
[ 215.159004] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #91243409: comm python3: reading directory lblock 0
[ 215.159496] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #91243409: comm python3: reading directory lblock 0
[ 215.180537] EXT4-fs (nvme0n1p6): shut down requested (2)
[ 215.191567] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #89664914: comm runc:[2:INIT]: reading directory lblock 0
[ 215.191645] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #90866413: comm runc:[2:INIT]: reading directory lblock 0
[ 215.192104] EXT4-fs error (device nvme0n1p7): ext4_journal_check_start:84: comm dockerd: Detected aborted journal
[ 215.272456] EXT4-fs (nvme0n1p7): Remounting filesystem read-only

IMG_0684
IMG_06844242×1986 1.23 MB

Your Umbrel is now accessible at:
http://umbrel.local
http://192.168.1.240
http://192.168.1.235

umbrel login:
umbrel login:
umbrel login:
umbrel login: [ 276.607846] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #89665022: comm tor: reading directory lblock 0
[ 276.607847] EXT4-fs error (device nvme0n1p7): ext4_get_inode_loc:4528: inode #89663457: block 358613662: comm tor: unable to read itable block
[ 276.607921] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #89665010: comm tor: reading directory lblock 0
[ 276.611131] EXT4-fs error (device nvme0n1p7): ext4_rename:4010: inode #89663457: comm tor: mark_inode_dirty error
[ 276.612533] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #89616762: comm tor: reading directory lblock 0
[ 276.620427] EXT4-fs error (device nvme0n1p7): __ext4_find_entry:1682: inode #91128573: comm tor: reading directory lblock 0
[ 276.630401] EXT4-fs error (device nvme0n1p7): ext4_get_inode_loc:4528: inode #89660621: block 358613484: comm tor: unable to read itable block
[ 276.636053] EXT4-fs error (device nvme0n1p7): ext4_reserve_inode_write:5764: IO failure
[ 276.643616] EXT4-fs (nvme0n1p7): Delayed block allocation failed for inode 91237480 at logical offset 0 with max blocks 39 with error 5
[ 276.643627] EXT4-fs (nvme0n1p7): This should not happen!! Data will be lost
[ 276.643627] EXT4-fs (nvme0n1p7):
[ 276.743850] EXT4-fs (nvme0n1p6): shut down requested (2)

IMG_0685
IMG_06853576×2682 1.02 MB

Is there an easyish fix for this or is accepting that I once again have to reflash the SSD and resync the blockchain is the easyish option?

2

i would try a new cord.

I dont have any fan and its 105 f here outside. I dont overheat. I just have those cheap sticky heat syncs

3

Hey @josh

Are you able to boot another Linux live environment (distro doesn’t matter) on your Pi? If so, then you should be able to do a filesystem check / repair using fsck.

You can run it on the target volume that your umbrelOS is installed on:

sudo fsck.ext4 /dev/sdXY

Let me know if you need help with any of this.

4

Hey @josh, What power supply are you using for your pi? Also for the nvme drive is it connected via usb? I am wondering if a power issues is what initially caused your filesystem issues.

5

Good point to verify, maybe a power source more capable would be advisable.

6

Hi @cws0016 I’m using the official Raspberry Pi PSU. and the nvme drive is connected via this NVMe base

8

Thank you for your offer @473x31, but it appeared there was a problem drive itself, so I sent it back and now have a new one. It’s the same model and size. Before it crashed, I thought it might have something to do with the Monero node app which has recently become available on the UmrbelOS App Store, which was causing it to crash and overheat despite the cooling system I had installed. I’m only suspicious of this app because before I installed it, there was no issue with overheating or crashing, besides UmbrelOS itself being unstable.

Having received the replacement drive, I am now trying again with the Monero Node App, but this time there’s no crashing or overheating, so I’m not sure. I’ve been running a node since 2017 and using Umbrel since mid-2021, and haven’t experienced any issues this catastrophic.