I’ve upgraded from 0.4.3 to 0.4.4 a few days ago but my Umbrel node is still showing sync state. Bitcoin.d seems to be synchronized but Lightning node is not working…
My debug log: https://umbrel-paste.vercel.app/cae28354c30afeea610d1be398c38278
Need help from some of the experts on the community…
Thanks in advance
Same problem.
In the log file i see that the bitcoin and electrs docker containers restart every 10-20 seconds.
Same problem here too. Troubleshooter did not find any issues. I check if my electrs docker containers also restarting every few seconds when I get home.
I found my problem.
The last update somehow destroyed the blockchain on the SSD.
I found the following line in the debug.log file
ERROR: ReadRawBlockFromDisk: Read from block file failed: CAutoFile::read: end of file: iostream error for CBlockDiskPos(nFile=0, nPos=16830050).
My only solution is to reinstall Umbrel, and wait weeks till the Blockchain is synchronized.
Did you change any password between upgrades?
I am starting to think we need to clone a synched backup SSD for situation like these.
node has been running fine for about 4-5 month without any issue until the last update. No password or anything else was changed between upgrades.
I just ran the command “docker-compose logs electrs” in SSH. There are quite a few errors showing up.
e.g.
electrs | Error: failed to open bitcoind cookie file: /data/.bitcoin/.cookie
electrs |
electrs | Caused by:
electrs | No such file or directory (os error 2)
or
electrs | Error: daemon not available
electrs |
electrs | Caused by:
electrs | JSON-RPC error: transport error: Didn’t receive response data in time, timed out.
full log is here with many more errors shown.
Could please anyone of the developers have a look to see if this can be fixed.
Guys… patience is the key.
This update is bringing a new version of Electrum server. That means it will create a new index database.
So be patient until is re-indexing
You should see in electrs logs something like this when is done
The command to check electrs logs I mention it in the Troubleshooting manual
That manual is there TO BE READ.
Mate, I know you are just trying to help but being patient and waiting does not help when the node just goes in circles for days. There are error messages in the file I had attached yesterday. It would be good if someone who understands the errors could have look to find out what’s wrong here.
One example:
electrs | [2021-10-12T13:47:16.164Z INFO electrs::index] indexing 2000 blocks: [415731…417730]
electrs | [2021-10-12T13:47:17.373Z INFO electrs::db] closing DB at /data/db/bitcoin
electrs | Error: initial sync failed
electrs |
electrs | Caused by:
electrs | 0: failed to get block 000000000000000004021ca4dab4f2dbefd61685a3aab9d5b84abe4b6195344f
electrs | 1: p2p failed to recv
electrs | 2: I/O error: unexpected end of file
electrs | Config { network: Bitcoin, db_path: “/data/db/bitcoin”, daemon_dir: “/data/.bitcoin”, daemon_auth: CookieFile(“/data/.bitcoin/.cookie”), daemon_rpc_addr: V4(10.21.21.8:8332), daemon_p2p_addr: V4(10.21.21.8:8333), electrum_rpc_addr: V4(0.0.0.0:50001), monitoring_addr: V4(127.0.0.1:4224), wait_duration: 10s, jsonrpc_timeout: 15s, index_batch_size: 10, index_lookup_limit: Some(200), auto_reindex: true, ignore_mempool: false, sync_once: false, server_banner: “Umbrel v0.4.4”, args: }
electrs | [2021-10-12T13:47:19.991Z INFO electrs::metrics::metrics_impl] serving Prometheus metrics on 127.0.0.1:4224
electrs | [2021-10-12T13:47:20.211Z INFO electrs::db] “/data/db/bitcoin”: 2097 SST files, 9.618480734 GB, 0.808878916 Grows
electrs | [2021-10-12T13:47:22.754Z INFO electrs::chain] loading 415730 headers, tip=000000000000000000bfce1643b9dd13480c9a37ba8441cc4477f0df1704ba92
electrs | [2021-10-12T13:47:25.208Z INFO electrs::chain] chain updated: tip=000000000000000000bfce1643b9dd13480c9a37ba8441cc4477f0df1704ba92, height=415730
OK this commands seem to have fixed the problem:
sudo systemctl stop umbrel-startup && sudo rm -rf ~/umbrel/electrs/!(electrs.toml) && sudo systemctl start umbrel-startup
After about 10 minutes umbrel finally caught up with syncing the blocks. Hopefully it stays like that.
The above command only fixed it for a short period of time, the next day it was back to loop the blocks in circles.
Next I tried to install the latest version 0.4.4 on a new mSD card so I could keep the old mSD just in case. That started all normally but the it was still doing the same thing as before, syncing to 100% then start again from about 99.95% and so on.
Next I manually downgraded the node to version 0.4.1 using this command:
cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.1
This is now running for about 2 days with no issue.
Hope that helps others having the same issue. For now I keep running the old version.
Same problem here - Juhurus could you tell me how I can get access to Umbrel Terminal from Windows machine to run the the downgrade line?
Sorry complete noob here
Same problem here too. I need understand how to downgrade the version manually. I am also a noob, some guidance needed.
Thank You
This would be a great option to have exposed to the Settings UI so that users can roll back in case of new version issues.
SSH into your Umbrel, I always use Putty but you can use Windows 10 PowerShell as well.
Once you connected through SSH use this command to downgrade to Version 0.4.1:
cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.1
This worked for me is since running fine again.
Or you can just reflash the mSD card with latest umbrelOS that is 0.4.4
No, that does not work. I flashed a new mSD card with the latest version 0.4.4 and it had exactly the same issue as after the normal update. For me only the version 0.4.1 works. Waiting for the next update to see if that fixes the issue.
Thanks juhurus, I finally managed to login through windows Shell using ssh umbrel@umbrel.local. I then ran cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.1 but got some errors and when trying to reboot now the Umbrel interface tells me “Pulling new Containers…” and its just hanging there forever.
When trying to rerun cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.1 I get the following
You are in ‘detached HEAD’ state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b
An update is already in progress. Exiting now.
Any idea whats going on?
Did you use the original mSD card or a new one? As I said above I did not want to loose anything and I was uncertain where all the channel info and balance is stored that’s why I just setup a brand new card with 0.4.4. All channels and balance was there so it must not be be stored on the mSD card.
But because it had the exact same issue as the original I then downgraded with the command
cd ~/umbrel && sudo ./scripts/update/update --repo getumbrel/umbrel#v0.4.1
which worked for me, but I haven’t tried the original mSD yet to see if it would work now too.
Ah gotcha - yeah i used same mSD card. Will try with a fresh one then
