Upgrade to 0.4.16 went wrong - "upgrading installed apps"

Support and Troubleshooting
1

Hi,
I’ve tried to upgrade my umbrel from 4.15 to 4.16. Unfortunately I cant log in anymore after that.
Right now web interface says “upgrading installed apps”.
Can login via ssh (although it seems very slow - although it might be just my lan network - I’m using some “internet over power cables” adaptors).
I’ve reached out for help in telegram group and were able (thanks to @Indomitorum) to start lnd and other services, waited for it to update the chainstate but once finished it wasn’t showing my balance (bitcoin-cli getbalance) and showed some sats in lncli walletbalance (but not sure if the right amount since it’s been a while since it died so I’m not sure if its right…).
After that we decided it might be broken container - so I’ve done:
sudo scripts/stop
sudo rm .env && sudo ./scripts/configure
sudo reboot
sudo scripts/start
After reboot I still cannot access web interface - and lnd log says wallet locked (even after doing lncli unlock):

tail -f lnd/logs/bitcoin/mainnet/lnd.log

Blockquote

2022-03-25 21:51:51.354 [INF] LTND: Creating local graph and channel state DB instances
2022-03-25 21:51:55.280 [INF] CHDB: Checking for schema update: latest_version=24, db_version=24
2022-03-25 21:51:55.281 [INF] LTND: Database(s) now open (time_to_open=6.316880626s)!
2022-03-25 21:51:55.281 [INF] LTND: We’re not running within systemd
2022-03-25 21:51:55.281 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create a wallet, lncli unlock to unlock an existing wallet, or lncli changepassword to change the password of an existing wallet and unlock it.
2022-03-25 21:51:59.476 [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:51:59.476 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:51:59.477 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:51:59.477 [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:04.739 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
q2022-03-25 21:52:19.662 [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:19.662 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:19.663 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:19.663 [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:19.771 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:31.061 [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:31.062 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:31.062 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:31.062 [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:34.789 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:40.940 [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:40.940 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:40.940 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:40.940 [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:45.617 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:45.690 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:52:48.942 [INF] LNWL: Opened wallet
2022-03-25 21:52:49.379 [INF] CHRE: Primary chain is set to: bitcoin
2022-03-25 21:52:49.646 [ERR] LTND: unable to create partial chain control: status code: 401, response: “”
2022-03-25 21:52:49.647 [ERR] LTND: Shutting down because error in main method: error creating wallet config: unable to create partial chain control: status code: 401, response: “”
2022-03-25 21:52:53.065 [INF] LTND: Version: 0.14.2-beta commit=v0.14.2-beta, build=production, logging=default, debuglevel=info
2022-03-25 21:52:53.066 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2022-03-25 21:52:53.068 [INF] RPCS: RPC server listening on 0.0.0.0:10009
2022-03-25 21:52:53.072 [INF] RPCS: gRPC proxy started at 0.0.0.0:8080
2022-03-25 21:52:53.073 [INF] LTND: Opening the main database, this might take a few minutes…
2022-03-25 21:52:53.073 [INF] LTND: Opening bbolt database, sync_freelist=false, auto_compact=false
2022-03-25 21:52:54.191 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: waiting to start, RPC services not available
2022-03-25 21:52:55.498 [INF] LTND: Creating local graph and channel state DB instances
2022-03-25 21:52:59.115 [INF] CHDB: Checking for schema update: latest_version=24, db_version=24
2022-03-25 21:52:59.115 [INF] LTND: Database(s) now open (time_to_open=6.041924995s)!
2022-03-25 21:52:59.116 [INF] LTND: We’re not running within systemd
2022-03-25 21:52:59.116 [INF] LTND: Waiting for wallet encryption password. Use lncli create to create a wallet, lncli unlock to unlock an existing wallet, or lncli changepassword to change the password of an existing wallet and unlock it.
2022-03-25 21:53:03.392 [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:03.393 [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:03.392 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:03.393 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:04.846 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:13.530 [ERR] RPCS: [/lnrpc.Lightning/SubscribeInvoices]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:13.530 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:13.530 [ERR] RPCS: [/lnrpc.Lightning/GetInfo]: wallet locked, unlock it to enable full RPC access
2022-03-25 21:53:13.530 [ERR] RPCS: [/lnrpc.Lightning/ListChannels]: wallet locked, unlock it to enable full RPC access`

Blockquote

My debug log:
https://0bin.net/paste/LQaO1xtJ#J8uNYkh4LyqXuYpMsNuFSqxpPNBg1HeIFn14xT7I2j+

What should I attempt next to revive it?

Thanks

2

I’ve reflashed new microSD card and followed instructions from this thread (LND `lncli unlock` command 'succeeds' but wallet isn't unlocked) and done:

rm ~/umbrel/lnd/lnd.conf && ~/umbrel/scripts/configure && sudo reboot

And now lost ability to connect via ssh… None of the passwords work (default: moneyprintergobrrr or the one I’ve chosen while installing) - got few more passwords saved in my browser history but they’re for different ports numbers so different services…

How should I proceed? Web gui now says:
Error: System service failed

Something went wrong when starting Umbrel. Need help? Feel free to jump on our community forum or Telegram chat.

Can someone point me what to do next?

3

Sorry, no help, just my sympathy:

I have the same issue after trying to upgrade yesterday…
It froze at “upgrading installed apps” and I can’t log in after that.

Hope you manage to solve it, so I can copy your method(!)

4

Still no luck…
I’ve swapped the old sd card back in so I can now login back with ssh.
Still showing dreaded “Updating installed apps” on web interface after doing sudo scripts/start.
Uploaded new debug log on 0bin:
https://0bin.net/paste/2JwIvg8i#l0tfY1QqIiFsRex6tRWXy8FPfLYNFSobFo-r2mMaHcG
Can anyone advice on next steps?

5

Everything basic seems running.
Some app is problematic with mixed versions confusion you did.
Remove all apps. Flash and use SD with version 0.4.15.
Wait to sync wallets, can take lot of time.