First, a huge THANKS to people like @DarthCoin and @Hakuna and everyone else who volunteers time here, to help us faceless nodlers. Hoping a kind soul will try helping me solve:
--------------- PROBLEM:
After working successfully for 16 months, the container for my LND became lost / horked:
umbrel@umbrel:~ $ /home/umbrel/lncli
Error: No such container: lnd
--------------- SCOPE:
The apps which are unrelated to LND (Bitcoin Node, mempool, etc) are ok:
umbrel@umbrel:~ $ /home/umbrel/umbrel/scripts/app compose bitcoin exec bitcoind bitcoin-cli getblockcount
753717
Installing and uninstalling other apps causes no errors.
--------------- SYMPTOMS:
LND-related apps ok since April 2021, but either my upgrade to 0.5.0, or a recent hard reset, causes apps to consistently throw these errors:
Ride The Lightning (http://umbrel.local:3001/rtl/) throws:
Error 500
Error: connect ehostunreach 10.21.21.9:8080
ThunderHub (http://umbrel.local:3000/login) throws:
UnableToConnectToThisNode
Lightning Node (http://umbrel.local:2101/) throws:
The purple progress line is stuck at about 55%
BlueWallet Lightning (http://umbrel.local:3008) throws:
There was an error connecting to BlueWallet Lightning. Error code: ECONNREFUSED
--------------- HARDWARE:
Raspberry Pi
SSD: 594GB of 983GB
RAM: 2.8 GB of 4 GB
CPU: Normal temperature
Ethernet (not WiFi)
--------------- OS:
Umbrel: 0.5.1-build-7ab0914
umbrel@umbrel:~ $ uname -a
Linux umbrel 5.10.103-v8+ #1529 SMP PREEMPT Tue Mar 8 12:26:46 GMT 2022 aarch64 GNU/Linux
--------------- LND:
umbrel@umbrel:~ $ ps -ef | grep lnd
root 1171 1159 0 19:52 ? 00:00:00 fswatch -0 --event Updated /home/umbrel/umbrel/scripts/backup/…/…/app-data/lightning/data/lnd/data/chain/bitcoin/mainnet/channel.backup
umbrel 4433 4395 0 19:52 ? 00:00:02 boltzd --lnd.host=“10.21.21.9” --lnd.macaroon="/lnd/data/chain/bitcoin/mainnet/admin.macaroon" --lnd.certificate="/ln/tls.cert" --rpc.rest.host=“10.21.21.63” --rpc.rest.port=“9003”
umbrel 12533 12513 6 20:23 ? 00:00:00 /sbin/docker-init – docker-entrypoint.sh /bin/sh -c cp $LND_CERT_FILE /lndhub/ && cp $LND_ADMIN_MACAROON_FILE /lndhub/ && cd /lndhub && npm start
LND does see the network:
umbrel@umbrel:~ $ /home/umbrel/umbrel/scripts/app compose lightning exec lnd lncli getnetworkinfo
{
“graph_diameter”: 12,
“avg_out_degree”: 5.708576881784564,
“max_out_degree”: 143,
“num_nodes”: 16766,
“num_channels”: 47855,
“total_network_capacity”: “127320983370”,
“avg_channel_size”: 2660557.587921847,
“min_channel_size”: “1050”,
“max_channel_size”: “500000000”,
“median_channel_size_sat”: “1000000”,
“num_zombie_chans”: “127199”
}
…and LND does see my 42 channels:
umbrel@umbrel:~ $ /home/umbrel/umbrel/scripts/app compose lightning exec lnd lncli listchannels | grep chan_id | wc -l
42
…but calling the LND container fails:
umbrel@umbrel:~ $ /home/umbrel/lncli
Error: No such container: lnd
------------------------- DOCKER:
umbrel@umbrel:~ $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3b8c642ed14f getumbrel/umbrel-bitcoin:v0.3.0 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 3006/tcp bitcoin_server_1
bb4d6584e3af bluewalletorganization/lndhub:v1.4.1 “docker-entrypoint.s…” 25 minutes ago Up 21 minutes 3000/tcp bluewallet_lndhub_1
e21cabdda1f7 getumbrel/umbrel-electrs:v1.0.2 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 3006/tcp electrs_app_1
5317fb2da9c2 lncm/bitcoind:v23.0 “bitcoind -chain=mai…” 25 minutes ago Up 25 minutes 8080/tcp, 18332-18333/tcp, 18443-18444/tcp, 28332-28333/tcp, 0.0.0.0:8332-8333->8332-8333/tcp, :::8332-8333->8332-8333/tcp bitcoin_bitcoind_1
497754a60e2c getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 0.0.0.0:3004->3004/tcp, :::3004->3004/tcp lightning-terminal_app_proxy_1
47e85f8bec4a lncm/tor:0.4.7.8 “tor” 25 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp bitcoin_tor_server_1
afbe970b0eb5 lncm/tor:0.4.7.8 “tor” 25 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp lightning-terminal_tor_server_1
b122c3541b86 lightninglabs/lightning-terminal:v0.7.1-alpha “litd --uipassword_e…” 25 minutes ago Up 21 minutes 8443/tcp, 9735/tcp, 10009/tcp lightning-terminal_web_1
9247a2edf8c9 getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 0.0.0.0:2100->2100/tcp, :::2100->2100/tcp bitcoin_app_proxy_1
40322d671ac5 lncm/tor:0.4.7.8 “tor” 25 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp electrs_tor_server_1
dee7ef9e5a79 lncm/tor:0.4.7.8 “tor” 25 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp ride-the-lightning_tor_server_1
59cd8054be8a shahanafarooqui/rtl:0.12.1 “/data/entrypoint.sh” 25 minutes ago Up 25 minutes 3000/tcp ride-the-lightning_web_1
71f26fa18cb4 getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 0.0.0.0:3001->3001/tcp, :::3001->3001/tcp ride-the-lightning_app_proxy_1
314c7f5108da getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 0.0.0.0:2102->2102/tcp, :::2102->2102/tcp electrs_app_proxy_1
6beb3f86b07b getumbrel/electrs:v0.9.4 “electrs” 25 minutes ago Up 25 minutes 4224/tcp, 0.0.0.0:50001->50001/tcp, :::50001->50001/tcp electrs_electrs_1
361413c5df19 getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 0.0.0.0:3008->3008/tcp, :::3008->3008/tcp bluewallet_app_proxy_1
ae78a5bf035e redis:6.2.2-buster “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 6379/tcp bluewallet_redis_1
ea055fe23608 boltz/boltz-lnd:1.2.6 “boltzd --lnd.host=”…" 25 minutes ago Up 25 minutes 9002-9003/tcp ride-the-lightning_boltz_1
c209e9b3acb3 linuxserver/transmission:version-3.00-r5 “/init” 25 minutes ago Up 25 minutes 9091/tcp, 0.0.0.0:51413->51413/tcp, 0.0.0.0:51413->51413/udp, :::51413->51413/tcp, :::51413->51413/udp transmission_server_1
7f69e4092356 getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 25 minutes ago Up 25 minutes 0.0.0.0:3800->3800/tcp, :::3800->3800/tcp snowflake_app_proxy_1
97bedbccaeac lncm/tor:0.4.7.8 “tor” 25 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp bluewallet_tor_server_1
454647bfed1c mempool/frontend:v2.4.0 “/patch/entrypoint.s…” 26 minutes ago Up 25 minutes 80/tcp mempool_web_1
3b8bfd5fdfd4 lncm/tor:0.4.7.8 “tor” 26 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp snowflake_tor_server_1
996e50a85310 getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 0.0.0.0:3006->3006/tcp, :::3006->3006/tcp mempool_app_proxy_1
79c0b9050da5 lncm/tor:0.4.7.8 “tor” 26 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp lightning_tor_server_1
dde8e5aba377 getumbrel/gotty:v1.0.1 “gotty --port 3800 -…” 26 minutes ago Up 25 minutes 8080/tcp snowflake_web_1
eae4ef17adcb lncm/tor:0.4.7.8 “tor” 26 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp transmission_tor_server_1
675234e1c99f getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 0.0.0.0:2101->2101/tcp, :::2101->2101/tcp lightning_app_proxy_1
df803773badd getumbrel/snowflake:v2.1.0 “/bin/proxy -log /da…” 26 minutes ago Up 25 minutes snowflake_proxy_1
8196a4f947cf mempool/backend:v2.4.0 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 8999/tcp mempool_api_1
3931ef61cd6d getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 0.0.0.0:9091->9091/tcp, :::9091->9091/tcp transmission_app_proxy_1
da62c7f0527b getumbrel/umbrel-lightning:v1.0.4 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 3006/tcp lightning_app_1
476170eac65e mariadb:10.5.12 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 3306/tcp mempool_mariadb_1
ac7532510295 lightninglabs/lnd:v0.15.1-beta “lnd --listen=0.0.0.…” 26 minutes ago Up 25 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:9735->9735/tcp, :::9735->9735/tcp, 0.0.0.0:10009->10009/tcp, :::10009->10009/tcp lightning_lnd_1
6767702e4039 lncm/tor:0.4.7.8 “tor” 26 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp mempool_tor_server_1
99abfb34bb4f apotdevin/thunderhub:v0.13.15 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 3000/tcp thunderhub_web_1
d281707422ef lncm/tor:0.4.7.8 “tor” 26 minutes ago Up 25 minutes 9050-9051/tcp, 29050-29051/tcp thunderhub_tor_server_1
33bb6b8d9a32 getumbrel/app-proxy:v0.5.1 “docker-entrypoint.s…” 26 minutes ago Up 25 minutes 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp thunderhub_app_proxy_1
e3d34a33a069 nginx:1.17.8 “nginx -g 'daemon of…” 26 minutes ago Up 26 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp nginx
37786f4b41f5 getumbrel/manager:v0.5.0 “docker-entrypoint.s…” 26 minutes ago Up 26 minutes 3006/tcp manager
b7baf59e48e6 getumbrel/dashboard:v0.5.4 “docker-entrypoint.s…” 26 minutes ago Up 26 minutes 3004/tcp dashboard
26423e80523e lncm/tor:0.4.7.8 “tor” 26 minutes ago Up 26 minutes 9051/tcp, 127.0.0.1:9050->9050/tcp, 29050-29051/tcp tor
dd6c08bf7550 getumbrel/auth-server:v0.5.1 “docker-entrypoint.s…” 26 minutes ago Up 26 minutes 0.0.0.0:2000->2000/tcp, :::2000->2000/tcp auth
umbrel@umbrel:~ $ docker ps | grep lnd
bb4d6584e3af bluewalletorganization/lndhub:v1.4.1 “docker-entrypoint.s…” 25 minutes ago Up 22 minutes 3000/tcp bluewallet_lndhub_1
ea055fe23608 boltz/boltz-lnd:1.2.6 “boltzd --lnd.host=”…" 26 minutes ago Up 25 minutes 9002-9003/tcp ride-the-lightning_boltz_1
ac7532510295 lightninglabs/lnd:v0.15.1-beta “lnd --listen=0.0.0.…” 26 minutes ago Up 25 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:9735->9735/tcp, :::9735->9735/tcp, 0.0.0.0:10009->10009/tcp, :::10009->10009/tcp lightning_lnd_1
--------------- lnd.conf:
umbrel@umbrel:~ $ cat /home/umbrel/umbrel/app-data/lightning/data/lnd/lnd.conf
Automatically reset with every Umbrel update!
[Application Options]
listen=0.0.0.0:9735
rpclisten=0.0.0.0:10009
restlisten=0.0.0.0:8080
Allow push payments
accept-keysend=true
tlsextraip=10.21.21.9
tlsextradomain=umbrel.local
tlsautorefresh=1
Do not include IPs in the RPC TLS certificate
#################### tlsdisableautofill=1
#################### feeurl=
#-------------- From https://github.com/alexbosworth/run-lnd
Mark unpayable, unpaid invoices as deleted
Removed to stabilze >.13------------------ gc-canceled-invoices-on-startup=1
Removed to stabilze >.13------------------ gc-canceled-invoices-on-the-fly=1
#------------------------------ added June 10, 2021 during 0.3.12
alias=PRIVATE
color=#fff700
maxpendingchannels=199
minchansize=3000000
[protocol]
protocol.wumbo-channels=true
[Bitcoind]
bitcoind.rpchost=10.21.21.8
bitcoind.rpcuser=umbrel
bitcoind.rpcpass=PRIVATE
ZMQ listeners
bitcoind.zmqpubrawblock=tcp://10.21.21.8:28332
bitcoind.zmqpubrawtx=tcp://10.21.21.8:28333
[Bitcoin]
bitcoin.active=1
bitcoin.mainnet=1
Default to neutrino as the node is
automatically switched to bitcoind once
IBD is complete
bitcoin.node=bitcoind
bitcoin.defaultchanconfs=2
[neutrino]
Testnet neutrino peers that are automatically
uncommented if Umbrel is configured for testnet
neutrino.addpeer=testnet1-btcd.zaphq.io
neutrino.addpeer=testnet2-btcd.zaphq.io
[tor]
tor.active=1
tor.control=10.21.21.11:29051
tor.socks=10.21.21.11:9050
tor.targetipaddress=10.21.21.9
tor.password=PRIVATE
tor.v3=1
IMPORTANT (to me):
I have significant funds in the node wallet (both on-chain and LN) so I don’t want to punt and restart from scratch by erasing the SSD. I do have a backup of my (42) channels.
REQUEST:
Any ideas or solutions?