How to reduce size of channel.db file for LND

Guides
1

DISCLAIMER: This guide is for advanced node operators, NOT new nodes or small personal nodes. Is more for those nodes that are doing a lot of routing every day, closing / opening a lot of channels, merchants with a lot of generated invoices, failed invoices, with a lot of traffic generated.

So if you are just a noob starting a fresh node now, IGNORE this guide.

To explain a bit what is about. The channel.db file located in /umbrel/lnd/data/graph/mainnet/ in time it grows bigger and bigger until is quite hard to manage it and file corruption can be most probable to a larger file than a smaller one. Also this file is loaded in memory, so for smaller it is, less memory in use will be. It affect also the response time of your node for routing payments.

So, in conclusion it is quite important to keep this file as lower is possible.
Why I did this test? I was reading this post on Github about the aspect of dealing with large db file and slowliness of a node.

Now I am glad that I did it and I would like to share it with you. Those that are seeing lately that you node is acting weird and is slow, worth doing this. It is safe, works easy and can save you from future trouble.

PROCEEDING:

  1. Stop your Umbrel node - SSH into your node and run
cd umbrel
sudo ./scripts/stop
  1. Edit lnd.conf file
sudo nano ~/umbrel/lnd/lnd.conf

Add the following line and section at the end:

[bolt]
db.bolt.auto-compact=true
db.bolt.auto-compact-min-age=0

In the section [Application Options] add the following lines too, for a better cleanup:

# Mark unpayable, unpaid invoices as deleted
gc-canceled-invoices-on-startup=1
gc-canceled-invoices-on-the-fly=1
# Avoid historical graph data sync
ignore-historical-gossip-filters=1
# Avoid slow startup time
sync-freelist=1
# Avoid high startup overhead
stagger-initial-reconnect=1
# Avoid auto force close channels
payments-expiration-grace-period=30m
  1. Check the size of actual channel.db file:
cd ./umbrel/lnd/data/graph/mainnet/
sudo du -ah

It will show you the files in that folder and size in MB of each. You can use the same command after the compacting process to see the difference. Also in lnd logs you can see it mentioned of how much was reduced in size.

  1. Start your node normally: sudo ./scripts/start

  2. WAIT! In this moment you need patience, a lot of patience. Don’t do anything stupid, do not restart the node or unplug it. It is normal that is taking more time to start.
    What is doing? First step, is catching up the bitcoin blocks, could takes 5-15min, then is making a copy of channel.db file and start compacting it.
    Check the bitcoin status with (login SSH into your node):

cd umbrel
docker-compose logs bitcoin
docker-compose logs lnd

See when the compaction is finished you will see a line where it says the initial size and new size of the file.
In my case, from 2GB file, went to 500MB in 2h:30m waiting.

DONE! Now your node is ready and should be more faster in response and routing.

6 Likes
2

Thank you! Was looking for this a while back never found decent instructions.

At what point is this necessary? I guess it depends on hardware and I really should move from my Rasp Pi 4 but…

Right now my channel.db is 475M.

Was your node down for 2.5 hrs whilst compacting? I’d rather do more frequent and quicker compaction than wait till it gets crazy.

3

Nothing to worry at this tiny level. From 2-3GB maybe you should start compacting.

yes, once you see it reaches 2-3GB you can activate it in lnd.conf and make the first compaction. Then at each restart it will do it again, so will be quicker next times when you restart due to updates or maintenance.
Example:
1st time compact from 2GB to 400MB
2nd time, after let’s say 30-60 days will compact from 1.2 GB to 500MB
3rd time after another 60 days will compact from 2.2GB to 800MB
and so on, keeping a low level of this file.

But don’t be too stressed out under 4-5GB.

1 Like
4

Thanks! hugely helpful. I think I’m going to move my node off the Pi and onto a more substantial repurposed Lenovo laptop with 16GB of RAM and much better specs than the Pi sooner rather than later.

Only now I saw the answer you gave to quicker startup so I’m going to add those options to the lnd.conf file now.

5

hmmm I would not do that.
If you really want to move from Pi, I suggest a better solution as a NUC/Barebone/small desktop

A laptop always will have the weak point in getting hot with such intense I/O on the drive. So will be even worse than a Pi…

Using umbrel in a laptop I would recommend only for testing, playing around, learning but not to be a “stable node”

6

Interesting… I’m using the laptop now as another Ubuntu machine which actually watches Hive blockchain and LND (reading invoices via a subscription to my Umbrel). It’s been very good and the battery backup (even though it’s battery is pretty old) means that it lives through occasional power wobbles.

7

If you trust it as being a strong laptop and not having a heat level that can compromise the drive… then go ahead. But I have to warn you: many users using a laptop, had problems.

Also take in consideration a fresh install with Debian OS not Ubuntu. Debian is more stable and have a strong recovery system of files at startup (in case you had power issues). BTW UmbrelOS is aactually a debianOS adapted to umbrel…

1 Like
8

@DarthCoin this grace-period setting helps with channel size or is this a good hygiene settings that you should have anyways?

9

Yes, is more hygiene.

1 Like
11

@DarthCoin are those lines kept when LND (or umbrel) is updated? or is it good practice to do it every now and then?

Also, it’s possible to add a cronjob for it so it can be run in the background automagically? or once the lnd.conf is edited it will run autonomously thanks to db.bolt.auto-compact=true?

12

Yes, after umbrel 0.3.1 lnd.conf is not reset to default anymore.

13

I have tried this just as an exercise and to have things in place if I ever need to use it. My channel.db went from 577MB to 340MB so this was definitely a good exercise. Thanks for the tutorial.

Next up would probably be a cronjob to do this every once in a while. In my case once every 2 months or so would be plenty.

14

No, no need to do it in cronjob. Don’t complicate things more than is necessary.
Just by adding that line in lnd.conf file is enough and at each restart of node (when you update or do maintenance) it will compact it.
Also if your db file is less than 2GB is nothing to worry.

1 Like
15

Ah thanks for explaining :100:

16

I’m in desperate need of doing this, thanks for posting! One issue I’m having, I cannot find my config file after ssh into my node. I got new internet and now have to use my IP address to ssh into my node instead of umbrel@umbrel.local like I used before. Now I use ssh -t umbrel@[IP Address] and then enter my password. Assuming I’m missing something easy here, how do I find my config file?

17

Works perfectly. From 4.3G to 1.6GB in less than 10 minutes.

18

Hi @DarthCoin is there any update to this guide? I can see that the directory structure has changed since you wrote it.
I can see a lnd.conf at ~/umbrel/app-data/lightning/data/lnd/lnd.conf
And I can see a channel.db inside ~/umbrel/app-data/lightning/data/lnd/data/graph/mainnet/

Nevertheless, adding your configs to the lnd.conf doesn’t make a difference, even after reboot. Any idea what could be wrong? I can see another umbrel-lnd.conf at ~/umbrel/app-data/lightning/data/lnd/umbrel-lnd.conf that has auto compacting disabled. Maybe this one overrides the manual config?

1 Like
19

also would be curious for an update here. I added the lines to my config file but nothing happened after restart, DB stayed the same size

1 Like
20

Ok, I edited the following file, rebooted, and then the compaction happened:
~/umbrel/app-data/lightning/data/lnd/umbrel-lnd.conf

Nevertheless this file is auto generated and not supposed to be manually edited. That’s why it gets overridden after each reboot, so you will have to edit it again next time you want compaction to happen. It makes sense create a backup after you edited it with your values.

Again, I would appreciate if there is proper way to do this on Umbrel v0.5.x

1 Like
21

I found the solution for the current Umbrel (v0.5.4). It is actually really convenient.

Open the Lightning Node app and click on “…” top right → Advanced Settings → Optimization.

1 Like