Apps won’t start, I can’t lose my photos. Thank you very much.
=====================
= Umbrel debug info =
Umbrel version
0.5.4
Memory usage
total used free shared buff/cache available
Mem: 11G 1,8G 6,8G 2,0M 3,3G 9,8G
Swap: 4,1G 0B 4,1G
total: 14.8%
immich: 12.2%
system: 2.6%
Memory monitor logs
2023-09-08 20:36:04 Memory monitor running!
2023-09-08 20:48:14 Memory monitor running!
2023-09-08 23:06:51 Memory monitor running!
2023-09-08 23:33:24 Memory monitor running!
2023-09-08 23:37:35 Memory monitor running!
2023-09-08 23:41:09 Memory monitor running!
2023-09-08 23:44:46 Memory monitor running!
2023-09-08 23:51:30 Memory monitor running!
2023-09-09 00:01:13 Memory monitor running!
2023-09-09 00:03:38 Memory monitor running!
Filesystem information
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 469G 452G 0 100% /
/dev/sda2 469G 452G 0 100% /
Karen logs
Stopping dashboard …
Stopping tor_proxy …
Stopping auth …
Stopping tor_proxy … done
Stopping auth … done
Stopping nginx … done
Stopping manager … done
Stopping dashboard … done
Removing nginx …
Removing manager …
Removing dashboard …
Removing tor_proxy …
Removing auth …
Removing nginx … done
Removing manager … done
Removing dashboard … done
Removing tor_proxy … done
Removing auth … done
Removing network umbrel_main_network
karen is running in /home/waldyr/umbrel/events
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
karen is running in /home/waldyr/umbrel/events
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
karen is running in /home/waldyr/umbrel/events
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
karen is running in /home/waldyr/umbrel/events
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
Got signal:
karen is getting triggered!
./karen: line 75: /home/waldyr/umbrel/events/triggers/: Is a directory
karen is running in /home/waldyr/umbrel/events
Got signal: debug
karen is getting triggered!
Docker containers
NAMES STATUS
immich_proxy_1 Restarting (1) 48 seconds ago
immich_microservices_1 Up 25 seconds
immich_server_1 Up 25 seconds
immich_web_1 Up 2 minutes
immich_redis_1 Up 2 minutes
immich_postgres_1 Restarting (1) 48 seconds ago
immich_machine-learning_1 Up 2 minutes
immich_app_proxy_1 Restarting (1) 37 seconds ago
immich_typesense_1 Up 2 minutes
nginx Up 2 minutes
manager Up 2 minutes
tor_proxy Restarting (1) 50 seconds ago
dashboard Up 2 minutes
auth Restarting (1) 37 seconds ago
Umbrel logs
Attaching to manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/system/memory HTTP/1.0” 304 - “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/system/storage HTTP/1.0” 200 216 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/system/info HTTP/1.0” 304 - “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/system/uptime HTTP/1.0” 200 3 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/system/remote-tor-access-status HTTP/1.0” 304 - “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/apps HTTP/1.0” 304 - “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/apps?installed=1 HTTP/1.0” 304 - “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:11 GMT] “GET /v1/system/get-update HTTP/1.0” 304 - “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:14 GMT] “POST /v1/system/debug HTTP/1.0” 200 17 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
manager | ::ffff:10.21.21.2 - - [Sat, 09 Sep 2023 00:06:15 GMT] “GET /v1/system/debug-result HTTP/1.0” 200 23 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188”
manager |
manager | umbrel-manager
Tor Proxy logs
Attaching to tor_proxy
tor_proxy | Sep 09 00:05:25.343 [warn] You have a ControlPort set to accept connections from a non-local address. This means that programs not running on your computer can reconfigure your Tor. That’s pretty bad, since the controller protocol isn’t encrypted! Maybe you should just listen on 127.0.0.1 and use a tool like stunnel or ssh to encrypt remote connections to your control port.
tor_proxy | Sep 09 00:05:25.343 [notice] Opening Socks listener on 10.21.21.11:9050
tor_proxy | Sep 09 00:05:25.344 [notice] Opened Socks listener connection (ready) on 10.21.21.11:9050
tor_proxy | Sep 09 00:05:25.344 [notice] Opening Control listener on 10.21.21.11:29051
tor_proxy | Sep 09 00:05:25.344 [notice] Opened Control listener connection (ready) on 10.21.21.11:29051
tor_proxy | Sep 09 00:05:25.346 [warn] Error creating directory /tmp/.tor: No space left on device
tor_proxy | Sep 09 00:05:25.346 [notice] Closing partially-constructed Socks listener connection (ready) on 10.21.21.11:9050
tor_proxy | Sep 09 00:05:25.346 [notice] Closing partially-constructed Control listener connection (ready) on 10.21.21.11:29051
tor_proxy | Sep 09 00:05:25.346 [warn] Failed to parse/validate config: Couldn’t create private data directory “/tmp/.tor”
tor_proxy | Sep 09 00:05:25.346 [err] Reading config failed–see warnings above.
App logs
immich
Attaching to immich_proxy_1, immich_microservices_1, immich_server_1, immich_web_1, immich_redis_1, immich_postgres_1, immich_machine-learning_1, immich_app_proxy_1, immich_typesense_1
app_proxy_1 | warning Skipping preferred cache folder “/tmp/.yarn-cache-1000” because it is not writable.
app_proxy_1 | warning Skipping preferred cache folder “/tmp/.yarn-cache” because it is not writable.
app_proxy_1 | info Visit yarn run | Yarn for documentation about this command.
app_proxy_1 | error Yarn hasn’t been able to find a cache folder it can use. Please use the explicit --cache-folder option to tell it what location to use, or make one of the preferred locations writable.
app_proxy_1 | yarn run v1.22.19
app_proxy_1 | warning Skipping preferred cache folder “/home/node/.cache/yarn” because it is not writable.
app_proxy_1 | warning Skipping preferred cache folder “/tmp/.yarn-cache-1000” because it is not writable.
app_proxy_1 | warning Skipping preferred cache folder “/tmp/.yarn-cache” because it is not writable.
app_proxy_1 | error Yarn hasn’t been able to find a cache folder it can use. Please use the explicit --cache-folder option to tell it what location to use, or make one of the preferred locations writable.
app_proxy_1 | info Visit yarn run | Yarn for documentation about this command.
microservices_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
microservices_1 | [Nest] 7 - 09/09/2023, 12:06:10 AM ERROR e[38;5;3m[TypeOrmModule] Unable to connect to the database. Retrying (6)…
microservices_1 | Error: getaddrinfo EAI_AGAIN immich_postgres_1
microservices_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
microservices_1 | [Nest] 7 - 09/09/2023, 12:06:13 AM ERROR e[38;5;3m[TypeOrmModule] Unable to connect to the database. Retrying (7)…
microservices_1 | Error: getaddrinfo EAI_AGAIN immich_postgres_1
microservices_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
microservices_1 | [Nest] 7 - 09/09/2023, 12:06:16 AM ERROR e[38;5;3m[TypeOrmModule] Unable to connect to the database. Retrying (8)…
microservices_1 | Error: getaddrinfo EAI_AGAIN immich_postgres_1
microservices_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
postgres_1 | chmod: /var/run/postgresql: Operation not permitted
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2023-09-09 00:04:35.107 UTC [1] FATAL: could not write lock file “postmaster.pid”: No space left on device
postgres_1 | chmod: /var/run/postgresql: Operation not permitted
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2023-09-09 00:05:27.438 UTC [1] FATAL: could not write lock file “postmaster.pid”: No space left on device
proxy_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
proxy_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
proxy_1 | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
proxy_1 | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-set-env-variables.envsh
proxy_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
proxy_1 | 20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/default.conf.template to /etc/nginx/conf.d/default.conf
proxy_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
proxy_1 | /docker-entrypoint.sh: Configuration complete; ready for start up
proxy_1 | 2023/09/09 00:05:26 [emerg] 1#1: mkdir() “/tmp/proxy_temp” failed (28: No space left on device)
proxy_1 | nginx: [emerg] mkdir() “/tmp/proxy_temp” failed (28: No space left on device)
machine-learning_1 | [09/09/23 00:02:22] INFO Initialized request thread pool with 8 threads.
machine-learning_1 | [09/09/23 00:03:34] INFO Starting gunicorn 21.2.0
machine-learning_1 | [09/09/23 00:03:34] INFO Listening at: http://0.0.0.0:3003 (8)
machine-learning_1 | [09/09/23 00:03:34] INFO Using worker: uvicorn.workers.UvicornWorker
machine-learning_1 | [09/09/23 00:03:34] INFO Booting worker with pid: 9
machine-learning_1 | [09/09/23 00:04:04] e[1;7;31mCRITICAL WORKER TIMEOUT (pid:9)
machine-learning_1 | [09/09/23 00:04:04] ERROR Worker (pid:9) was sent code 134!
machine-learning_1 | [09/09/23 00:04:04] INFO Booting worker with pid: 25
machine-learning_1 | [09/09/23 00:04:32] INFO Created in-memory cache with unloading disabled.
machine-learning_1 | [09/09/23 00:04:32] INFO Initialized request thread pool with 8 threads.
server_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
server_1 | [Nest] 7 - 09/09/2023, 12:06:09 AM ERROR e[38;5;3m[TypeOrmModule] Unable to connect to the database. Retrying (6)…
server_1 | Error: getaddrinfo EAI_AGAIN immich_postgres_1
server_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
server_1 | [Nest] 7 - 09/09/2023, 12:06:12 AM ERROR e[38;5;3m[TypeOrmModule] Unable to connect to the database. Retrying (7)…
server_1 | Error: getaddrinfo EAI_AGAIN immich_postgres_1
server_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
server_1 | [Nest] 7 - 09/09/2023, 12:06:15 AM ERROR e[38;5;3m[TypeOrmModule] Unable to connect to the database. Retrying (8)…
server_1 | Error: getaddrinfo EAI_AGAIN immich_postgres_1
server_1 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
typesense_1 | I20230909 00:03:49.694859 354 collection_manager.cpp:1207] Loading collection assets-v8
typesense_1 | I20230909 00:04:26.481678 354 collection_manager.cpp:1313] Loaded 16384 documents from assets-v8 so far.
typesense_1 | I20230909 00:04:26.615137 355 collection_manager.cpp:1313] Loaded 32768 documents from faces-v1 so far.
typesense_1 | I20230909 00:04:50.602133 263 batched_indexer.cpp:284] Running GC for aborted requests, req map size: 0
typesense_1 | I20230909 00:04:59.936703 354 collection_manager.cpp:1313] Loaded 32768 documents from assets-v8 so far.
typesense_1 | I20230909 00:05:05.201756 355 collection_manager.cpp:1313] Loaded 65536 documents from faces-v1 so far.
typesense_1 | I20230909 00:05:29.268067 355 collection_manager.cpp:1324] Indexed 82538/82538 documents into collection faces-v1
typesense_1 | I20230909 00:05:29.268119 355 collection_manager.cpp:228] Loaded 2 collection(s) so far
typesense_1 | I20230909 00:05:34.116178 354 collection_manager.cpp:1313] Loaded 49152 documents from assets-v8 so far.
typesense_1 | I20230909 00:05:51.612941 263 batched_indexer.cpp:284] Running GC for aborted requests, req map size: 0
redis_1 | 1:M 09 Sep 2023 00:03:31.343 * monotonic clock: POSIX clock_gettime
redis_1 | 1:M 09 Sep 2023 00:03:31.345 * Running mode=standalone, port=6379.
redis_1 | 1:M 09 Sep 2023 00:03:31.345 # Server initialized
redis_1 | 1:M 09 Sep 2023 00:03:31.345 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see vm.max_map_count growing steadily when vm.overcommit_memory is 2 · Issue #1328 · jemalloc/jemalloc · GitHub. To fix this issue add ‘vm.overcommit_memory = 1’ to /etc/sysctl.conf and then reboot or run the command ‘sysctl vm.overcommit_memory=1’ for this to take effect.
redis_1 | 1:M 09 Sep 2023 00:03:31.351 * Loading RDB produced by version 6.2.12
redis_1 | 1:M 09 Sep 2023 00:03:31.351 * RDB age 40779 seconds
redis_1 | 1:M 09 Sep 2023 00:03:31.351 * RDB memory usage when created 6.53 Mb
redis_1 | 1:M 09 Sep 2023 00:03:31.385 # Done loading RDB, keys loaded: 51, keys expired: 16.
redis_1 | 1:M 09 Sep 2023 00:03:31.385 * DB loaded from disk: 0.039 seconds
redis_1 | 1:M 09 Sep 2023 00:03:31.385 * Ready to accept connections
web_1 | Listening on 0.0.0.0:3000
web_1 | Listening on 0.0.0.0:3000
==== Result ====
==== END =====
Forwarding the discussion from Telegram to here →
Looks like we have plenty of RAM and are running on Linux, Immich looks like it’s also stuck booting,
Can you attempt a reboot of your Linux machine?
Then as well let me know the version to confirm with the command in terminal:
uname -a
Also, I would recommend refreshing the browser after a reboot, and after a couple minutes see if it then becomes response, if possible let me know if there’s any issue, we can try to go to Manage Apps > Red X on Immich > Try to then reinstall it from App Store,
Let me know the result and we can keep troubleshooting
Linux waldyr 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
If I uninstall the immich app and install it, will I lose the photos?
How do I restart?
The restart command is the following
sudo ~/umbrel/scripts/app restart immich
Actually looking over this- I believe the issue is the storage being 100% used.
You will need to upgrade your SSD or allocate more storage to the instance (in the case of it being a VM), and then we can clone the files from the current one onto your new SSD.
Hope that makes sense
I have done. The application stays the same way. Does not start.
Did you see my last part. Please let me know.
I’m sorry. I’m using it on an Ubuntu Server/Umbrel notebook. How do I add more storage? I have external SDs.
Thank you very much!!! God bless you!!! The problem was that the SSD was full so it wasn’t opening the applications. I cloned it via Clonezilla to a larger SSD and now the applications start again.
1 Like
Great to hear! Have a nice week