This is the second installment in a three-part series.
While the initial setup worked, it lacked the well deserved ergonomics and developer experience. Using portainer was fun until you wanted to look at the files and run Linux commands directly on the host. No doubt portainer has its perks but for the purpose of my lab it didn't stand when compared to native SSH
Move away from Portainer
The primary change was that I moved from portainer to managing all the containers myself using docker compose, the flow was as follows for a new application i wanted to add is the following
Why?
Portainer is great if you want to run single container application, while most of my persistent containers were single container application i had started experimentation with multi container application/services. Even though portainer offered the docker compose edition I was not a fan of it. I preferred using SSH and making changes through it. It allowed greater control over my containers and most importantly the host.
Retirement of old & addition of new services
With change in overall architecture I retired few services including uptimekuma, nginx proxy manager (replaced by traefik), pihole, gitea and minecraft server. Most was deemed no longer needed. Though this removed most of the services I ran initially quite a few services were later added in the lab, such as
Actual Budget – After trying out different budgeting solution, I settled on this. It features envelop budgeting, comprehensive dashboards, sync and mainly offline usage (through PWA). It also offers an desktop app though the website is as powerful as the app.
Immich – This is an alternative to google photos, I recently started maintaining a personal photo and video library and this serves as the perfect choice for that. It also features person detection and categorization, metadata extraction, grouping by location and many others, serving as the ideal choice for self hosted photo-video library
PaperlessNgx – This is the best document aggregation platform that I have used, and saved a lot of time also. With inbuilt OCR, conversion to PDF and comprehensive search and filtration support this makes document management, resurfacing and most importantly retrieval much easier
Reactive Resume – This is the self hosted resume builder with inbuilt MCP integration, allowing quick changes and tailoring of resume by AI.
Zerobyte – This serves as the backup solution for the whole server, from all configurations, to backing important files, services data, etc. Zerobyte offers multiple repository options including google drive, S3 compatible storage, and many more making it suitable for my setup.
Discontinued services
These includes very good projects. Although all of these is not currently running as they are not actively required, they served extremely well then I needed them.
Anytype Server – Serves as the self hosted sync server for Anytype, discontinued as their public offering suffices my needs
Audiobookshelf – Serves as complete audio-book solution, discontinued due to lack of usage
Booklore – Serves as complete E-Book management solution, discontinued due to lack of usage
Solidtime – The best self-hosted time management software, discontinued due to lack of usage
Pihole – Add blocker and local DNS, discontinued as I moved to public DNS
Nginx Proxy Manager – A web based Nginx container, featuring web based config editing, discontinued as I moved to Traefik
Uptimekuma – The comprehensive uptime monitor, discontinued as no external service was being monitored, when the server was down the monitor was also down, losing its usefulness
Gitea – The lightweight GitHub like git server, discontinued as it was only for experimentation purposes
Setting up Traefik
Nginx proxy manager was a phenomenal choice to start with, but as the number of services grew it became a tedious task to manually update them one by one. Introduction of Traefik was the next best option for me. It offered many advantages for my use case such as
- —Everything in code
- —Integration with tailscale for SSL certs
- —configuration stays with the container
- —dashboard showing everything
- —Changes done automatically while starting/upgrading the container
Since my whole lab setup was a bunch of isolated compose stacks Traefik couldn't reach them. Since i have moved away from opening a port for each service a common infra network was created. the infra network (infra_default) is just the default network of the compose project, which is added to all the web facing containers in all the application. and set as an external network in all remaining project
This effectively increases the isolation between projects as there are no open ports (every compose project has its dedicated network to communicate between different services) and only the web facing container can be reached by Traefik and subsequently by the user
Though this setup is more complex than expected, the benefits consistently outweighs the cost of implementation as well of usage. A detailed guide on the exact setup that I use will soon be available.
All this modifications to the server was fruitful at the end and resulted in many lifestyle upgrades of my own. Alongside these upgrades, upgrades in the networking was also made that was fruitful in usage of these services which will be discussed in the next part