Younes El HamriData engineering · MoroccoAvailable 5 days/week

Multi-service ingestion and processing stack on a single host

Client
Own infrastructure
Sector
Platform
Period
2025
Stack
Docker Compose, Postgres, Python, Cloudflare Tunnel
Public inbound ports20closed
Hosts21consolidated
Problem

A scraping and dashboard app and a media processing engine needed to run together, share a database, and be reachable over a domain without exposing the host.

What I did

Consolidated both onto one host with Docker Compose, an internal-only network, health-gated startup ordering, and an outbound tunnel instead of open inbound ports.

The choices worth explaining

No published ports. Containers talk over an internal Docker network and are reachable from outside only through an outbound tunnel. Nothing listens on a public interface, so there is no inbound firewall rule to get wrong and no certificate to renew.

Health-gated ordering. The tunnel does not start until the application reports healthy. Without that, a restart briefly routes traffic to a container that is still booting, which surfaces as a confusing gateway error rather than a clear one.

Stateless containers, state on volumes. Every container can be rebuilt and replaced without losing anything, which is what makes a deploy a non-event rather than a maintenance window.

Honest limits

This is one host. It is the right answer at this size, because the operational simplicity is worth more than the redundancy would be. But it is a single point of failure, and the design would need rethinking before it carried anything a client depended on.