---
name: infra-ops
description: >
  Canonical conventions for Manohar's self-hosted infrastructure (Hetzner CX32 +
  Dokploy + Tailscale + Forgejo). Use whenever creating or editing a service,
  writing a Dokploy compose file, running SSH ops on the server, deploying via
  Forgejo, or touching networking/UFW. Encodes the script-first workflow, compose
  label requirements, overlay-vs-bridge networking rules, and the deploy loop so
  these directions never need restating.
---

# Infra Ops — house style

## Server
- Host `manohar-ubuntu`: Hetzner CX32 (4 vCPU / 7.6 GB / 75 GB), Ubuntu 24, Docker 29, Helsinki.
- SSH (Tailscale-only; user is always `root`):
  ```
  SSH_AUTH_SOCK=$(launchctl getenv SSH_AUTH_SOCK) ssh -i ~/.ssh/id_ed25519 root@100.75.128.45 'bash -s' < /local/script.sh
  ```
  - Tailscale IP `100.75.128.45` | public IPv4 `77.42.82.225`
  - NEVER use `-t` (no pseudo-TTY). NEVER heredoc over SSH.
  - Tailscale node idle = online, not down. Re-auth prompt is normal: approve, then kill+restart any wedged session.

## Script-first (never deviate)
- Write scripts locally to `~/MyProjects/` via Desktop Commander `write_file` (NOT the sandbox).
- Execute remotely via the ssh pipe above (`'bash -s' < script.sh`).
- Never patch files in place on the server bypassing git.
- Backup-before-change: write a rollback script to `/opt/<service>/` before modifying configs.
- Dead-man's-switch for risky ops: a verify step that proves success before the change is trusted.

## Dokploy compose conventions
Dokploy deploys compose as a **swarm stack**, so Traefik routing needs BOTH label sets:
- container-level `labels:` (docker provider) AND `deploy: labels:` (swarm provider) — mirror them exactly.
- No `container_name:` (swarm assigns names).
- Attach `dokploy-network` (`external: true`) for Traefik ingress.
- Deploy only through the Dokploy UI (not `docker stack deploy` by hand).
- `/etc/dokploy/compose/*/code/` is OVERWRITTEN on every redeploy — never treat it as source of truth.
- Standard Traefik labels (replace SVC / HOST / PORT):
  ```
  traefik.enable=true
  traefik.docker.network=dokploy-network
  traefik.http.routers.SVC.rule=Host(`HOST`)
  traefik.http.routers.SVC.entrypoints=websecure
  traefik.http.routers.SVC.tls.certresolver=letsencrypt
  traefik.http.services.SVC.loadbalancer.server.port=PORT
  ```
- Scaffold to copy: `templates/dokploy-service.compose.yml`

## Networking (the rules that bite)
- `dokploy-network` is a swarm **OVERLAY** → containers on it CANNOT reach the host
  (not `10.0.1.1`, not the Tailscale IP) and cannot cleanly egress to a tailnet peer.
- To reach the host OR a tailnet peer from a container, give it a second **bridge**
  network; its gateway (`172.x.0.1`) is the host, which then routes/masquerades out.
  Precedents: n8n → `172.19.0.1`; tiger-bridge `tiger-net` → `172.18.0.1`; ha-proxy uses this for tailnet egress.
- UFW: `ufw allow` covers bridge subnets (172.x). It does NOT expose docker-published
  ports — those need `ufw-docker allow PORT` (DOCKER-USER chain).
- Always `ufw reload` after rule changes; verify with `iptables -L ufw-user-input -n -v`.

## Deploy loop
- Git-driven services: source in `~/MyProjects/<svc>/`, Forgejo remote `git.manohargupta.com/manohar/<svc>`.
  Push → Forgejo webhook → Dokploy rebuild. No manual server steps.
- infra repo = local `~/MyProjects/deployments/` (remote `manohar/infra`), pushes over HTTPS:443.
  Flat `*.compose.yml` files and per-service subfolders are both fine.
- Manual (non-Dokploy) stacks — Tiger `/opt/tiger/`, LiteLLM, code-server — compose lives in the repo, deployed by hand.

## Working style
- Root cause before fix; state tradeoffs between fix paths.
- One mini-question / understanding check per major topic.
- Explicit risk flag before any change touching security, stability, or data.
- Token-efficient: batch ops, don't re-explain established context.
- Don't redo security hardening (UFW/ufw-docker/fail2ban/SSH) — it's done.