MMilunaCloud
Back to all articles
GitOpsCI/CDDevOpsHomelabInfrastructure

Why I Ditched Flux: The Case for GitHub Actions and Tailscale in Lightweight Deployments

Written by Miguel Ángel Luna
Published on

For years, the software industry has repeated the same dogmatic mantra: true GitOps must strictly be Pull-based. If you do not have an in-cluster operator reconciling state against a Git repository every three minutes, you are allegedly deploying like it is 2010.

When I built my high-availability homelab with two mini PCs (as recounted in part one of this series), I embraced full orthodoxy: I deployed Flux CD, encrypted every secret using Mozilla SOPS with Age keys, and set up intricate Kustomization and HelmRelease manifests for every workload.

On paper, the promise sounded ideal: declarative immutability, complete auditability, and zero production credentials ever leaving the cluster perimeter.

In practice, it devolved into an operational nightmare. Every Sunday afternoon, I found myself untangling broken reconciliation loops because an operator ran out of memory, or dissecting cryptic in-cluster secret synchronization failures following a node restart.

This article examines why I decided to kill Flux in my infrastructure, how I redesigned deployments around a deterministic Push model using GitHub Actions and Tailscale, and why for small-to-medium infrastructures, the dogma of Pull-based GitOps often turns into an accidental complexity trap.

Technical infographic: Homelab GitOps - Pull with Flux CD vs Push with GitHub Actions and Tailscale

The Pull-Based GitOps Dogma and Where It Breaks Down

The Pull model (championed by industry giants like Flux CD and ArgoCD) was conceived to solve very specific enterprise challenges:

  • Large corporations managing hundreds of Kubernetes clusters behind impenetrable enterprise firewalls where opening inbound ports is strictly prohibited.
  • Large organizations where application engineers must not have direct network access to production infrastructure.
  • Multi-tenant environments where continuous local drift detection is a regulatory requirement.
[ Traditional Pull GitOps ]
Git Repository <────── Continuous Polling (every 3 min) ────── In-Cluster Operator (Flux/Argo)
                                                                        │
                                                                        ▼
                                                               Applies to API Server

However, transplanting this architecture to a two-node homelab or a modest infrastructure imposes steep hidden taxes rarely discussed in conference keynotes:

1. The memory tax of background operators

Running three or four Flux controllers (source-controller, kustomize-controller, helm-controller, notification-controller) continuously burns between 400 MB and 800 MB of baseline RAM. On resource-constrained hardware, that footprint is an unjustifiable waste for an environment where deployments occur only a few times a week.

2. The in-cluster secret operator labyrinth

To avoid committing plaintext secrets to Git, the standard Pull workflow encrypts sensitive values with SOPS. But the operational friction didn’t stem from SOPS itself (which is an exceptional Age-based encryption tool), but rather from running an in-cluster daemon decrypting manifests on every reconciliation loop:

  1. Keeping controllers running permanently inside the cluster to intercept and decrypt resources.
  2. If an Age private key fell out of sync after node recreation, or if a single secret decryption failed, Flux’s reconciler entered infinite error loops, halting deployments for completely unrelated services.
  3. The entire cluster seized up due to the fragility of an in-cluster decryption chain.

3. Asynchronous and blind feedback loops

In a Pull model, when you execute a git push, your CI pipeline turns green almost immediately because its only duty is verifying that the commit landed on GitHub. You have no immediate idea whether the container actually launched successfully in production.

You are forced to open terminal sessions, inspect operator logs, or wait minutes for the next reconciliation cycle. If the image fails due to a CrashLoopBackOff or a malformed environment variable, the failure happens in silence inside the cluster.

The Pragmatic Alternative: Deterministic Push via Tailscale

When migrating my homelab to HashiCorp Nomad, I re-evaluated the deployment lifecycle from first principles: what if we removed all in-cluster operators entirely and let the continuous integration pipeline push deployments deterministically?

The traditional objection to Push has always been security: “We don’t want to expose our orchestrator’s API to the public internet or store long-lived static credentials on GitHub.”

This is where Tailscale changes the equation.

[ Git Push to main ] 
        │
        ▼
[ GitHub Actions Runner ]
        │  1. Ephemeral auth via Tailscale (OAuth tag:ci)
        │  2. Zero application secrets in the runner
        ▼
[ Private Tailscale Mesh ]
        │  3. make nomad-jobs-apply against private API (:4646)
        ▼
[ Nomad Cluster (:4646) ] ──> [ Jobs scheduled consuming Nomad Variables ]

How the ephemeral connection works

  1. The GitHub Actions runner requires no static IPs and no permanent certificates.
  2. Using the official Tailscale GitHub Action and OAuth credentials, the runner joins my private virtual network (tailnet) as a single-use ephemeral node.
  3. It is assigned a strictly scoped tag (tag:ci).
  4. Tailscale ACLs restrict tag:ci exclusively to Nomad’s private API port (:4646). It cannot communicate with any other machine or internal service.
  5. As soon as the GitHub Actions job finishes, the ephemeral node is destroyed and leaves no lingering access.

Pragmatic Secrets: SOPS + Nomad Variables Instead of Operators or CI Sprawl

Secret management in CI/CD often swings between two painful extremes:

  1. In-cluster over-engineering: Running a heavy HashiCorp Vault cluster (with unsealing rituals, intricate ACL policies, and memory overhead) or maintaining in-cluster decryption daemons (Flux/SealedSecrets) continuously decrypting manifests on every reconciliation cycle.
  2. CI secret sprawl: Dumping dozens of database passwords and production tokens into GitHub Secrets, and piping an endless wall of -var="secret=${...}" parameters through every deployment run. This entangles sensitive config with CI workflows, risks exposing secrets in runner logs, and requires re-triggering pipelines just to rotate an environment variable.

In sr-infra, we adopted a far cleaner, professional middle ground: safe versioning with SOPS in Git and decoupled synchronization into native Nomad Variables.

1. Git as the single source of truth (Encrypted with SOPS & Age)

Application secrets live in dedicated environment files under nomad/secrets/*.enc.env. They are encrypted with Age keys via Mozilla SOPS, making them completely safe to version-control in Git. Each file declares its destination metadata in header comments:

# NOMAD_NAMESPACE=tooling
# NOMAD_VAR_PATH=nomad/jobs/tooling/postgres-backup
DB_HOST="127.0.0.1"
DB_USER="postgres"
DB_PASSWORD="my_secure_password"

2. Decoupled in-memory synchronization

Secrets are never decrypted on every commit or by in-cluster daemons. They are synchronized once (or whenever updated) directly from the developer workstation using an automated script (scripts/nomad_secrets_sync.sh or make nomad-secrets-sync):

# Decrypts in-memory and uploads directly into the Nomad Variables store
make nomad-secrets-sync

The script runs sops -d entirely in memory and executes nomad var put -force against the cluster API. Plaintext secrets never touch host disks or cloud CI runners.

3. Native consumption in Nomad Jobs (template)

Nomad job specifications consume these variables natively using built-in template blocks:

task "backup" {
  template {
    destination = "secrets/backup.env"
    env         = true
    data        = <<EOF
{{ with nomadVar "nomad/jobs/tooling/postgres-backup" }}
DB_HOST="{{ .db_host }}"
DB_USER="{{ .db_user }}"
DB_PASSWORD="{{ .db_password }}"
{{ end }}
EOF
  }
}

Nomad renders these values securely into the container’s isolated memory at task startup.

4. Least privilege in CI/CD and frictionless rotation

  • GitHub Actions never touches application secrets: GitHub Secrets only holds ephemeral Tailscale credentials (TS_OAUTH_CLIENT_ID and TS_OAUTH_SECRET). The CI pipeline only handles job definitions, never business credentials or database passwords.
  • Decoupled rotation: Need to rotate a token or database password? You don’t make a Git commit and you don’t touch GitHub Actions. You edit the encrypted secret with sops nomad/secrets/..., run make nomad-secrets-sync, and Nomad automatically updates the variable store in memory, gracefully reloading or restarting affected tasks.

The Pipeline in Action: .github/workflows/nomad.yml

Below is the production-tested workflow used to validate, plan, and deploy jobs onto Nomad deterministically over Tailscale:

# .github/workflows/nomad.yml
name: Nomad CI/CD

on:
  pull_request:
    branches: [main]
    paths:
      - 'nomad/jobs/**'
  push:
    branches: [main]
    paths:
      - 'nomad/jobs/**'

jobs:
  validate:
    name: "Validate Nomad Jobs"
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Setup Nomad CLI
        uses: hashicorp/setup-nomad@v2

      - name: Validate Job Specifications
        run: make nomad-jobs-verify

  deploy:
    name: "Apply Nomad Jobs"
    needs: validate
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Setup Nomad CLI
        uses: hashicorp/setup-nomad@v2

      - name: Connect to Tailscale
        uses: tailscale/github-action@v2
        with:
          oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
          oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
          tags: tag:ci

      - name: Apply Nomad Jobs to Cluster
        env:
          NOMAD_ADDR: http://travolta.your-tailnet.ts.net:4646
        run: make nomad-jobs-apply

Real-World Operational Gains

After several months operating this pipeline, the improvements over the Flux era are indisputable:

Dimension Previous Approach (Flux CD Pull) Current Approach (GitHub Actions Push + Nomad)
Idle Memory Consumption ~600 MB burned by in-cluster controllers 0 MB (execution offloaded entirely outside cluster)
Secret Management SOPS + Age keys + in-cluster decryption pods SOPS in Git + decoupled in-memory Nomad Variables
Secrets in CI/CD In-cluster private keys needed for decryption Zero business secrets in GitHub (only Tailscale OAuth)
Feedback Latency 3–8 minutes (asynchronous polling) < 25 seconds with live execution logs
Public Attack Surface Zero public ports exposed Zero public ports exposed (routed over Tailscale mesh)
Disaster Recovery Complex key and controller restores Instant: a single make nomad-jobs-apply reschedules jobs

The Fine Print: What Trade-offs Are We Making?

In the spirit of technical honesty, this Push architecture involves compromises worth noting:

  1. Loss of continuous in-cluster drift detection:
    If someone manually alters or terminates a job via nomad job stop directly on the server, GitHub Actions will not detect or correct that drift until the next commit is pushed (unless you configure a scheduled workflow cron). In a small team or personal infrastructure, this is rarely an issue, but in larger organizations, unmonitored drift can accumulate.

  2. Dependency on GitHub Actions availability:
    If GitHub experiences an outage, automated deployments cannot proceed through the pipeline. However, because you retain direct Tailscale access to your cluster, you can still deploy manually in seconds from your local machine using make nomad-jobs-apply.

  3. Trusting Tailscale security boundaries:
    We delegate network tunneling to Tailscale. However, by decoupling secrets through SOPS and Nomad Variables, GitHub never stores production credentials or database passwords; it only holds ephemeral Tailscale OAuth keys (tag:ci restricted strictly to port :4646), radically shrinking the blast radius if CI is ever breached.

Conclusion: Choose Tools Sized for Your Problem

Pull-based GitOps with Flux or ArgoCD is an exceptional architecture when managing hundreds of clusters across distributed enterprise environments where internal networks cannot be reached directly.

However, in small setups, edge nodes, or homelabs, forcing a Pull model is often an exercise in accidental complexity. It introduces unnecessary abstraction layers, memory-hungry background daemons, and fragile secret chains to solve problems you simply do not have.

Pairing a lightweight orchestrator like Nomad, a secure private mesh like Tailscale, and the simplicity of GitHub Actions delivers the best of both worlds: zero open firewall ports, zero wasted in-cluster memory, and instant, actionable feedback on every commit.