HotShells WSP: Architecting a New Era — The Technical Blueprint Behind the Redesign

The Genesis: Why Rebuild from Ground Zero

Every platform reaches an inflection point where iterative patches become technical debt. For HotShells WSP, that moment arrived when the legacy stack — a patchwork of shared hosting, manual deployments, and fragile configuration management — could no longer support the velocity and reliability demands of a modern web workspace. The decision wasn't merely to redesign; it was to re-architect from first principles.

The new HotShells WSP (hotshells.net) emerges as a declaration: infrastructure should be code, deployments should be invisible, and developers should never wrestle with the platform they build upon.

Infrastructure as Immutable Code

Bare-Metal VPS Fleet with Terraform

We abandoned managed PaaS in favor of a curated fleet of bare-metal VPS instances across multiple geographic regions. Every server is provisioned declaratively through Terraform, ensuring that infrastructure state lives in version control — auditable, reviewable, and reproducible.

module "hotshells_node" {
  source  = "./modules/compute-node"
  count   = var.node_count
  region  = var.regions[count.index]
  
  specs = {
    cpu_cores    = 8
    memory_gb    = 32
    storage_gb   = 500
    network_gbps = 10
  }
  
  tags = {
    Project     = "hotshells-wsp"
    Environment = "production"
    Role        = "compute"
    ManagedBy   = "terraform"
  }
}

Container Orchestration: Nomad Over Kubernetes

After evaluating Kubernetes, k3s, and Nomad, we chose HashiCorp Nomad for its operational simplicity and native Consul integration. The cluster runs in a single binary per node — no control plane complexity, no etcd quorum concerns, and first-class support for non-containerized workloads (systemd, raw exec, QEMU).

Service discovery, health checking, and mTLS encryption are handled by Consul Connect, eliminating the need for a separate service mesh. Traefik serves as the edge router, automatically discovering services via Consul catalog and issuing Let's Encrypt certificates on-demand.

Zero-Downtime Deployment Pipeline

GitOps with Argo CD and Buildpacks

Every merge to main triggers a Cloud Native Buildpack build — no Dockerfiles to maintain, no base image drift. The resulting OCI images are signed with Cosign and pushed to our private Harbor registry with SBOM attestation.

Argo CD watches the GitOps repository and synchronizes Nomad job specifications declaratively. Rollbacks are a single git revert away. Canary deployments route 5% of traffic to new versions via Traefik weighted routing, with automated promotion based on SLO burn-rate alerts.

# .github/workflows/deploy.yml
name: Deploy to Production
on:
  push:
    branches: [main]
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build with Pack
        run: pack build hotshells/app --builder paketobuildpacks/builder:base
      - name: Sign and Push
        run: |
          cosign sign --yes hotshells/app:$GITHUB_SHA
          crane push hotshells/app:$GITHUB_SHA harbor.hotshells.net/app:$GITHUB_SHA
      - name: Update GitOps Manifest
        run: |
          yq -i '.spec.template.spec.taskGroups[0].tasks[0].config.image = "harbor.hotshells.net/app:'$GITHUB_SHA'"' gitops/nomad/hotshells.nomad
          git commit -am "chore: deploy $GITHUB_SHA"
          git push

Observability: The Nervous System

Unified Telemetry Stack

Metrics, logs, and traces converge in a single stack: VictoriaMetrics for long-term metric storage (10x compression over Prometheus), Loki for log aggregation with structured JSON parsing, and Tempo for distributed tracing. Grafana dashboards are code-generated via Jsonnet, ensuring every service exposes a standardized RED (Rate, Errors, Duration) dashboard automatically.

Alerting follows the "page only on symptoms, not causes" philosophy. SLO-based alerts fire only when error budgets burn faster than acceptable — eliminating noise while catching genuine user impact.

"True craft is not merely the assembly of components, but the patient discipline of listening to what the system requires."

— Hesham Kholef

Security by Default

Zero-Trust Network Fabric

Every service communicates over mTLS via Consul Connect. No service trusts the network; identity is cryptographically verified. External access terminates at Traefik with WAF rules (Coraza + OWASP CRS), rate limiting, and geographic blocking for administrative endpoints.

Secrets never touch disk or environment variables. HashiCorp Vault injects secrets at runtime via CSI driver, with dynamic database credentials rotated every 4 hours. SSH access to nodes is eliminated entirely — all management flows through SSM Session Manager with MFA.

Performance at the Edge

Global Anycast with Cloudflare Workers

Static assets and API responses are cached at 300+ Cloudflare PoPs via Workers KV and R2. Dynamic HTML streams from edge workers using HTMLRewriter for personalized fragments — achieving sub-100ms TTFB globally without origin round-trips for cached content.

Core Web Vitals are monitored synthetically via Checkly and real-user monitoring (RUM) via web-vitals library feeding into VictoriaMetrics. The budget: LCP < 1.2s, INP < 200ms, CLS < 0.05 — enforced as CI gates.

The Developer Experience: Local-First, Cloud-Native

Devcontainers as the Source of Truth

Every repository includes a .devcontainer specification. Developers spin up identical environments in VS Code, GitHub Codespaces, or locally via devcontainer up — same toolchain, same dependencies, same runtime as production. No "works on my machine" discrepancies.

Local development mirrors production via Telepresence intercepts: code runs locally but receives live production traffic, with full access to cluster services (databases, queues, caches) via encrypted tunnel.

Conclusion: Infrastructure as a Competitive Advantage

The new HotShells WSP is not merely a redesign — it is a redefinition of what a self-hosted web platform can be. By treating infrastructure as a first-class product, we've built a foundation that compounds: every automation investment pays dividends in velocity, every observability improvement reduces MTTR, every security default eliminates an entire class of incidents.

This is the platform we wished existed. Now it does. Welcome to the new era of HotShells WSP.

Explore the platform: HotShells WSP .