Build Environments Standard

Standard scope

Applies to: All HCS-governed repos, across all scopes — build/CI runner selection Domain: build-environments Status: Active

This standard defines how all HCS projects provision, use, and manage build environments. This is the single source of truth for runner and CI environment selection across both GitHub and GitLab. The automation.md standard's "Agent pools" section defers to this document.

---

Runner selection — priority order

Always use the highest-priority available option. Do not jump to cloud runners when a local option works.

Priority	Environment	When to use	Cost
1 — WSL (local VM)	Ubuntu 22.04 on the platform VM	Linux builds, scripts, containers — whenever the work is not Windows-only	Free
2 — HCS Linux runner (ACA Job)	KEDA-scaled GitHub+GitLab runner, rg-hcs-gh-runners-eus2-01	Standard CI/CD: both GitHub Actions and GitLab CI that need a cloud agent	Consumption — near zero
3 — bld-01 (nested Hyper-V)	Windows Server 2025, 10.10.10.10, this host only	Windows-only work: WinPE ISOs, PowerShell modules, ADK builds	Free (shared)
4 — Quick-deploy Windows server (fallback)	Ephemeral Windows VM, same RG + VNet as Tier 2	Tier 3 not available (off-host or bld-01 unusable); Windows-only work only	VM cost while running

Decision rule: if the work does not explicitly require Windows APIs, ADK, or COM objects, it runs on Tier 1 or Tier 2 (Linux). Reserve Tier 3 and 4 for true Windows-only builds.

---

Tier 1 — WSL (Ubuntu 22.04 on the platform VM)

Use directly on the platform VM host. No VM startup, no cost.

# Run a Linux build directly from PowerShell on the host
wsl --distribution Ubuntu-22.04 -- bash -c "cd /mnt/d/git/myrepo && make build"

Works for: bash scripts, make, cargo, go build, docker buildx, npm/pnpm, Python, az CLI.

---

Tier 2 — HCS Linux shared runner (ACA Job + KEDA)

Access details — GitHub Actions

Property	Value
Resource Group	rg-hcs-gh-runners-eus2-01
Container Apps Environment	cae-hcs-gh-runners-eus2-01
Region	eastus2
Job name	caj-hcs-gh-runner-eus2-01
Runner image	myoung34/github-runner:ubuntu-jammy
Labels	self-hosted,linux,ubuntu-22.04,hcs
KV secret	hcs-gh-runner-pat (GitHub PAT)
Infra location	D:\git\platform\.ado\runners\

Access details — GitLab CI

Property	Value
Resource Group	rg-hcs-gh-runners-eus2-01
Container Apps Environment	cae-hcs-gh-runners-eus2-01
Region	eastus2
App name	ca-hcs-gl-runner-eus2-01
Runner image	gitlab/gitlab-runner:ubuntu
Tags	hcs,linux,ubuntu-22.04
KV secret	hcs-gitlab-runner-token (GitLab registration token)
Infra location	D:\git\platform\.ado\runners\

GitHub Actions — using the HCS runner

jobs:
  build:
    runs-on: [self-hosted, linux, ubuntu-22.04, hcs]
    steps:
      - uses: actions/checkout@v4
      - name: Build
        run: make build

GitLab CI — using the HCS runner

build:
  tags:
    - hcs
    - linux
    - ubuntu-22.04
  script:
    - make build

Deploying / updating the runner infrastructure

# Deploy or update the full runner stack (GitHub + GitLab + Windows fallback)
D:\git\platform\.ado\runners\Deploy-Runners.ps1

# Update registration tokens when they rotate
D:\git\platform\.ado\runners\Update-RunnerRegistration.ps1

All registration tokens are in kv-hcs-vault-01. Never inline them in YAML or Bicep. See Key Vault standard.

---

Tier 3 — bld-01 (nested Hyper-V Windows build box)

Access details

Property	Value
VM name	bld-01
OS	Windows Server 2025 Standard (Desktop Experience)
vCPU	4
RAM	Dynamic 2–16 GB
VHDX	D:\Data\VMs\bld-01.vhdx
Switch	beacon-nat (Internal NAT, 10.10.10.0/24)
IP	10.10.10.10 (static)
Credentials	Administrator / kv-hcs-vault-01 → hcs-bld-vm-password
Host only	This specific platform VM (nested Hyper-V)

Power management is required. bld-01 is powered off when idle. Every build script must power it on, run the build, and restore the original power state. Use the standard wrapper below.

When to use Tier 3

• WinPE ISO generation (Windows ADK)
• PowerShell modules that use Windows-only APIs
• .NET Framework (not .NET 8+ Core) builds
• COM/WMI/Windows-specific testing

For anything that works on Linux, use Tier 1 or 2 instead.

---

Tier 4 — Quick-deploy Windows build server (fallback)

When bld-01 is unavailable (off-host session, VHDX corruption, different machine), deploy an ephemeral Windows VM into the same RG and VNet as the Tier 2 runner:

Property	Value
Resource Group	rg-hcs-gh-runners-eus2-01
VNet	same as CAE cae-hcs-gh-runners-eus2-01
OS	Windows Server 2025
Credentials	Administrator / kv-hcs-vault-01 → hcs-winbuild-vm-password
Deploy script	D:\git\platform\.ado\runners\Deploy-WindowsFallback.ps1
Lifecycle	Ephemeral — deallocate or delete after build completes

# Deploy fallback VM and run a build
D:\git\platform\.ado\runners\Deploy-WindowsFallback.ps1 -ScriptFile scripts\build.ps1

---

Why this standard exists

Azure VM run-command (V1 and V2) has critical limitations for build workloads:

• 90-minute hard timeout — any build longer than 90 min is killed
• Extension deadlocks — only one run-command can execute at a time; stuck extensions can block a VM for hours
• No shell access — without RDP or WinRM in the NSG, there is no escape hatch when a command hangs
• Provisioning state coupling — a VM stuck in Updating blocks all ARM operations

Local Hyper-V (Tier 3) eliminates all of these. WSL (Tier 1) eliminates them at zero cost for Linux builds. The shared ACA runner (Tier 2) handles cloud CI without these limitations.

---

Architecture overview

Platform VM (Azure WS2025)
├── WSL (Tier 1) — Ubuntu 22.04, Linux-native builds, no startup cost
├── Hyper-V (Tier 3)
│   └── bld-01  — Windows Server 2025, WinPE/ADK/PS modules
│       NAT switch: beacon-nat  10.10.10.0/24 → host .1, guest .10
└── ─────────────────────────────────────────────────────
Azure: rg-hcs-gh-runners-eus2-01 (eastus2)
├── Container Apps Environment: cae-hcs-gh-runners-eus2-01
│   ├── ACA Job (Tier 2): caj-hcs-gh-runner-eus2-01 (GitHub KEDA)
│   └── ACA App (Tier 2): ca-hcs-gl-runner-eus2-01 (GitLab always-on)
└── Windows VM (Tier 4 — ephemeral, deploy on demand only)

---

Standard build wrapper (Tier 3 — bld-01)

#Requires -Version 7.0
<#
.SYNOPSIS
    Runs a build script on the shared local Hyper-V build VM (bld-01).

.DESCRIPTION
    Implements the HCS build environment standard:
    1. Verifies bld-01 exists (deploys if not)
    2. Records power state
    3. Powers on if needed, waits for WinRM
    4. Runs the provided script block or file via PS Remoting
    5. Restores original power state

.PARAMETER ScriptBlock
    PowerShell script block to execute on the build VM.

.PARAMETER ScriptFile
    Path to a .ps1 file on the HOST to copy and run on the build VM.

.PARAMETER Timeout
    Seconds to wait for WinRM after powering on. Default: 300.

.EXAMPLE
    Invoke-BuildOnVm -ScriptBlock { cd C:\build\myrepo; git pull; .\build.ps1 }

.EXAMPLE
    Invoke-BuildOnVm -ScriptFile .\scripts\build.ps1
#>
[CmdletBinding()]
param(
    [Parameter(ParameterSetName = 'Block', Mandatory)]
    [scriptblock]$ScriptBlock,

    [Parameter(ParameterSetName = 'File', Mandatory)]
    [string]$ScriptFile,

    [int]$Timeout = 300
)

Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'

$VM_NAME    = 'bld-01'
$VM_IP      = '10.10.10.10'
$VM_USER    = 'Administrator'
$KV_NAME    = 'kv-hcs-vault-01'
$KV_SECRET  = 'hcs-bld-vm-password'
$SWITCH     = 'beacon-nat'
$VHDX_PATH  = 'D:\Data\VMs\bld-01.vhdx'
$DEPLOY_SCRIPT = 'D:\git\platform\scripts\build\Deploy-BuildVm.ps1'

function Write-Step { param([string]$Msg) Write-Host "  [bld] $Msg" }

# --- 0. Get credentials from Key Vault ---
Write-Step "Loading build VM credentials from Key Vault..."
$vmPassword = az keyvault secret show --vault-name $KV_NAME --name $KV_SECRET --query value -o tsv
$cred = New-Object System.Management.Automation.PSCredential(
    $VM_USER,
    (ConvertTo-SecureString $vmPassword -AsPlainText -Force)
)

# --- 1. Verify VM exists ---
$vm = Get-VM -Name $VM_NAME -ErrorAction SilentlyContinue
if (-not $vm) {
    Write-Step "bld-01 not found — deploying..."
    if (-not (Test-Path $DEPLOY_SCRIPT)) {
        throw "Deploy script not found: $DEPLOY_SCRIPT. Clone the platform repo first."
    }
    & $DEPLOY_SCRIPT
    $vm = Get-VM -Name $VM_NAME -ErrorAction Stop
}

# --- 2. Record power state ---
$originalState = $vm.State
Write-Step "VM current state: $originalState"

# --- 3. Power on if needed ---
if ($originalState -ne 'Running') {
    Write-Step "Starting bld-01..."
    Start-VM -Name $VM_NAME

    Write-Step "Waiting for WinRM at $VM_IP (up to $Timeout s)..."
    Set-Item WSMan:\localhost\Client\TrustedHosts -Value $VM_IP -Force -ErrorAction SilentlyContinue
    $elapsed = 0
    $ready = $false
    while ($elapsed -lt $Timeout) {
        Start-Sleep -Seconds 10
        $elapsed += 10
        try {
            Invoke-Command -ComputerName $VM_IP -Credential $cred `
                -ScriptBlock { $true } -ErrorAction Stop | Out-Null
            $ready = $true
            break
        } catch { }
    }
    if (-not $ready) { throw "bld-01 did not respond on WinRM within $Timeout seconds." }
    Write-Step "WinRM ready."
}

# --- 4. Run build ---
Write-Step "Running build on bld-01..."
try {
    if ($PSCmdlet.ParameterSetName -eq 'Block') {
        Invoke-Command -ComputerName $VM_IP -Credential $cred -ScriptBlock $ScriptBlock
    } else {
        $content = Get-Content $ScriptFile -Raw
        Invoke-Command -ComputerName $VM_IP -Credential $cred `
            -ScriptBlock ([scriptblock]::Create($content))
    }
} finally {
    # --- 5. Restore power state ---
    if ($originalState -ne 'Running') {
        Write-Step "Restoring VM to $originalState state..."
        Stop-VM -Name $VM_NAME -TurnOff -Force -ErrorAction SilentlyContinue
        Write-Step "VM powered off."
    } else {
        Write-Step "VM was already running — leaving it running."
    }
}

Write-Step "Build complete."

---

Deploying bld-01 (first time or recovery)

The deployment script: D:\git\platform\scripts\build\Deploy-BuildVm.ps1

Steps:

1. Creates the beacon-nat internal switch (idempotent)
2. Creates the NAT rule (idempotent)
3. Creates a dynamic 127 GB VHDX at D:\Data\VMs\bld-01.vhdx
4. Applies the WS 2025 WIM (index 2) from D:\Data\images\
5. Injects a valid Unattend.xml (≤15-char computer name, admin creds, WinRM)
6. Creates the Gen 2 Hyper-V VM
7. Boots and waits for OOBE to complete

One-time cost: 20–40 minutes on first deploy.

Prerequisites

Requirement	Check
Hyper-V installed	Get-WindowsFeature Hyper-V
WS 2025 ISO present	D:\Data\images\en-us_windows_server_2025_*.iso
D:\Data\VMs\ directory	Created by deploy script
kv-hcs-vault-01 accessible	az keyvault secret show ...

Critical: Windows computer names must be ≤ 15 characters (NetBIOS limit). If the name is longer, Setup silently rejects the Unattend.xml and OOBE blocks indefinitely.

---

Build tool matrix

Tool	Tier 1 WSL	Tier 2 ACA (Linux)	Tier 3 bld-01 (Windows)
WinPE ISO (ADK)	❌	❌	✅
PowerShell modules	❌	❌	✅
.NET / C#	✅	✅	✅
Node.js / TypeScript	✅	✅	✅
Azure Bicep	✅	✅	✅
Terraform	✅	✅	✅
Docker images	✅	✅	✅ (nested)
Linux-only tools	✅	✅	❌
Bash scripts	✅	✅	❌

---

Repo integration guide (Tier 3 only)

Only repos with genuine Windows-only builds need these steps. Most repos use Tier 2 via GitHub/GitLab CI labels.

1. Add the wrapper script

# In your repo: scripts/Invoke-BuildOnVm.ps1
# Copy from: D:\git\platform\templates\scripts\Invoke-BuildOnVm.ps1

2. Reference in CI/CD

ADO pipeline:

- task: PowerShell@2
  displayName: Build on bld-01
  inputs:
    filePath: scripts/Invoke-BuildOnVm.ps1
    arguments: -ScriptFile scripts/build.ps1
    pwsh: true

GitHub Actions (Tier 3 wrapper call from platform VM):

- name: Build on bld-01
  shell: pwsh
  run: scripts/Invoke-BuildOnVm.ps1 -ScriptFile scripts/build.ps1

3. Write your build script

Your repo's scripts/build.ps1 runs inside bld-01. It can assume:

• PowerShell 7 at C:\Program Files\PowerShell\7\pwsh.exe
• Git at C:\Program Files\Git\cmd\git.exe
• Azure CLI at az
• Windows ADK + WinPE at C:\Program Files (x86)\Windows Kits\10\
• .NET SDK at C:\dotnet\

---

MCP integration

This standard is served by the HCS Governance MCP:

Tool: get_standard
Domain: build-environments

Any AI client connected to the MCP can retrieve this document for runner selection guidance.

---

Related standards

• Scripting — PowerShell 7 conventions used in build scripts
• Automation — Pipeline-first principle; agent pool selection defers here
• Infrastructure — IaC for runner and build VM provisioning
• Key Vault — All runner registration tokens and VM passwords in kv-hcs-vault-01

---

Checklist: onboarding a repo to CI

Tier 2 (most repos):

• [ ] Add runs-on: [self-hosted, linux, ubuntu-22.04, hcs] to GitHub workflow, or tags: [hcs, linux, ubuntu-22.04] to GitLab CI
• [ ] Confirm the runner pool is healthy: D:\git\platform\.ado\runners\ status

Tier 3 (Windows-only repos only):

• [ ] Copy Invoke-BuildOnVm.ps1 to repo's scripts/ folder
• [ ] Write scripts/build.ps1 with repo-specific build steps
• [ ] Add CLAUDE.md entry: "Windows builds run on bld-01 via Invoke-BuildOnVm.ps1"
• [ ] Add kv-hcs-vault-01 access to repo's CI service principal
• [ ] Validate that bld-01 has all tools the build needs
• [ ] Test: Invoke-BuildOnVm -ScriptBlock { "UP:$env:COMPUTERNAME" }