security-research

kernelCTF Service (v5)

kernelCTF Service v5 is the remote evaluation and verification backend for Google’s kernelCTF vulnerability research and reward program. It provides an automated, secure environment to evaluate Linux kernel privilege escalation exploits, verify vulnerability triggers, and generate cryptographically signed verification flags.


Table of Contents


Architecture Overview

                          +-----------------------------------+
                          |      Participant / Researcher     |
                          +-----------------------------------+
                                            |
                                 TLS 1.3 (Port 1337)
                                            v
                          +-----------------------------------+
                          |     socat + server/service.sh     |
                          +-----------------------------------+
                                            |
                                            v
                          +-----------------------------------+
                          |         server/server.py          |
                          |  - Ingests exploit via memfd      |
                          |  - Menu & Action Dispatcher       |
                          |  - HMAC Flag Generation           |
                          +-----------------------------------+
                                            |
                                            v
                          +-----------------------------------+
                          |          server/qemu.sh           |
                          |      server/vm/run_vmlinuz.sh     |
                          +-----------------------------------+
                                            |
                     +----------------------+----------------------+
                     | (KVM Virtualization)                        |
                     v                                             v
        +-------------------------+                   +---------------------------+
        | Hardened Kernel Target  |                   |   LTS / LTS-KASAN Target  |
        | (e.g. hardened-v1-7.2)  |                   |   (e.g. lts-6.12.x)       |
        | - Flag leak validation  |                   | - Vulnerability trigger   |
        | - 20 evaluation runs    |                   | - Kernel crash detection  |
        +-------------------------+                   +---------------------------+

How It Works

1. Client Connection & Ingestion

  1. The service listens on port 1337 via socat with TLS 1.3 encryption (server/service.sh).
  2. Once connected, server/server.py prompts the participant with the menu of active targets and actions (lpe-test, vuln-test, evaluate, info).
  3. When an action is selected:
    • The client specifies the binary/script size (up to 50MB) and its SHA256 hash.
    • The binary data is streamed directly into an anonymous Linux memory file descriptor (memfd_create) via server/utils.py, avoiding writing untrusted exploit binaries to disk.

2. VM Execution & Isolation

  1. server/qemu.sh and server/vm/run_vmlinuz.sh launch QEMU with KVM acceleration using the specified kernel release (bzImage).
  2. Hardware virtualization features (such as Intel IBT / CET) are strictly validated upon boot inside server/vm/rootfs/init.
  3. The exploit file descriptor (/proc/self/fd/<fd>) and dynamic flag file are passed via QEMU drives/unmap devices and made accessible inside the VM:
    • /flag (mode 0400, owned by root).
    • /exploit (executable).
  4. Exploit commands run under an unprivileged user user (or root in root testing mode).
  5. Output separation:
    • Command output is captured across dedicated serial channels (/output on ttyS1).
    • Kernel logs (dmesg on ttyS0) are captured independently for authentic panic and KASAN bug analysis.

3. Evaluation Lifecycle & Actions

Action Target(s) Description
info All active targets Prints direct download links for target bzImage, vmlinux.gz, .config, lakitu_defconfig, and COMMIT_INFO.
lpe-test Hardened Target (hardened-v1-*) Single-run exploit test. Executes /exploit and checks if the participant can read and leak /flag.
vuln-test LTS & LTS-KASAN Targets (lts-*, lts-*-kasan) Runs /exploit --vuln-trigger. Verifies if the exploit triggers a genuine kernel panic or KASAN report in kernel dmesg.
evaluate All Targets (LTS, LTS-KASAN, Hardened) Full official evaluation workflow (rate-limited to 10 evaluation runs per researcher per open slot):
1. Phase 1: Runs --vuln-trigger up to 3 times on LTS-KASAN and LTS to confirm kernel crash.
2. Phase 2: Runs exploit 20 times on Hardened target to benchmark reliability and execution time.
3. Phase 3: If eligible and within the submission window, computes statistics and prints signed flag.

4. Flag Generation & Signing

Verification flags are signed with HMAC-SHA1 using a secret key configured in secrets/server_secrets.py:

kernelCTF{v5:<hardened_target>+<lts_target>:<flag_id>:<attributes>:<timestamp>:<binary_hash>:<researcher_hash>:<hmac_signature>}

Project Structure

.
├── config/                          # Configuration & release registries
│   ├── releases.yaml                # Target release registry & schedules
│   └── auto_release.yaml            # Channel filters & webhook settings
│
├── server/                          # Server backend & VM execution runtime
│   ├── service.sh                   # Socat TLS listener entrypoint (Port 1337)
│   ├── server.py                    # Main server interactive menu & dispatch
│   ├── utils.py                     # MemFd binary streaming, timing & process streamer
│   ├── qemu.sh                      # QEMU launcher & argument mapper
│   └── vm/                          # QEMU environment, initramfs builders & rootfs
│       ├── run_vmlinuz.sh
│       ├── update_rootfs_image.sh
│       └── rootfs/
│
├── tools/                           # Release pipeline & synchronization tasks
│   ├── auto_release.py              # GCS discovery daemon & notifier
│   ├── activate_releases.py         # Promotion from staging to active releases
│   ├── download_latest_releases.sh  # Manual release downloader helper
│   ├── pull_from_server.sh          # Sync logs/data from remote host
│   └── update_opensource.sh         # Upstream open-source sync tool
│
├── infra/                           # Deployment, VM setup, systemd & crontab
│   ├── deploy.sh                    # Remote VM staging & deployment orchestrator
│   ├── setup.sh                     # Idempotent host setup & dependency installer
│   ├── get_latest_lts.py            # LTS release parsing utility (used by setup.sh)
│   ├── server_cert_gen.sh           # TLS certificate generator
│   ├── systemd/
│   │   └── kernelctf.service        # Systemd service unit file
│   └── cron/
│       └── auto_release.cron        # Cron schedule definition
│
├── client/                          # Participant / researcher client tooling
│   ├── cli.py                       # Evaluation & test submission CLI client
│   ├── connect.sh                   # Interactive socat TLS remote shell client
│   ├── connect_local.sh             # Interactive socat TLS local shell client
│   └── server_cert.pem              # Public server certificate for TLS verification
│
├── tests/                           # Unit and integration test suite
│   ├── run_tests.sh                 # End-to-end multi-target test suite
│   ├── test_auto_release.py         # Unit tests for release automation
│   └── payloads/                    # Test exploit binaries & scripts
│       ├── test_exploit.sh
│       └── test_exploit_random.sh
│
├── secrets/                         # [Gitignored] Secrets, keys & certificates
│   ├── .gitignore                   # Ignores secrets while tracking examples
│   ├── server_secrets.py.example    # Example secrets template
│   ├── ssh_keys.txt.example         # Example SSH keys template
│   ├── server_secrets.py            # Active webhook URL, HMAC key, root hash
│   ├── kernelctf-vm-reader-sa-key.json # GCS service account key
│   ├── server_key.pem               # TLS private key
│   ├── server_cert_and_key.pem      # TLS certificate & private key
│   └── ssh_keys.txt                 # Authorized SSH keys for VM access
│
└── data/                            # [Gitignored] Runtime binary storage
    ├── releases/                    # Active downloaded production releases
    └── staging/                     # Staged upcoming releases

Kernel Releases & Automation

Kernel builds are hosted on Google Cloud Storage (gs://kernelctf-build/releases/).

Release Discovery (tools/auto_release.py)

Run periodically via cron (infra/cron/auto_release.cron):

  1. Checks GCS bucket for newly published builds matching configured release channels in config/auto_release.yaml.
  2. Downloads kernel images (bzImage, vmlinux.gz), configs, and commit info into ./data/staging/.
  3. Sends Discord notifications regarding upcoming release windows.

Release Activation (tools/activate_releases.py)

Used to safely promote releases from ./data/staging/ to ./data/releases/:

  1. Validates required kernel configuration symbols (e.g. CONFIG_IO_URING=y, mitigation configs).
  2. Verifies HTTP 200 accessibility for public artifacts on GCS.
  3. Promotes verified builds to active ./data/releases/ directory.

Server Setup & Deployment

Prerequisites

Host Setup (infra/setup.sh)

infra/setup.sh is idempotent and configures the host environment:

sudo ./infra/setup.sh

It performs:

Deploying to Remote VM (infra/deploy.sh)

To deploy local changes and secrets to a remote VM:

./infra/deploy.sh [TARGET_HOST]
# Example:
./infra/deploy.sh kernelctf.vrp.ctfcompetition.com

Testing & Client Evaluation

Using client/cli.py

client/cli.py interacts with local or remote kernelCTF server instances. By default, it connects to the remote production service over TLS.

# Query active target release info and artifact URLs (no binary required)
python3 client/cli.py --action info

# Evaluate exploit against remote server (default target)
python3 client/cli.py exploit_binary --action evaluate

# Run lpe-test against remote server
python3 client/cli.py exploit_binary --action lpe-test

# Run vuln-test against custom remote target
python3 client/cli.py exploit_binary --action vuln-test --remote kernelctf.example.com:1337

# Test exploit against local server instance
python3 client/cli.py tests/payloads/test_exploit.sh --action lpe-test --local

# Local evaluation testing with overrides
python3 client/cli.py tests/payloads/test_exploit.sh \
  --action evaluate \
  --local \
  --ignore-open-slots \
  --show-vm-output \
  --root

Running the Test Suite (tests/run_tests.sh)

Run the complete validation suite:

# Run against local server instance
./tests/run_tests.sh

# Run against remote TLS server
./tests/run_tests.sh --remote kernelctf.example.com:1337 -k

Tests include:

  1. Unit test suite discovery (tests/test_auto_release.py).
  2. lpe-test flag leak validation in root mode.
  3. vuln-test kernel crash detection via SysRq panic in root mode.
  4. vuln-test anti-spoofing verification (unprivileged userspace fake panic rejection).
  5. evaluate full 20-run loop with simulated intermittent exploits and flag generation.

Manual Interactive Shell (client/connect.sh)

To interactively connect to the service menu via TLS:

./client/connect.sh [TARGET_HOST:PORT]

Security & Anti-Spoofing Mitigations