Commands as byte streams.

Automation as code

Command buffers are for the work around software: building and releasing it, moving files and data between machines, setting up servers and containers. That work usually lives in shell scripts, task runners, and pipeline configuration — and as it grows conditionals, loops, and templating, the configuration becomes a programming language without a debugger, a formatter, or a test framework — and the tool itself is often replaced before the investment pays off.

Command buffers start from the other end: a real programming language, with the pipelines and file operations automation needs. Scripts get functions, libraries, and tests from the first line; they ship as one static binary; and the same program runs on a laptop, in CI, and across Linux, macOS, and Windows. The Tour of Go covers the language basics this page assumes.

Adoption is incremental. Nothing here asks for a rewrite: one script or one CI job — usually the flaky one — can move while everything around it stays put.

The model

• A Buffer is a command's execution as an io.Reader: the command starts on the first Read and is finished at io.EOF.
• A Machine is anything that can run a command — one interface, one method: Command(ctx, args...) Buffer.
• Machines wrap machines: ssh, ctr, and sub each take a Machine and return a new one, so a container on a remote host is just two calls.
• Files are buffers too: command.FS gives any Machine a filesystem, and copying between machines is io.Copy. Directories stream as tar archives.
• A Shell makes automation portable: declare the external commands you truly need, and everything else — files, directories, OS detection — goes through operations that work on any machine.

Every example on this page compiles and runs against the current releases of lesiw.io/command and lesiw.io/fs.

Reading runs the command

There is none of os/exec's ceremony — no Start, no Wait, no pipe plumbing. A buffer executes because something consumes it, the same way a shell pipeline only moves when something reads the pipe.

package main

import (
    "context"
    "fmt"
    "io"
    "log"

    "lesiw.io/command"
    "lesiw.io/command/mem"
)

func main() {
    ctx, m := context.Background(), mem.Machine()

    out, err := io.ReadAll(command.NewReader(ctx, m, "echo", "Hello, world!"))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s", out)
}

Run this program in the Go Playground — mem.Machine is an in-memory machine, so the example works anywhere.

Because a pipe is just bytes moving from a reader to a writer, piping one command into another is io.Copy:

// echo "hello, pipes" | tee hello.txt
_, err := io.Copy(
    command.NewWriter(ctx, m, "tee", "hello.txt"),
    command.NewReader(ctx, m, "echo", "hello, pipes"),
)

Run this program in the Go Playground. For pipelines of three or more commands, use command.Copy with command.NewFilter for the middle stages.

Three helpers cover the everyday cases, so most code never touches a buffer directly:

// version=$(go version)     — capture output, like command substitution.
version, err := command.Read(ctx, m, "go", "version")

// go test ./... >/dev/null  — run for the side effect, discard output.
err = command.Do(ctx, m, "go", "test", "./...")

// go vet ./...              — attach to the terminal, stream output live.
err = command.Exec(ctx, m, "go", "vet", "./...")

Environment variables travel on the context, so they compose the way the rest of Go does:

ctx = command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"})
err := command.Exec(ctx, m, "go", "build", ".")

Anything that runs a command is a machine

The interface has one method. Everything else in the package is built on it.

type Machine interface {
    Command(ctx context.Context, arg ...string) Buffer
}

m := sys.Machine()

m := ssh.Machine(sys.Machine(),
    "ssh", "user@host")

m := ctr.Machine(sys.Machine(),
    "alpine:latest")

m := sub.Machine(sys.Machine(),
    "busybox")

m := mem.Machine()

m := new(mock.Machine)

Machines take machines. ssh.Machine needs somewhere to run ssh; ctr.Machine needs somewhere to run docker. That somewhere is another Machine — so environments nest by construction:

// A container on a remote build host.
host := ssh.Machine(sys.Machine(), "ssh", "admin@build.example.com")
m := ctr.Machine(host, "golang:latest")
defer command.Shutdown(ctx, m)

sh := command.Shell(m, "go")
if err := sh.Exec(ctx, "go", "test", "./..."); err != nil {
    log.Fatal(err)
}

A prefix glued onto a command — ssh host …, docker exec …, env FOO=bar … — is a machine that hasn't been named yet. Once it is one, the code that runs commands no longer depends on where they run: the same program works on a laptop, in a container, and on a production host.

A machine doesn't have to be a computer. It can be a single command, enriched — here, a shim that injects an environment variable into every go invocation:

m := command.HandleFunc(sys.Machine(), "go",
    func(ctx context.Context, args ...string) command.Buffer {
        ctx = command.WithEnv(ctx, map[string]string{
            "GOFLAGS": "-trimpath",
        })
        return sys.Machine().Command(ctx, args...)
    })

flags, err := command.Read(ctx, m, "go", "env", "GOFLAGS")
// flags == "-trimpath"

Files are buffers too

command.FS turns any machine into a filesystem, accessed through lesiw.io/fs — a context-aware cousin of io/fs with write support. On machines without native filesystem access, file operations are implemented with the commands the target system has: tee on Unix, Remove-Item on Windows. Your code doesn't see the difference — and Windows is an origin, not just a destination. The same program, compiled for and run on a Windows host:

sh := command.Shell(sys.Machine())
fmt.Println(sh.OS(ctx), sh.Arch(ctx))          // windows amd64
err := sh.WriteFile(ctx, "out.txt", data)      // PowerShell idioms underneath
err = sh.MkdirAll(ctx, `logs\2026`)            // no mkdir -p in sight

fsys := command.FS(m)

err := fs.WriteFile(ctx, fsys, "greeting.txt", []byte("Hello from fs!\n"))

data, err := fs.ReadFile(ctx, fsys, "greeting.txt")

Run this program in the Go Playground.

Copying a file between machines is the same io.Copy as piping commands. The operations scp, docker cp, and cp share one implementation:

local := command.Shell(sys.Machine())
remote := command.Shell(ssh.Machine(sys.Machine(), "ssh", "deploy@prod.example.com"))

// Stream a local build to the remote host.
_, err := io.Copy(
    remote.CreateBuffer(ctx, "/opt/app/server"),
    local.OpenBuffer(ctx, "bin/server"),
)

A directory is a buffer too: a tar archive, indicated by a trailing slash. Copying a directory into a container looks exactly like copying a file:

box := ctr.Machine(sys.Machine(), "alpine:latest")
defer command.Shutdown(ctx, box)

local, remote := command.Shell(sys.Machine()), command.Shell(box)

dst, err := remote.Create(ctx, "/app/")   // trailing slash: a directory
if err != nil {
    log.Fatal(err)
}
defer dst.Close()
src, err := local.Open(ctx, "src/")
if err != nil {
    log.Fatal(err)
}
defer src.Close()
if _, err := io.Copy(dst, src); err != nil {
    log.Fatal(err)
}

Because commands and files are the same shape, they mix in one pipeline. This example dumps a database, compresses it in flight, and lands it on another machine, with no intermediate files:

backup := ssh.Machine(m, "ssh", "backup@vault.example.com")
_, err := command.Copy(
    command.NewWriter(ctx, backup, "tee", "db.sql.gz"),
    command.NewReader(ctx, m, "pg_dumpall"),
    command.NewFilter(ctx, m, "gzip"),
)

Shells make automation portable

command.Shell exists to keep automation portable. It wraps a machine with two things: operations that work on any machine for everything a filesystem can do, and an explicit list of the external commands your automation is allowed to run. Anything you didn't declare fails with command not found at the moment of the call — the dependency list lives at the top of the file, and it stays short because most of what shell scripts shell out for doesn't need a command at all.

sh := command.Shell(sys.Machine(), "go")

if err := sh.Exec(ctx, "go", "vet", "./..."); err != nil {
    log.Fatal(err)
}

ver, err := sh.ReadFile(ctx, "VERSION")
if err != nil {
    ver = []byte("dev")
}

if err := sh.MkdirAll(ctx, "bin"); err != nil {
    log.Fatal(err)
}
err = sh.Exec(
    command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"}),
    "go", "build", "-ldflags", "-X main.version="+string(ver),
    "-o", "bin/app", ".",
)

The build declares only go. Reading VERSION and creating the output directory go through the Shell's portable methods, which work on any machine — including Windows, where cat and mkdir -p do not exist. Both libraries' CI suites (command, fs) run the tests — not just the build — on Linux, macOS, Windows, FreeBSD, and Alpine.

Automation you can test

Automation written against Machine is code, so it tests like code. mock.Machine records every call and returns whatever you program:

func Deploy(ctx context.Context, sh *command.Sh) error {
    branch, err := sh.Read(ctx, "git", "branch", "--show-current")
    if err != nil {
        return fmt.Errorf("read branch: %w", err)
    }
    return sh.Exec(ctx, "git", "push", "origin", branch)
}

func TestDeploy(t *testing.T) {
    m := new(mock.Machine)
    m.Return(strings.NewReader("main\n"), "git", "branch", "--show-current")

    sh := command.Shell(m, "git")
    if err := Deploy(t.Context(), sh); err != nil {
        t.Fatal(err)
    }

    got := mock.Calls(sh, "git")
    want := []mock.Call{
        {Args: []string{"git", "branch", "--show-current"}},
        {Args: []string{"git", "push", "origin", "main"}},
    }
    if !cmp.Equal(want, got) {
        t.Errorf("git calls mismatch (-want +got):\n%s", cmp.Diff(want, got))
    }
}

No git repository, no network, no side effects — and the test asserts the exact commands your automation would have run in production.

When a command fails

Automation spends much of its life handling failure, so failures carry their evidence. A failed command returns a *command.Error with the exit code and captured stderr:

err := command.Do(ctx, m, "go", "build", ".")

var cerr *command.Error
if errors.As(err, &cerr) {
    fmt.Println("exit code:", cerr.Code)
    fmt.Printf("stderr:\n%s", cerr.Log)
}

When a multi-stage pipeline fails, the error reports every stage and its outcome, so the failing machine and command are never a mystery:

_, err := command.Copy(
    command.NewWriter(ctx, m, "wc", "-c"),
    command.NewReader(ctx, m, "echo", "not gzip data"),
    command.NewFilter(ctx, m, "gzip", "-d"),
)
fmt.Println(err)
// <*command.reader>
//     <success>
//
// <*command.filter>
//     exit status 1
//         gzip: unknown compression format

Output produced before a failure is not lost: reads return the bytes that arrived, and the error follows. Cancellation follows the context. Cancel the ctx and a running command is killed; the pending Read returns the error, and no process is left behind. NewReader's Close does the same for early exits from a pipe.

And failures are programmable in tests, so the error path gets exercised as easily as the happy path:

m.Return(command.Fail(&command.Error{Code: 1}), "git", "push")

if err := Deploy(t.Context(), sh); err == nil {
    t.Error("Deploy() should fail when the push fails")
}

Inside the CI you already have

Command buffers don't replace a CI system; they replace the scripts inside it. A pipeline job runs a Go program:

deploy:
  stage: deploy
  script: go run ./ops deploy

Reproducing that job on a laptop is the same command: go run ./ops deploy. The YAML shrinks to a list of triggers; the logic — the part that breaks — lives in a program that runs anywhere, so a failing job can be reproduced and debugged without pushing a commit to find out. The orchestrator stops mattering: GitLab, Jenkins, GitHub Actions, or nothing at all — a job becomes a program invocation, and an artifact becomes a file or directory stream the program moves itself. Porting can start with a single job, usually the flaky one, while the rest of the pipeline stays put.

Why Go for automation

Any compiled language could take automation out of configuration formats. Go's particulars make it a good choice:

• It keeps working. The Go 1 compatibility promise means automation written today still builds years from now.
• Dependencies are solved. Modules give automation code the same sharing and versioning as any other code.
• Concurrency is ordinary. Backing up one host and backing up a hundred are the same code plus a goroutine per host.

Command buffers are not a replacement for a five-line shell script. They're for the automation that outgrew one — the build that needs a real conditional, the deploy that spans three machines, the script someone finally asked you to write a test for.

Get started

go get lesiw.io/command

A complete program:

package main

import (
    "context"
    "fmt"
    "log"

    "lesiw.io/command"
    "lesiw.io/command/sys"
)

func main() {
    ctx, m := context.Background(), sys.Machine()

    version, err := command.Read(ctx, m, "go", "version")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(version)
}

Keep reading

• lesiw.io/command — full API documentation, including a cookbook of shell idioms translated to Go.
• lesiw.io/fs — the filesystem abstraction: local, remote, and in-memory filesystems behind one interface.
• The cmdbuf guide — a condensed, example-first reference for writing idiomatic command buffer code. Written with coding agents in mind; humans welcome too.
• Source on GitHub.