# Getting started

nunatak profiles an application without modifying it:

```sh
nunatak run -- ./my_binary
nunatak run -- mpirun -n 8 ./solver --input case.nml
```

The application runs once, its stdout/stderr untouched, and its exit code
is propagated - `nunatak run -- ./solver && post_process` behaves exactly
like the bare command. The measurements land in a **Run**: a single
self-sufficient directory under `.nunatak/`, named
`<project>-<date>-<time>`, that survives `scp`, archiving, and being
attached to a ticket. Its path is printed at the end of the run.

On Linux, sampling is collected with `perf`. On other platforms, or when
`perf` is missing, the run still executes: the missing capability is
announced **before** launch as a named degradation with the way forward,
and the Run simply carries fewer measurements.

## Hotspot names

Sampled addresses are attributed to their **physical function** - the
thing with a symbol, an extent and an address, that you can recompile,
isolate and compare - with `llvm-symbolizer` (LLVM 17 or newer, an
external dependency that `doctor` locates and invokes). Compile with `-g`
to reach line-level attribution; without it, the symbol table still
names functions.

Every Hotspot declares how far its attribution could go - its
**resolution level**: `line`, `function`, `symbol` or `unresolved`. A
failed attribution never degrades the measurement: that time really was
spent at that address, and the Hotspot is displayed `module+0x3a1c`
rather than being attached to the nearest symbol. Kernel and vdso
samples stay unresolved by design, and without LLVM the run still
measures - the names are simply missing, and the degradation says so.

`function` and `symbol` name the same situation - a name without a
source position - but not the same remedy: `function` comes from the
symbol table of a binary built without `-g`, `symbol` from the dynamic
symbols of a stripped module, where the way forward is a debuginfo
package, not a recompile.

## Source in the Run

For every Hotspot attributed at line level, the run embeds a **source
extract** - never a whole file: the body of the physical function and
its hot inline frames, a few context lines around. The file is searched
in three steps: the path DWARF recorded, then the `--source-map OLD=NEW`
rewriting (repeatable flag, or a `[source_map]` table in
`nunatak.toml`), then a basename search under the repository root. On
multiple ambiguous matches nunatak does not choose: the Hotspot stays
without source, and the extract carries the reason instead of the text.

A resolved file is checked against the **line-table fingerprint** the
compiler recorded (clang emits an MD5 per source file by default): if
you edited the file since the profiled binary was built, its lines have
moved, and the extract is refused with that reason rather than shown
wrong. Without a fingerprint - gcc emits none - the extract is accepted
as-is.

`--no-source` embeds no source text at all, for what must leave a
sensitive site: line numbers and measurements are kept.

## Checking the environment

```sh
nunatak doctor                  # tool inventory, permissions
nunatak doctor -- ./my_binary   # + target binary inspection
nunatak doctor --json
```

`doctor` invokes the tools instead of trusting their presence on `PATH`.
A cheap subset of it runs automatically at the start of every `run`.

Given a command, `doctor` also inspects the target binary and announces
how far attribution will go - line level with debug information,
function level with a bare symbol table, symbol level on a stripped
binary - **before** any compute time is spent, with the remedy when one
exists (`-g`, or keeping the symbol table).

## Exit codes

The application's code is propagated in the general case. Reserved codes,
in the manner of `timeout`: **127** command not found, **126** found but
not executable, **125** nunatak failure before launch, **121** violation
of `--strict`.

Without `--strict`, a degradation never fails the run. With it, any named
degradation becomes an error - for scripted use and performance CI.

## Configuration

Three TOML layers, by increasing precedence: site (`/etc/nunatak.toml`),
project (`nunatak.toml` at the repository root), command-line flags.

```toml
name = "solver"          # Run naming; --name always wins
runs_dir = "/scratch/me/runs"

[tools]
perf = "/opt/perf/bin/perf"
llvm-symbolizer = "/usr/lib/llvm-19/bin/llvm-symbolizer"

[source_map]
"/build/app" = "/home/me/app"    # where the build tree lives now

[thresholds]
coverage = 0.8           # multiplexing coverage below which a value degrades
```

Every effective value, thresholds included, is recorded in the Run's
provenance: a threshold can be tuned, it cannot be tuned silently.

## The Run directory

```
.nunatak/solver-20260809-142233/
  manifest.json        machine snapshot, provenance, passes, degradations
  pivot/               measurements, events, attribution detail (Parquet)
  collect/             raw collector outputs (perf.data, perf script text)
```

The manifest is plain JSON, readable without nunatak. The pivot holds
measured data only; analyses are recomputed on demand by later commands,
so a Run remains fully exploitable years after being written.

The attribution detail - the inlining chain and the weight of every
sampled address of each named Hotspot - is part of the measured pivot:
it is what lets a later command ventilate a Hotspot by line and by
inline frame on a machine where the binary and the symbolizer no longer
exist. It is detail *inside* a Hotspot, never a unit of analysis.

The machine snapshot records the **allocation shape** alongside the
hardware: the cores this job actually received (affinity mask) and its
cgroup CPU and memory limits. A Machine is that couple, not a node - a
job given 8 cores of a 128-core node is a different Machine than the
whole node, and its ceilings will be measured for those 8 cores.

Until a calibration has measured the Machine, the Run carries
**theoretical FLOP/s ceilings**: the microarchitecture's per-cycle
capability, crossed with the exposed frequency and scaled to the
allocation. They are always of quality `estimated`, with the reason -
and an unknown microarchitecture yields no ceiling at all rather than
an extrapolation. Memory-bandwidth ceilings only exist measured.

**Measured ceilings** come from an embedded microbenchmark kernel -
a STREAM-style triad for memory bandwidth, FMA chains in intrinsics for
the FLOP/s peaks - compiled locally with whatever compiler the machine
offers and run as a separate process, never inside the Python
interpreter. A ceiling is the **maximum of its repetitions**, never
their mean: it is an upper bound. Polluted conditions - dispersed
repetitions, concurrent load, a kernel built without SIMD, a value far
above the theoretical peak - downgrade the ceiling to `estimated` with
the reason, they never discard it.

## Calibrating the Machine

The calibration triggers by itself at the **first `run` on an unknown
Machine**, before the application launches - the only moment the node
is truly yours. The profile is cached (keyed by hardware plus
allocation shape) and reused by every later Run on the same Machine.

```sh
nunatak calibrate            # spend the budget in a dedicated job
nunatak calibrate --force    # recalibrate despite a cached profile
nunatak run --no-calibrate -- ./solver   # skip it: ceilings stay theoretical
```

Ceilings are measured in priority order within a ~60 s budget - memory
bandwidth and the double-precision peak first, because without them
there is no roofline. A partial profile stays exploitable; whatever was
not measured keeps its theoretical, `estimated` value. When nothing can
be measured (no compiler on the machine), nothing is cached: the next
run tries again.

## Counter groups

On a microarchitecture nunatak knows (AMD Zen 2/3/4 today), sampling
attributes more than time: a **FLOP counter** and the **DRAM demand
fills**, scaled to bytes, ride along with `task-clock`. Each auxiliary
event uses a fixed period - every sample is worth exactly its period,
so the totals match `perf stat` within a fraction of a percent, and the
interrupt rate stays bounded by construction.

Honesty travels with the numbers: DRAM bytes come from demand fills
only (hardware prefetchers bypass them, and sampling prefetch events
inflates what they measure - an observer effect), so those Measurements
are `estimated` with that reason; Zen does not split FLOPs by
precision, so a placement against the double-precision peak says so
too. An unknown microarchitecture samples time alone, and a kernel that
rejects the event names degrades to time-only without ever running the
application twice.

## MPI runs

`nunatak run -- mpirun -n 8 ./solver` starts one launcher here and
eight ranks wherever the scheduler placed them. Collection then has two
layers, with two costs. The **counting layer** covers **every rank** at
constant cost: nunatak interposes a small shim inside each rank -
between the launcher and the application, without touching either -
which counts time, cycles and instructions around the whole rank and
writes its aggregates into the Run directory itself. Those per-rank
totals are what reveal load imbalance, and they carry no Hotspot:
attributing them to functions is the sampling layer's job.

Each rank writes home before it exits, so a Run stays **one
directory** whatever the number of ranks and nodes - the retrieval is
done before the job epilogue, when the allocation still exists. A rank
whose node has no usable `perf` runs bare and is declared by number
(`counting-unavailable`); ranks that the world size announces but that
left nothing behind are declared too (`counting-incomplete`). Silence
about a missing rank would read as "nothing ran there".

The shim propagates the application's exit code and never touches its
stdout or stderr, and a launcher that fails to resolve its application
is left alone: nunatak wraps a launch it understands, it never guesses.

Under a launcher, sampling attributes **time alone** for now: hardware
events belong to the ranks' counting - an outer sampler holding the
same physical counters corrupts what the ranks measure, a fact
measured on real hardware, not a precaution.

## What the analysis says

The analysis engine is a **pure function of (pivot, Machine)**: nothing
it produces is persisted, everything is recomputed on demand, so a Run
stays analyzable years later. For every Hotspot above the statistical
floor it states a **Diagnostic**: the share of the run, the roofline
placement - DRAM arithmetic intensity, achieved FLOP/s against the
envelope `min(compute peak, bandwidth x intensity)` - the imbalance
across Loci, and a **classification**: `imbalance`, `latency-bound`,
`memory-bound` or `compute-bound`. A classification states a regime,
never a cause.

Every derived quantity carries its **lineage** and its Quality,
propagated as the worst of its inputs: a number displayed `measured` is
measured end to end. Where a source counter does not exist - no FLOP
counter collected, no clock in seconds - the fact is `unavailable` with
the reason, never approximated.

## The summary at the end of the run

The log closes on three moments: the **summary**, the degradations
again - their announcements scrolled past long ago in a job log - and
the paths. The summary is the report's first reading level: the
sampling coverage first, then the findings ordered by decreasing share
of the sampled time, each with its quantified evidence:

```
summary: 1 Hotspot above the statistical floor holds 100% of the sampled time (2051 samples of task-clock over 2.06 s)
  main (line) - 100% of the sampled time - latency-bound
    achieved 1.22 GFLOP/s of 125 GFLOP/s attainable: 1.0% of the envelope
    DRAM intensity 1.21 flop/byte
    downgraded to estimated: demand fills only: hardware-prefetched traffic is not counted; ...
Run: .nunatak/workload-20260811-142233
Report: .nunatak/workload-20260811-142233/report.html
```

Reading only the log teaches fewer details than the report, never less
about how solid the numbers are: a downgraded value states its reason, a
Hotspot that cannot be placed says why where the placement was expected,
and what is missing is gathered under **"what this report does not
say"** - the time below the statistical floor aggregated as "others",
the time attributed to no name, the envelope ceilings that are only
estimated - instead of being scattered across footnotes or, worse,
omitted.

## The HTML report

Every measuring run also writes `report.html` into the Run directory: a
**self-contained page** - no CDN, no font, no request of any kind -
that opens on a cluster without a server and still reads in ten years
from an archived file. It carries three reading levels, in the same
vocabulary as the terminal summary.

The **synthesis** opens the page: coverage, findings with direct access
to their Hotspot, "what this report does not say". Below it, the
**inventory** lists every Hotspot above the statistical floor, sortable
by any numeric column and filterable by regime, estimated Quality or
missing source. Quality and resolution level are separate columns and
never look alike: Quality is color and shape (measured plain, estimated
hatched - a downgraded row shows its reasons on hover), the resolution
level a neutral text label. An empty cell means the quantity is
**unavailable** for that Hotspot, not that it is zero - the table says
so under its last row, and the below-floor aggregate "others" closes
the table as a row of its own.

Opening a Hotspot - from a finding or an inventory row - **substitutes**
the inventory with its **detail**; the two never sit side by side, and
the way back is the explicit button or `Escape`. The detail is where
the **roofline** lives: contextualized on one Hotspot, the device is
implicit and the chart is correct by construction - the memory diagonal
stops at the ridge (a unit-tested invariant), the selected Hotspot uses
the same Quality encoding as everywhere (full disc measured, dashed
outline estimated), the other placeable Hotspots stay as pale points
for scale. A Hotspot that cannot be placed says why **in the spot where
the chart was expected**, never a blank. Beside the chart: the metrics
with their absences written `unavailable`, the **source annotated with
samples per line** - line numbers and distribution survive `--no-source`,
only the text is withheld - and the ventilation by inline frame.

```sh
nunatak report              # regenerate the report of the most recent Run
nunatak report <run-dir>    # or of a specific one
nunatak report --no-source  # a shareable variant: no source text at all
```

Regenerating is a real operation: the analysis is never persisted, so
the report is recomputed from the measured pivot - after an upgrade, or
on a machine that only received the Run directory.

`--no-source` writes `report-no-source.html` next to the full report,
never in its place. The payload is stripped **before the page exists**,
so the shared file never contained a line of code - a page-side toggle
would be a trap, hiding text that stays embedded. Line numbers and the
per-line sample distribution remain: one still sees where the time
goes.

The **Provenance** - commit and tree state, collectors with their
versions, observed dependencies, the effective configuration with its
thresholds - is a drawer unfolding from the report's header: never a
dialog, never in the main view.

The page is rendered by a compiled TypeScript mini-app embedded in the
package. Installed wheels carry it already compiled: the packaging hook
builds it where the wheel is built, so Node is needed only there -
never on the machines that install or run nunatak. Building a wheel
from the sdist without npm still succeeds; the result simply lacks the
bundle. On a development checkout it is built once with
`npm install && npm run build` in `report-app/`. Wherever the bundle is
missing, the run continues and announces the named degradation
`report-unavailable` - a missing capability never fails a run.