Keep C: responsive during backups; add file logging, mount UI, RPC debounce

kleb commented

2026-07-14 17:48:47 +02:00

Owner

Problem

On Windows, a folder backup (e.g. Downloads) made the whole C: drive feel
unavailable for the duration of the run. The system's own logs ruled out a
volume fault: no volsnap/Ntfs/disk error, only a benign "shadow copy healthy"
event. The cause is I/O starvation — the fstree walker read one
file per CPU (GOMAXPROCS) with plain buffered os.Open at normal I/O
priority and no throttle, saturating the disk queue and evicting the Windows
file cache. Two secondary problems surfaced alongside it: the service kept no
inspectable log (slog went to stderr, discarded under the SCM), and the TUI
spammed "multiple RPC errors" whenever the machine was busy or the service
restarted.

Changes

Keep C: responsive (priority)

platform.OpenBackupRead (backupfile_windows.go / backupfile_other.go):
opens source files with FILE_FLAG_SEQUENTIAL_SCAN and a low per-handle
I/O priority hint (SetFileInformationByHandle, FileIoPriorityHintInfo),
so backup reads are scheduled below interactive I/O and don't thrash the file
cache. Wired into the fstree content + ADS read path. This is per-handle, so
it never throttles the service's own RPC threads.
New service.max_backup_read_concurrency (0 = auto = min(GOMAXPROCS, 4)),
wired to fstree.Options.Workers via Dispatcher.SetBackupReadConcurrency
(live-applied on settings update).

Inspectable logs

New rotating file sink (service.log_file, default
<state_dir>/logs/kbackupd.log, rotated at 10 MiB), in addition to stderr.
Dependency-free. This is what finally lets the service log be read under the
Windows SCM.

Tame transient RPC-error spam

TUI heartbeat tolerates a single blip before declaring "connection lost";
the dashboard's 2s poll debounces transient failures and clears on recovery;
inherently-transient live-stream ends are no longer surfaced.

Mount / browse a snapshot from the TUI

Snapshots tab: m mounts the selected snapshot as a read-only drive
(prompting for a mount point), M lists active mounts with u to unmount —
reusing the already-built MountService. Windows needs the WinFsp driver.

Tests

Adds unit tests for the worker resolver, log rotation + path resolution,
OpenBackupRead (Windows), the new config validation, and the TUI
heartbeat/dashboard debounce. go build ./..., go vet ./..., and
go test ./... all pass.

Not included / follow-up

Runtime verification (C: responsiveness during a real backup, actual WinFsp
mount, log-file creation under the live service) must be done on a disposable
Windows VM, not the maintainer's host.
Doc additions for the installer README (WinFsp requirement, log path) are
held back because that file is part of separate, not-yet-committed installer
work; fold them in when that lands.
Optional hard MB/s bandwidth cap (deliberately deferred — the per-handle low
priority self-tunes).

## Problem On Windows, a folder backup (e.g. Downloads) made the whole **C: drive feel unavailable** for the duration of the run. The system's own logs ruled out a volume fault: no volsnap/Ntfs/disk error, only a benign "shadow copy healthy" event. The cause is **I/O starvation** — the fstree walker read one file per CPU (`GOMAXPROCS`) with plain buffered `os.Open` at normal I/O priority and no throttle, saturating the disk queue and evicting the Windows file cache. Two secondary problems surfaced alongside it: the service kept no inspectable log (slog went to stderr, discarded under the SCM), and the TUI spammed "multiple RPC errors" whenever the machine was busy or the service restarted. ## Changes **Keep C: responsive (priority)** - `platform.OpenBackupRead` (`backupfile_windows.go` / `backupfile_other.go`): opens source files with `FILE_FLAG_SEQUENTIAL_SCAN` and a **low per-handle I/O priority hint** (`SetFileInformationByHandle`, `FileIoPriorityHintInfo`), so backup reads are scheduled below interactive I/O and don't thrash the file cache. Wired into the fstree content + ADS read path. This is per-handle, so it never throttles the service's own RPC threads. - New `service.max_backup_read_concurrency` (0 = auto = `min(GOMAXPROCS, 4)`), wired to `fstree.Options.Workers` via `Dispatcher.SetBackupReadConcurrency` (live-applied on settings update). **Inspectable logs** - New rotating file sink (`service.log_file`, default `<state_dir>/logs/kbackupd.log`, rotated at 10 MiB), in addition to stderr. Dependency-free. This is what finally lets the service log be read under the Windows SCM. **Tame transient RPC-error spam** - TUI heartbeat tolerates a single blip before declaring "connection lost"; the dashboard's 2s poll debounces transient failures and clears on recovery; inherently-transient live-stream ends are no longer surfaced. **Mount / browse a snapshot from the TUI** - Snapshots tab: `m` mounts the selected snapshot as a read-only drive (prompting for a mount point), `M` lists active mounts with `u` to unmount — reusing the already-built `MountService`. Windows needs the WinFsp driver. ## Tests Adds unit tests for the worker resolver, log rotation + path resolution, `OpenBackupRead` (Windows), the new config validation, and the TUI heartbeat/dashboard debounce. `go build ./...`, `go vet ./...`, and `go test ./...` all pass. ## Not included / follow-up - Runtime verification (C: responsiveness during a real backup, actual WinFsp mount, log-file creation under the live service) must be done on a disposable Windows VM, not the maintainer's host. - Doc additions for the installer README (WinFsp requirement, log path) are held back because that file is part of separate, not-yet-committed installer work; fold them in when that lands. - Optional hard MB/s bandwidth cap (deliberately deferred — the per-handle low priority self-tunes).

kleb added 1 commit

2026-07-14 17:48:47 +02:00

Keep C: responsive during backups; add file logging, mount UI, RPC debounce 292f18905d

Folder backups saturated the source disk (unthrottled GOMAXPROCS-wide
buffered reads at normal I/O priority), making Windows C: feel unavailable
for the whole run -- Windows logs showed no volume fault, only I/O
starvation. Changes:

- platform.OpenBackupRead: open source files with FILE_FLAG_SEQUENTIAL_SCAN
  and a low per-handle I/O priority hint (SetFileInformationByHandle,
  FileIoPriorityHintInfo); wired into the fstree read path (content + ADS).
- service.max_backup_read_concurrency knob (0=auto=min(GOMAXPROCS,4)) wired
  to fstree.Options.Workers via Dispatcher.SetBackupReadConcurrency.
- File log sink (service.log_file, default <state_dir>/logs/kbackupd.log,
  rotated at 10 MiB) so service logs are inspectable under the Windows SCM,
  where stderr is discarded.
- TUI: debounce transient heartbeat/dashboard-poll RPC errors and stop
  surfacing inherently-transient live-stream ends (the "multiple RPC errors"
  noise).
- TUI: mount a snapshot as a read-only drive from the Snapshots tab
  (m mounts, M lists/unmounts), reusing the existing MountService.

Adds unit tests for the worker resolver, log rotation/path resolution,
OpenBackupRead (Windows), config validation, and the TUI debounce logic.

kAgent commented

2026-07-14 17:48:50 +02:00

Collaborator

kReview review

Verdict: no findings

No findings to address in the reviewed diff.

No meaningful correctness, security, or workflow regressions are provable from the supplied diff.

Reviewed by kReview at 67c13fbedd. This comment is conservative and based only on the PR diff and metadata.

Est. cost ~$2.65 total (425.5k in / 22.0k out) · this run ~$0.29 (50.4k in / 1.9k out) / gpt-5.6-sol.

## kReview review **Verdict:** no findings No findings to address in the reviewed diff. No meaningful correctness, security, or workflow regressions are provable from the supplied diff. _Reviewed by kReview at `67c13fbedd`. This comment is conservative and based only on the PR diff and metadata._ _Est. cost ~$2.65 total (425.5k in / 22.0k out) · this run ~$0.29 (50.4k in / 1.9k out) / gpt-5.6-sol._

kleb added 1 commit

2026-07-14 17:51:36 +02:00

Fix data race on live read-concurrency updates in PR feedback f388493cbb

kleb added 1 commit

2026-07-14 17:56:43 +02:00

Address PR feedback: emit poll error once, int64 concurrency, close rotating log in test 0acaabc1fd

kleb added 1 commit

2026-07-14 17:59:55 +02:00

Cover two log rotations in test (os.Rename replaces existing backup) 13a1a06726

kleb added 1 commit

2026-07-14 18:08:18 +02:00

Address PR feedback: 8-byte align priority-hint buffer; robust log-rotation recovery aedcfdaf20