AUSM

MIT · macOS 13+

Subscribe to your context.

Your activity, as typed events.

npm install @ausm/ausm

Node 18+. The native helpers ship prebuilt.

Watch it run, or click inside the Mac to try it.

draft.md

Study plan, week 3

Most context switches happened right after a copy, which suggests the clipboard is a good moment to offer help.

Anind K. Dey · Personal and Ubiquitous Computing, 2001

Understanding and Using Context

Context is any information that can be used to characterize the situation of an entity.

SensorsSelect, copy, type
    MaterialsApp › window › tab
    DetectorsRule fired, task named

      Quick start

      Save as app.mjs, run node app.mjs, then copy something.

      import { createRuntime } from '@ausm/ausm/runtime';
      
      const ausm = createRuntime();
      
      ausm.onCopyPasteEvent((e) => console.log(e.action, e.contentType));
      ausm.onTextSelectionEvent((e) => console.log(e.selectedText));
      await ausm.initialize();
      COPY TEXT
      Sensors

      Accessibility for whatever runs it: your terminal in development, your app once packaged. Screenshots also need Screen Recording.

      Materials

      Screen Recording, enableScreenStreaming and materialManager.enabled, and Python 3.9+ with:

      pip install -r node_modules/@ausm/device-macos-material-manager/py/requirements.txt
      Detectors

      Materials on, and a vision model: CEREBRAS_API_KEY, a Vertex project, or a local OpenAI-compatible server. Without one they stay off.

      A missing permission or Python package is only logged, as [AUSM] … on stderr. Nothing throws, and no events arrive.

      Native Mac app?

      Five sensors also ship as a Swift package that compiles into your app. No Node.

      .package(url: "https://github.com/foundation-interface/ausm.git", from: "0.1.0")
      import ScreenshotMonitorKit
      
      let shots = ScreenshotMonitor(options: .init())
      shots.onCapture = { capture in
        Task { @MainActor in show(capture.image) }
      }
      shots.start()

      Products: ScreenshotMonitorKit, CursorMonitorKit, ClipboardMonitorKit, WindowMonitorKit, TextInputMonitorKit.

      • Sensors only. Materials, memory and detectors live in the Node runtime.
      • The same code as npm. The Node runtime runs these kits as helper processes. Here they link into your app.
      • One path per sensor. A kit and its Node helper running together report every event twice.
      • Sign your app. It holds the Accessibility and Screen Recording grants, and macOS drops them on every rebuild without a stable signing identity.

      Built for coding agents

      One skill teaches Claude Code the API and the privacy rules.

      npx skills add foundation-interface/ausm

      Or and paste it into AGENTS.md.

      • Show what I copied in the last hour, grouped by app.
      • Write a detector that notices when I'm stuck on an error.