diff --git a/README.en.md b/README.en.md
index e9b41cd4..a0be4286 100644
--- a/README.en.md
+++ b/README.en.md
@@ -33,10 +33,10 @@ A **locally runnable version** repaired from the leaked Claude Code source, with
- `--print` headless mode for scripts and CI
- MCP server, plugin, and Skills support
- Custom API endpoint and model support ([Third-Party Models Guide](docs/en/guide/third-party-models.md))
-- **Computer Use desktop control** — [Guide](docs/en/features/computer-use.md)
- **Memory System** (cross-session persistent memory) — [Usage Guide](docs/memory/01-usage-guide.md)
- **Multi-Agent System** (agent orchestration, parallel tasks, Teams collaboration) — [Usage Guide](docs/agent/01-usage-guide.md) | [Implementation](docs/agent/02-implementation.md)
- **Skills System** (extensible capability plugins, custom workflows) — [Usage Guide](docs/skills/01-usage-guide.md) | [Implementation](docs/skills/02-implementation.md)
+- **Computer Use desktop control** — [Guide](docs/en/features/computer-use.md) | [Architecture](docs/en/features/computer-use-architecture.md)
- Fallback Recovery CLI mode (`CLAUDE_CODE_FORCE_RECOVERY_CLI=1 ./bin/claude-haha`)
---
@@ -136,10 +136,10 @@ export PATH="$HOME/path/to/claude-code-haha/bin:$PATH"
|------|------|
| [Environment Variables](docs/en/guide/env-vars.md) | Full env var reference and configuration methods |
| [Third-Party Models](docs/en/guide/third-party-models.md) | Using OpenAI / DeepSeek / Ollama and other non-Anthropic models |
-| [Computer Use](docs/en/features/computer-use.md) | Desktop control (screenshots, mouse, keyboard) |
| [Memory System](docs/memory/01-usage-guide.md) | Cross-session persistent memory usage and implementation |
| [Multi-Agent System](docs/agent/01-usage-guide.md) | Agent orchestration, parallel tasks and Teams collaboration |
| [Skills System](docs/skills/01-usage-guide.md) | Extensible capability plugins, custom workflows and conditional activation |
+| [Computer Use](docs/en/features/computer-use.md) | Desktop control (screenshots, mouse, keyboard) — [Architecture](docs/en/features/computer-use-architecture.md) |
| [Global Usage](docs/en/guide/global-usage.md) | Run claude-haha from any directory |
| [FAQ](docs/en/guide/faq.md) | Common error troubleshooting |
| [Source Fixes](docs/en/reference/fixes.md) | Fixes compared with the original leaked source |
diff --git a/README.md b/README.md
index c637e7c0..2e2e8d4d 100644
--- a/README.md
+++ b/README.md
@@ -33,10 +33,10 @@
- `--print` 无头模式（脚本/CI 场景）
- 支持 MCP 服务器、插件、Skills
- 支持自定义 API 端点和模型（[第三方模型使用指南](docs/guide/third-party-models.md)）
-- **Computer Use 桌面控制** — [使用指南](docs/features/computer-use.md)
- **记忆系统**（跨会话持久化记忆）— [使用指南](docs/memory/01-usage-guide.md)
- **多 Agent 系统**（多代理编排、并行任务、Teams 协作）— [使用指南](docs/agent/01-usage-guide.md) | [实现原理](docs/agent/02-implementation.md)
- **Skills 系统**（可扩展能力插件、自定义工作流）— [使用指南](docs/skills/01-usage-guide.md) | [实现原理](docs/skills/02-implementation.md)
+- **Computer Use 桌面控制** — [功能指南](docs/features/computer-use.md) | [架构解析](docs/features/computer-use-architecture.md)
- 降级 Recovery CLI 模式（`CLAUDE_CODE_FORCE_RECOVERY_CLI=1 ./bin/claude-haha`）
---
@@ -136,10 +136,10 @@ export PATH="$HOME/path/to/claude-code-haha/bin:$PATH"
|------|------|
| [环境变量](docs/guide/env-vars.md) | 完整环境变量参考和配置方式 |
| [第三方模型](docs/guide/third-party-models.md) | 接入 OpenAI / DeepSeek / Ollama 等非 Anthropic 模型 |
-| [Computer Use](docs/features/computer-use.md) | 桌面控制功能（截屏、鼠标、键盘） |
| [记忆系统](docs/memory/01-usage-guide.md) | 跨会话持久化记忆的使用与实现 |
| [多 Agent 系统](docs/agent/01-usage-guide.md) | 多代理编排、并行任务执行与 Teams 协作 |
| [Skills 系统](docs/skills/01-usage-guide.md) | 可扩展能力插件、自定义工作流与条件激活 |
+| [Computer Use](docs/features/computer-use.md) | 桌面控制功能（截屏、鼠标、键盘）— [架构解析](docs/features/computer-use-architecture.md) |
| [全局使用](docs/guide/global-usage.md) | 在任意目录启动 claude-haha |
| [常见问题](docs/guide/faq.md) | 常见错误排查 |
| [源码修复记录](docs/reference/fixes.md) | 相对于原始泄露源码的修复内容 |
diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts
index 7186b847..226844c9 100644
--- a/docs/.vitepress/config.mts
+++ b/docs/.vitepress/config.mts
@@ -11,12 +11,6 @@ const zhSidebar = [
{ text: '常见问题', link: '/guide/faq' },
],
},
- {
- text: '功能',
- items: [
- { text: 'Computer Use', link: '/features/computer-use' },
- ],
- },
{
text: '记忆系统',
collapsed: false,
@@ -45,6 +39,14 @@ const zhSidebar = [
{ text: '实现原理', link: '/skills/02-implementation' },
],
},
+ {
+ text: 'Computer Use',
+ collapsed: false,
+ items: [
+ { text: '功能指南', link: '/features/computer-use' },
+ { text: '架构解析', link: '/features/computer-use-architecture' },
+ ],
+ },
{
text: '参考',
collapsed: true,
@@ -66,12 +68,6 @@ const enSidebar = [
{ text: 'FAQ', link: '/en/guide/faq' },
],
},
- {
- text: 'Features',
- items: [
- { text: 'Computer Use', link: '/en/features/computer-use' },
- ],
- },
{
text: 'Memory System',
collapsed: false,
@@ -100,6 +96,14 @@ const enSidebar = [
{ text: 'Implementation', link: '/en/skills/02-implementation' },
],
},
+ {
+ text: 'Computer Use',
+ collapsed: false,
+ items: [
+ { text: 'Guide', link: '/en/features/computer-use' },
+ { text: 'Architecture', link: '/en/features/computer-use-architecture' },
+ ],
+ },
{
text: 'Reference',
collapsed: true,
diff --git a/docs/agent/01-usage-guide.md b/docs/agent/01-usage-guide.md
index 534c995e..6b003953 100644
--- a/docs/agent/01-usage-guide.md
+++ b/docs/agent/01-usage-guide.md
@@ -3,7 +3,7 @@
> 让 Claude Code 同时调度多个专业代理，并行处理复杂任务。
-多 Agent 系统 · 六种内置 Agent · 如何生成 Agent · 后台任务管理 · Agent Teams · 自定义 Agent · 权限模式 · 快速参考
+多 Agent 系统 · 六种内置 Agent · 如何生成 Agent · 后台任务管理 · Agent Teams · 自定义 Agent · 权限模式 · 快速参考
diff --git a/docs/agent/02-implementation.md b/docs/agent/02-implementation.md
index 19c9137f..f8256d88 100644
--- a/docs/agent/02-implementation.md
+++ b/docs/agent/02-implementation.md
@@ -3,7 +3,7 @@
> 深入剖析多 Agent 编排的架构设计、生成流程、上下文传递和协作机制。
-架构总览 · 生成流程 · 工具池系统 · 上下文传递 · Teams 内部机制 · 后台任务引擎 · DreamTask · Worktree 隔离 · 权限同步 · 生命周期数据流 · 源文件索引 · Feature Flags
+架构总览 · 生成流程 · 工具池系统 · 上下文传递 · Teams 内部机制 · 后台任务引擎 · DreamTask · Worktree 隔离 · 权限同步 · 生命周期数据流 · 源文件索引 · Feature Flags
diff --git a/docs/agent/03-agent-framework.md b/docs/agent/03-agent-framework.md
index d5239bf8..49ce689c 100644
--- a/docs/agent/03-agent-framework.md
+++ b/docs/agent/03-agent-framework.md
@@ -3,7 +3,7 @@
> 从源码视角剖析全球最流行 AI Code Editor 背后的 Agent 架构设计哲学。
-核心循环 · 提示词工程 · 工具系统 · 上下文管理 · 技能与插件 · 权限与安全 · 故障恢复 · 对比分析 · 成功之道
+核心循环 · 提示词工程 · 工具系统 · 上下文管理 · 技能与插件 · 权限与安全 · 故障恢复 · 对比分析 · 成功之道
diff --git a/docs/en/agent/01-usage-guide.md b/docs/en/agent/01-usage-guide.md
index 48ff198f..4bd23193 100644
--- a/docs/en/agent/01-usage-guide.md
+++ b/docs/en/agent/01-usage-guide.md
@@ -3,7 +3,7 @@
> Let Claude Code orchestrate multiple specialized agents to handle complex tasks in parallel.
-Multi-Agent System · Six Built-in Agents · Spawning Agents · Background Tasks · Agent Teams · Custom Agents · Permission Modes · Quick Reference
+Multi-Agent System · Six Built-in Agents · Spawning Agents · Background Tasks · Agent Teams · Custom Agents · Permission Modes · Quick Reference
diff --git a/docs/en/agent/02-implementation.md b/docs/en/agent/02-implementation.md
index b1b94ec5..5db0b49d 100644
--- a/docs/en/agent/02-implementation.md
+++ b/docs/en/agent/02-implementation.md
@@ -3,7 +3,7 @@
> A deep dive into the architecture, spawn flow, context passing, and collaboration mechanisms of multi-agent orchestration.
-Architecture · Spawn Flow · Tool Pool · Context Passing · Teams Internals · Task Engine · DreamTask · Worktree Isolation · Permission Sync · Lifecycle Data Flow · Source Index · Feature Flags
+Architecture · Spawn Flow · Tool Pool · Context Passing · Teams Internals · Task Engine · DreamTask · Worktree Isolation · Permission Sync · Lifecycle Data Flow · Source Index · Feature Flags
diff --git a/docs/en/agent/03-agent-framework.md b/docs/en/agent/03-agent-framework.md
index bf273623..2cbd172b 100644
--- a/docs/en/agent/03-agent-framework.md
+++ b/docs/en/agent/03-agent-framework.md
@@ -3,7 +3,7 @@
> Deconstructing the architecture behind the world's most popular AI code editor — from source code to design philosophy.
-Core Loop · Prompt Engineering · Tool System · Context Management · Skills & Plugins · Permissions · Recovery · vs LangChain · Why It Works
+Core Loop · Prompt Engineering · Tool System · Context Management · Skills & Plugins · Permissions · Recovery · vs LangChain · Why It Works
diff --git a/docs/en/features/computer-use-architecture.md b/docs/en/features/computer-use-architecture.md
new file mode 100644
index 00000000..1232552e
--- /dev/null
+++ b/docs/en/features/computer-use-architecture.md
@@ -0,0 +1,602 @@
+# Computer Use Architecture Deep Dive
+
+> A deep dive into the Computer Use implementation: from MCP tool definitions to Python Bridge, from 9-layer security gates to feature flag bypasses.
+
+
+Patch Environment ·
+Layered Architecture ·
+MCP Tool Layer ·
+Security Gates ·
+Python Bridge ·
+Interaction Loop ·
+Source File Index
+
+
+
+
+---
+
+## 1. Patch Environment Overview
+
+The original Claude Code's Computer Use feature (internal codename **Chicago**) depends on three components that are not publicly available:
+
+| Component | Purpose | Availability |
+|-----------|---------|-------------|
+| `@ant/computer-use-swift` | Screenshots, display enumeration | Anthropic private npm package |
+| `@ant/computer-use-input` | Mouse/keyboard simulation | Anthropic private npm package |
+| GrowthBook remote config | Feature flags, kill switch | Anthropic internal service |
+
+Our approach: **preserve the original MCP tool definitions and security mechanisms, only replace the execution layer and feature flag controls**.
+
+
+
+### What We Changed
+
+```
+Original Claude Code Claude Code Haha (Patched)
+──────────────────── ─────────────────────────
+@ant/computer-use-swift ──replaced──→ Python Bridge (mac_helper.py)
+@ant/computer-use-input ──replaced──→ pyautogui + pyobjc
+GrowthBook feature flags ──bypassed──→ gates.ts hardcoded return true
+Subscription check (Max/Pro) ──bypassed──→ getChicagoEnabled() = true
+Build macro CHICAGO_MCP ──replaced──→ true
+isDefaultDisabledBuiltin ──modified──→ returns false
+```
+
+### What We Kept Intact
+
+- **MCP tool definitions** (24 tools with their schema and parameter validation)
+- **9-layer security gates** (TCC permissions, app allowlist, permission tiers, pixel validation, etc.)
+- **App classification system** (191 bundle IDs categorized with permission mappings)
+- **Session context management** (global lock, screenshot cache, state synchronization)
+- **Keyboard shortcut blocklist** (system-level dangerous operation interception)
+
+### Feature Flag Bypass Details
+
+The original code uses three layers of gating to restrict Computer Use access:
+
+```typescript
+// Original code (simplified)
+function getChicagoEnabled(): boolean {
+ // Layer 1: GrowthBook remote config
+ const config = getDynamicConfig('tengu_malort_pedway')
+ // Layer 2: Subscription check
+ const hasSubscription = hasRequiredSubscription() // Max/Pro
+ // Layer 3: Build-time macro
+ return feature('CHICAGO_MCP') && config.enabled && hasSubscription
+}
+```
+
+Our modification:
+
+```typescript
+// gates.ts — our change
+export function getChicagoEnabled(): boolean {
+ return true // ← all three gate layers bypassed
+}
+```
+
+**Note**: Sub-gates (pixelValidation, mouseAnimation, etc.) still retain the original logic and can be controlled via configuration.
+
+---
+
+## 2. Layered Architecture
+
+Computer Use uses a **6-layer architecture** with clear responsibilities and boundaries:
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│ Layer 1 — MCP Tool Interface │
+│ tools.ts: 24 tool schemas + parameter validation │
+│ buildComputerUseTools() → MCP Tool Definition │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 2 — Tool Dispatch & Security Control │
+│ toolCalls.ts: handleToolCall() + 9 security gates │
+│ deniedApps.ts: 191 app classifications + permission tiers │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 3 — MCP Server Binding │
+│ mcpServer.ts: session context + global lock + screenshot │
+│ bindSessionContext() → per-call overrides │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 4 — CLI Integration │
+│ wrapper.tsx: permission dialogs + state read/write │
+│ setup.ts: MCP config initialization │
+│ gates.ts: feature flags (bypassed) │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 5 — Python Bridge IPC [PATCH] │
+│ pythonBridge.ts: venv mgmt + JSON RPC + error handling │
+│ callPythonHelper(command, payload) → T │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 6 — Python Runtime Execution [PATCH] │
+│ mac_helper.py: pyautogui + mss + pyobjc │
+│ 660 lines of Python implementing all system interactions │
+└─────────────────────────────────────────────────────────────┘
+```
+
+Layers marked **[PATCH]** are our replaced/new code. **All other layers** are preserved from the original Claude Code.
+
+### Why This Layering?
+
+| Layer | Source | Reusability |
+|-------|--------|------------|
+| Layer 1-2 | `vendor/computer-use-mcp/` | Platform-agnostic, reusable for Electron, Web hosts |
+| Layer 3 | `vendor/computer-use-mcp/` | Platform-agnostic, standard MCP protocol |
+| Layer 4 | `utils/computerUse/` | CLI-specific, bound to app state |
+| Layer 5-6 | `utils/computerUse/` + `runtime/` | macOS-specific, Python implementation |
+
+---
+
+## 3. MCP Tool Layer
+
+### 24 Tools Overview
+
+Computer Use exposes 24 tools to the model via MCP (Model Context Protocol):
+
+| Category | Tools | Permission Tier |
+|----------|-------|----------------|
+| **Permission** | `request_access`, `list_granted_applications` | None required |
+| **Screenshot** | `screenshot`, `zoom` | read |
+| **Mouse Click** | `left_click`, `double_click`, `triple_click` | click |
+| **Mouse Advanced** | `right_click`, `middle_click`, `left_click_drag` | full |
+| **Mouse Move** | `mouse_move`, `cursor_position`, `scroll` | click |
+| **Mouse Low-level** | `left_mouse_down`, `left_mouse_up` | full |
+| **Keyboard** | `type`, `key`, `hold_key` | full |
+| **Application** | `open_application`, `switch_display` | full |
+| **Clipboard** | `read_clipboard`, `write_clipboard` | full |
+| **Batch** | `computer_batch` | Inherits from sub-operations |
+| **Wait** | `wait` | None required |
+
+### Coordinate System
+
+The model interacts with the screen through two coordinate modes:
+
+```
+pixels mode (default):
+ Model sees screenshot size (1176 x 784)
+ Model outputs coordinate [588, 392]
+ scaleCoord() conversion:
+ x_logical = (588 * displayWidth / 1176) + originX
+ y_logical = (392 * displayHeight / 784) + originY
+
+normalized_0_100 mode:
+ Model outputs coordinate [50, 50] (percentage)
+ scaleCoord() conversion:
+ x_logical = (50 / 100) * displayWidth + originX
+ y_logical = (50 / 100) * displayHeight + originY
+```
+
+Screenshot dimensions are calculated by `imageResize.ts` to ensure:
+- Long edge <= 1568 pixels
+- Token budget <= 1568 (vision encoder at 28px/token)
+- Aspect ratio preserved
+
+### App Classification System
+
+`deniedApps.ts` precisely classifies 191 applications:
+
+**Browsers** (55 bundle IDs) -> tier `read`
+```
+Safari, Chrome, Firefox, Arc, Edge, Opera, Brave, Vivaldi...
+Reason: browser operations should use Chrome MCP, not blind clicking
+```
+
+**Terminals** (102 bundle IDs) -> tier `click`
+```
+Terminal, iTerm2, VS Code, Cursor, JetBrains IDEs, Xcode...
+Reason: terminal operations should use Bash Tool, limited to click only
+```
+
+**Trading** (34 bundle IDs) -> tier `read`
+```
+Webull, Fidelity, Interactive Brokers, Binance, Kraken...
+Reason: financial operations are extremely high-risk, screenshot only
+```
+
+**Completely Blocked** (policy deny list):
+```
+Netflix, Spotify, Apple Music, Kindle...
+Reason: copyright compliance, rejected without permission dialog
+```
+
+---
+
+## 4. Security Gate System
+
+Every input action (click, keyboard, drag) must pass through **9 security gates** before execution:
+
+
+
+### Gate Details
+
+#### Gate 1: Kill Switch
+```typescript
+if (adapter.isDisabled()) return errorResult("Computer Use is disabled")
+```
+Reads `getChicagoEnabled()` — always returns `true` in patched version.
+
+#### Gate 2: TCC Permission Check
+```typescript
+await adapter.ensureOsPermissions()
+// → Python: check_permissions
+// → Accessibility: osascript "tell System Events..."
+// → Screen Recording: CGDisplayCaptureDisplay()
+```
+Reports error if macOS Accessibility or Screen Recording permissions are missing.
+
+#### Gate 3: Global Mutex Lock
+```typescript
+await tryAcquireComputerUseLock(sessionId)
+// File lock: ~/.claude/computer-use.lock
+// JSON: { sessionId, pid, acquiredAt }
+```
+Ensures only one Claude session can control the computer at a time. Supports stale PID recovery.
+
+#### Gate 4: Hide Non-Allowlisted Apps
+```typescript
+await executor.prepareForAction(allowlistBundleIds)
+// Hide all app windows not in the allowlist
+// Ensures screenshots only contain authorized apps
+```
+
+#### Gate 5: Frontmost App Check
+```typescript
+const frontmost = await executor.getFrontmostApp()
+if (!allowlist.includes(frontmost.bundleId)) {
+ return errorResult("Application not authorized")
+}
+```
+Even after passing the allowlist, verifies the current foreground app is authorized.
+
+#### Gate 6: Permission Tier Check
+
+Three-tier permission model:
+
+| Tier | Allowed Operations | Prohibited Operations |
+|------|-------------------|----------------------|
+| `read` | Screenshot viewing | Any input action |
+| `click` | Left click, scroll | Right-click, drag, keyboard input |
+| `full` | All operations | No restrictions |
+
+```typescript
+function tierSatisfies(tier: CuAppPermTier, required: ActionKind): boolean {
+ const order = { read: 0, click: 1, full: 2 }
+ return order[tier] >= order[required]
+}
+```
+
+**Anti-subversion**: If permissions are insufficient, the response includes a `TIER_ANTI_SUBVERSION` hint to prevent the model from bypassing restrictions via AppleScript or System Events.
+
+#### Gate 7: Clipboard Guard
+
+**Threat model**:
+```
+1. Agent calls write_clipboard("rm -rf /")
+2. Switches to Terminal (click-tier allows clicking)
+3. Model clicks Terminal's paste button
+4. Malicious command executed
+```
+
+**Protection**:
+```
+When click-tier app becomes frontmost:
+ → Save current clipboard content (stash)
+ → Clear clipboard
+ → Re-clear after each operation
+
+When non-click-tier app becomes frontmost:
+ → Restore original clipboard content
+```
+
+#### Gate 8: Pixel Validation (Staleness Guard)
+
+```
+Last screenshot Current actual screen
+┌────────────┐ ┌────────────┐
+│ Button A │ │ Dialog │ ← UI has changed
+│ [756,342] │ │ Confirm? │
+└────────────┘ └────────────┘
+
+Validation: sample 9x9 pixel grid at [756,342]
+ → Compare last screenshot vs live screenshot pixels
+ → Different → reject click + prompt to re-screenshot
+ → Same → allow click
+```
+
+**Note**: In patched version, `pixelValidation` is off by default (`hostAdapter.cropRawPatch()` returns null).
+
+#### Gate 9: System Shortcut Interception
+
+`keyBlocklist.ts` blocks dangerous shortcuts:
+
+| Shortcut | Dangerous Action |
+|----------|-----------------|
+| `Cmd+Q` | Quit application |
+| `Shift+Cmd+Q` | Log out |
+| `Option+Cmd+Esc` | Force quit dialog |
+| `Cmd+Tab` | App switcher |
+| `Cmd+Space` | Spotlight |
+| `Ctrl+Cmd+Q` | Lock screen |
+
+---
+
+## 5. Python Bridge Mechanism
+
+
+
+### Architecture Design
+
+Original Claude Code uses compiled Swift native modules (.node NAPI plugins) to directly call macOS APIs. We replaced this layer with **Python subprocess + JSON RPC**:
+
+```
+TypeScript (Bun Runtime) Python (venv)
+┌────────────────────┐ ┌────────────────────┐
+│ executor.ts │ │ mac_helper.py │
+│ │ execFile() │ │
+│ callPythonHelper │ ──────────────→ │ main() │
+│ ('click', │ command + │ ├─ parse argv │
+│ {x:756,y:342}) │ --payload JSON │ ├─ dispatch() │
+│ │ │ └─ click() │
+│ ← JSON.parse ──── │ ←────────────── │ pyautogui │
+│ {ok:true, │ stdout JSON │ │
+│ result:true} │ │ json_output(...) │
+└────────────────────┘ └────────────────────┘
+```
+
+### Bootstrap Flow
+
+First call to `callPythonHelper()` automatically completes environment setup:
+
+```
+ensureBootstrapped()
+ │
+ ├─ Check if .runtime/venv/bin/python3 exists
+ │ └─ Missing → python3 -m venv .runtime/venv/
+ │
+ ├─ Check if pip is available
+ │ └─ Missing → python3 -m ensurepip --upgrade
+ │
+ ├─ Compute SHA256 of runtime/requirements.txt
+ │ └─ Compare with .runtime/requirements.sha256
+ │ └─ Different → pip install -r requirements.txt
+ │ Write new SHA256 hash
+ │ └─ Same → skip installation
+ │
+ └─ Ready, return venv Python path
+```
+
+**Dependencies** (`runtime/requirements.txt`):
+
+| Library | Purpose |
+|---------|---------|
+| `mss` | High-performance screen capture |
+| `Pillow` | JPEG encoding and image processing |
+| `pyautogui` | Mouse click, keyboard input |
+| `pyobjc-core` | macOS Objective-C bridge |
+| `pyobjc-framework-Cocoa` | NSWorkspace (app management), NSPasteboard (clipboard) |
+| `pyobjc-framework-Quartz` | CGDisplay (monitors), CGWindow (window list) |
+
+### Command Mapping
+
+`mac_helper.py` (660 lines) implements the following commands:
+
+| Command | Python Implementation | Return Value |
+|---------|----------------------|-------------|
+| `screenshot` | `mss.grab()` + `PIL.Image` JPEG encoding | `{base64, width, height, displayWidth, displayHeight}` |
+| `zoom` | `mss.grab(region)` region capture | `{base64, width, height}` |
+| `click` | `pyautogui.moveTo()` + `pyautogui.click()` | `true` |
+| `key` | `pyautogui.hotkey()` / `pyautogui.press()` | `true` |
+| `type` | `pyautogui.write(interval=0.008)` | `true` |
+| `drag` | `pyautogui.dragTo(duration=0.2)` | `true` |
+| `scroll` | `pyautogui.scroll()` / `pyautogui.hscroll()` | `true` |
+| `hold_key` | `pyautogui.keyDown()` + `sleep` + `pyautogui.keyUp()` | `true` |
+| `frontmost_app` | `NSWorkspace.frontmostApplication()` | `{bundleId, displayName}` |
+| `list_displays` | `CGGetActiveDisplayList()` + `CGDisplayBounds()` | `[DisplayGeometry...]` |
+| `open_app` | `NSWorkspace.launchApplicationAtURL_options_` | `void` |
+| `read_clipboard` | `NSPasteboard.stringForType_()` | `string` |
+| `write_clipboard` | `NSPasteboard.setString_forType_()` | `void` |
+| `check_permissions` | `osascript` + `CGDisplayCaptureDisplay` | `{accessibility, screenRecording}` |
+
+### Error Handling
+
+```python
+# mac_helper.py unified error handling
+def main():
+ try:
+ result = dispatch(command, payload)
+ json_output({"ok": True, "result": result})
+ except Exception as e:
+ error_output({"ok": False, "error": {"message": str(e)}})
+```
+
+TypeScript side:
+```typescript
+// pythonBridge.ts
+const parsed = JSON.parse(stdout)
+if (!parsed.ok) {
+ throw new Error(parsed.error.message) // → MCP tool error
+}
+return parsed.result
+```
+
+---
+
+## 6. Screenshot-Analyze-Act Loop
+
+A complete Computer Use interaction consists of multiple **screenshot-analyze-act** cycles:
+
+### Typical Interaction Flow
+
+```
+User: "Open NetEase Music and search for a song"
+
+┌─ Cycle 1: Discover and open application ─────────────────┐
+│ │
+│ Step 1: request_access │
+│ → Permission dialog, user authorizes allowed apps │
+│ → Set allowedApps, grantFlags │
+│ │
+│ Step 2: screenshot │
+│ → Full screen capture → JPEG encode → base64 │
+│ → Cache screenshot dimensions (lastScreenshotDims) │
+│ → Return to model │
+│ │
+│ Step 3: Model analyzes screenshot │
+│ → "NetEase Music not on desktop, need to open it" │
+│ → Decides to call open_application │
+│ │
+│ Step 4: open_application("com.netease.163music") │
+│ → Gates 1-9 all pass │
+│ → Python: NSWorkspace.launchApplicationAtURL_() │
+│ │
+└───────────────────────────────────────────────────────────┘
+
+┌─ Cycle 2: Locate search box ─────────────────────────────┐
+│ │
+│ Step 5: screenshot │
+│ → Full screen (app now open) │
+│ → Update lastScreenshotDims │
+│ │
+│ Step 6: Model analyzes screenshot │
+│ → Vision identifies search box at (756, 342) │
+│ → Decides to click search box │
+│ │
+│ Step 7: left_click({coordinate: [756, 342]}) │
+│ → Gate 4: Hide non-allowlisted apps │
+│ → Gate 5: Frontmost is NetEase Music ✓ │
+│ → Gate 6: tier=full >= click ✓ │
+│ → scaleCoord(756, 342) → screen coordinates │
+│ → Python: pyautogui.click(x_logical, y_logical) │
+│ │
+└───────────────────────────────────────────────────────────┘
+
+┌─ Cycle 3: Type search query ─────────────────────────────┐
+│ │
+│ Step 8: type({text: "my favorite song"}) │
+│ → Gate 6: tier=full >= keyboard ✓ │
+│ → Python: pyautogui.write("...", interval=0.008) │
+│ │
+│ Step 9: screenshot │
+│ → Confirm search results appeared │
+│ │
+│ Step 10: left_click({coordinate: [...]}) │
+│ → Click target song │
+│ │
+└───────────────────────────────────────────────────────────┘
+```
+
+### Coordinate Conversion
+
+**What the model sees** vs **actual screen** are different coordinate spaces:
+
+```
+Physical screen (2560 x 1600, Retina 2x)
+ ├─ Logical size: 1280 x 800
+ └─ Physical pixels: 2560 x 1600
+
+After imageResize:
+ ├─ Scaled size: 1176 x 735 (≤1568px budget)
+ └─ This is the screenshot size the model "sees"
+
+Model outputs coordinate: [588, 368] (in image space)
+
+scaleCoord conversion:
+ x_logical = (588 / 1176) * 1280 + originX = 640
+ y_logical = (368 / 735) * 800 + originY = 400
+
+Python executes:
+ pyautogui.moveTo(640, 400) ← logical coords (macOS handles Retina)
+```
+
+### Screenshot Cache & State Sync
+
+```
+bindSessionContext closure
+ │
+ ├─ lastScreenshot (in-memory)
+ │ ├─ base64: JPEG data (for pixel validation)
+ │ ├─ width/height: model-visible dimensions
+ │ └─ displayWidth/displayHeight/originX/originY: display geometry
+ │
+ └─ AppState.computerUseMcpState (persisted)
+ ├─ allowedApps: AppGrant[] — authorized app list
+ ├─ grantFlags: {...} — clipboard/system shortcut permissions
+ ├─ selectedDisplayId?: number — selected display
+ ├─ lastScreenshotDims?: {...} — screenshot geometry (survives restart)
+ └─ hiddenDuringTurn?: Set — apps hidden this turn
+```
+
+---
+
+## 7. Key Source File Index
+
+### vendor/computer-use-mcp/ (Original Code Layer)
+
+| File | Lines | Responsibility |
+|------|-------|---------------|
+| `types.ts` | 622 | Permission model, session context, all type definitions |
+| `tools.ts` | 707 | 24 MCP tool schemas and parameter validation |
+| `toolCalls.ts` | 1600+ | **Core**: tool dispatch, 9 security gates, permission flow |
+| `deniedApps.ts` | 554 | 191 app classifications (browser/terminal/trading) and permission mappings |
+| `sentinelApps.ts` | 44 | Sensitive app warning labels (shell/filesystem/system_settings) |
+| `mcpServer.ts` | 314 | MCP server factory, session context binding, global lock |
+| `pixelCompare.ts` | 172 | Click target pixel validation (staleness guard) |
+| `imageResize.ts` | 109 | Screenshot dimension calculation (API image transcoder) |
+| `keyBlocklist.ts` | 154 | System shortcut interception (Cmd+Q, Cmd+Tab, etc.) |
+| `executor.ts` | 101 | ComputerExecutor interface definition |
+| `subGates.ts` | 20 | Feature flag sub-gate presets |
+
+### utils/computerUse/ (CLI Adaptation Layer)
+
+| File | Lines | Responsibility | Patched? |
+|------|-------|---------------|----------|
+| `executor.ts` | 231 | ComputerExecutor Python bridge implementation | Yes, rewritten |
+| `pythonBridge.ts` | 111 | Python subprocess management, venv bootstrap, JSON RPC | Yes, new |
+| `hostAdapter.ts` | 54 | HostAdapter implementation (permission checks, flag reading) | Partially |
+| `gates.ts` | 51 | GrowthBook feature flags (`getChicagoEnabled` bypass) | Yes, modified |
+| `wrapper.tsx` | 300+ | Session context construction, permission dialogs, lock management | Unchanged |
+| `setup.ts` | 54 | MCP config initialization | Unchanged |
+| `computerUseLock.ts` | 216 | Global file lock (`~/.claude/computer-use.lock`) | Unchanged |
+| `common.ts` | 62 | Constants (server name, bundle ID) | Unchanged |
+| `cleanup.ts` | — | Turn-end cleanup (app restore, clipboard restore) | Unchanged |
+| `toolRendering.tsx` | — | Tool result UI rendering | Unchanged |
+
+### runtime/ (Python Runtime)
+
+| File | Lines | Responsibility | Patched? |
+|------|-------|---------------|----------|
+| `mac_helper.py` | 660 | All system interactions in Python | Yes, new |
+| `requirements.txt` | 6 | Python dependency declarations | Yes, new |
+
+---
+
+## 8. Design Trade-offs
+
+### Why Python Bridge?
+
+| Dimension | Native Swift (.node) | Python Bridge |
+|-----------|---------------------|---------------|
+| **Performance** | ~0ms (in-process call) | ~50-100ms (subprocess startup) |
+| **Readability** | Not readable after compilation | 660 lines of clear Python |
+| **Modifiability** | Requires Swift build environment | Edit .py files directly |
+| **Dependencies** | Specific Bun version NAPI | Any Python 3.8+ |
+| **Cross-platform** | macOS only | pyautogui/mss are natively cross-platform |
+| **User experience** | Imperceptible | Imperceptible (model thinking takes 2-5s) |
+
+**Conclusion**: The 50-100ms extra latency is completely negligible in Computer Use scenarios — the model typically takes 2-5 seconds for screenshot analysis and decision-making, so users won't notice the additional 100ms for underlying operations.
+
+### Approaches We Tried But Abandoned
+
+**Approach 1: Extract native .node modules**
+- Successfully extracted `computer-use-swift.node` (ARM64 424KB) from the Claude Code binary
+- Synchronous methods worked, but **Swift async method continuations never resumed**
+- Root cause: .node files compiled for Claude Code's built-in Bun, incompatible with user's Bun version
+
+**Approach 2: Empty stub packages**
+- Code compiled but all operations threw errors — no actual execution capability
+
+---
+
+## Related Documentation
+
+- [Computer Use Guide](./computer-use.md) — Usage, quick start, environment variables
+- [Source Fixes](/en/reference/fixes) — Detailed records of other fixes and patches
diff --git a/docs/en/features/images/01-computer-use-architecture.jpg b/docs/en/features/images/01-computer-use-architecture.jpg
new file mode 100644
index 00000000..65f331b8
Binary files /dev/null and b/docs/en/features/images/01-computer-use-architecture.jpg differ
diff --git a/docs/en/features/images/02-computer-use-security-gates.jpg b/docs/en/features/images/02-computer-use-security-gates.jpg
new file mode 100644
index 00000000..95a29b90
Binary files /dev/null and b/docs/en/features/images/02-computer-use-security-gates.jpg differ
diff --git a/docs/en/features/images/03-computer-use-python-bridge.jpg b/docs/en/features/images/03-computer-use-python-bridge.jpg
new file mode 100644
index 00000000..c8cbc041
Binary files /dev/null and b/docs/en/features/images/03-computer-use-python-bridge.jpg differ
diff --git a/docs/en/features/images/04-computer-use-patch.jpg b/docs/en/features/images/04-computer-use-patch.jpg
new file mode 100644
index 00000000..176be531
Binary files /dev/null and b/docs/en/features/images/04-computer-use-patch.jpg differ
diff --git a/docs/en/memory/01-usage-guide.md b/docs/en/memory/01-usage-guide.md
index 46ab5ac8..9c06b01c 100644
--- a/docs/en/memory/01-usage-guide.md
+++ b/docs/en/memory/01-usage-guide.md
@@ -3,7 +3,7 @@
> Let Claude Code remember who you are, what you prefer, and what's happening in your project across sessions.
-Memory System · Four Memory Types · Trigger Saving · Storage Location · Manage Memories · Lifecycle · Quick Reference
+Memory System · Four Memory Types · Trigger Saving · Storage Location · Manage Memories · Lifecycle · Quick Reference
diff --git a/docs/en/memory/02-implementation.md b/docs/en/memory/02-implementation.md
index ddf2da7c..788188c7 100644
--- a/docs/en/memory/02-implementation.md
+++ b/docs/en/memory/02-implementation.md
@@ -3,7 +3,7 @@
> From system prompt injection to background auto-extraction, dissecting every technical detail of the memory system.
-Architecture · Path Resolution · Prompt Injection · Auto-Extraction · Intelligent Retrieval · Memory Scanning · Agent Memory · Team Sync · Constants · Data Flow
+Architecture · Path Resolution · Prompt Injection · Auto-Extraction · Intelligent Retrieval · Memory Scanning · Agent Memory · Team Sync · Constants · Data Flow
diff --git a/docs/en/memory/03-autodream.md b/docs/en/memory/03-autodream.md
index fb983eb4..cba2bef0 100644
--- a/docs/en/memory/03-autodream.md
+++ b/docs/en/memory/03-autodream.md
@@ -3,7 +3,7 @@
> Claude "dreams" -- silently reviewing recent sessions in the background to consolidate, update, and prune memories, much like the human brain organizes memories during sleep.
-AutoDream · Trigger Conditions · Consolidation Process · Security · UI · Configuration · Comparison · Source Code
+AutoDream · Trigger Conditions · Consolidation Process · Security · UI · Configuration · Comparison · Source Code
diff --git a/docs/en/skills/01-usage-guide.md b/docs/en/skills/01-usage-guide.md
index f29cc6ca..a7e2b9d2 100644
--- a/docs/en/skills/01-usage-guide.md
+++ b/docs/en/skills/01-usage-guide.md
@@ -3,7 +3,7 @@
> Skills are the extensible capability engine of Claude Code, allowing you to define custom automated workflows using Markdown files.
-What Are Skills · Six Sources · Definition Format · Invocation · Execution Context · Conditional Activation · Permissions · Quick Reference
+What Are Skills · Six Sources · Definition Format · Invocation · Execution Context · Conditional Activation · Permissions · Quick Reference
diff --git a/docs/en/skills/02-implementation.md b/docs/en/skills/02-implementation.md
index 78ecd02d..cd559605 100644
--- a/docs/en/skills/02-implementation.md
+++ b/docs/en/skills/02-implementation.md
@@ -3,7 +3,7 @@
> A deep dive into how Skills are discovered, loaded, injected, executed, and managed.
-Architecture · Discovery & Loading · Frontmatter Parsing · Injection · Execution Engine · Fork Execution · Conditional Activation · Hook Integration · Permission System · Complete Lifecycle · Source Index
+Architecture · Discovery & Loading · Frontmatter Parsing · Injection · Execution Engine · Fork Execution · Conditional Activation · Hook Integration · Permission System · Complete Lifecycle · Source Index
diff --git a/docs/features/computer-use-architecture.md b/docs/features/computer-use-architecture.md
new file mode 100644
index 00000000..d502d506
--- /dev/null
+++ b/docs/features/computer-use-architecture.md
@@ -0,0 +1,607 @@
+# Computer Use 架构深度解析
+
+> 深入解析 Computer Use 功能的底层实现：从 MCP 工具定义到 Python Bridge，从 9 层安全关卡到灰度控制绕过。
+
+
+补丁环境 ·
+分层架构 ·
+MCP 工具层 ·
+安全关卡 ·
+Python Bridge ·
+交互闭环 ·
+源文件索引
+
+
+
+
+---
+
+## 一、补丁环境总览
+
+Claude Code 原版的 Computer Use 功能（内部代号 **Chicago**）依赖三个不可公开获取的组件：
+
+| 组件 | 作用 | 获取方式 |
+|------|------|----------|
+| `@ant/computer-use-swift` | 屏幕截图、显示器枚举 | Anthropic 私有 npm 包 |
+| `@ant/computer-use-input` | 鼠标/键盘模拟 | Anthropic 私有 npm 包 |
+| GrowthBook 远程配置 | 灰度控制、功能开关 | Anthropic 内部服务 |
+
+我们的方案是：**保留原始 MCP 工具定义和安全机制不变，仅替换底层执行层和灰度控制**。
+
+
+
+### 我们做了什么
+
+```
+原始 Claude Code Claude Code Haha (补丁版)
+───────────────── ─────────────────────────
+@ant/computer-use-swift ──替换为──→ Python Bridge (mac_helper.py)
+@ant/computer-use-input ──替换为──→ pyautogui + pyobjc
+GrowthBook 灰度控制 ──绕过──→ gates.ts 硬编码 return true
+订阅检查 (Max/Pro) ──绕过──→ getChicagoEnabled() = true
+编译宏 CHICAGO_MCP ──替换为──→ true
+isDefaultDisabledBuiltin ──修改──→ 返回 false
+```
+
+### 我们没有改什么
+
+- **MCP 工具定义**（24 个工具的 schema 和参数校验）
+- **9 层安全关卡**（TCC 权限、应用白名单、权限等级、像素验证等）
+- **应用分类系统**（191 个 bundle ID 的分类和权限映射）
+- **会话上下文管理**（全局锁、截图缓存、状态同步）
+- **键盘快捷键阻止列表**（系统级危险操作拦截）
+
+### 灰度绕过细节
+
+原始代码通过三层门控限制 Computer Use 的访问：
+
+```typescript
+// 原始代码（简化）
+function getChicagoEnabled(): boolean {
+ // 层1：GrowthBook 远程配置
+ const config = getDynamicConfig('tengu_malort_pedway')
+ // 层2：订阅检查
+ const hasSubscription = hasRequiredSubscription() // Max/Pro
+ // 层3：编译时宏
+ return feature('CHICAGO_MCP') && config.enabled && hasSubscription
+}
+```
+
+我们的处理：
+
+```typescript
+// gates.ts — 我们的修改
+export function getChicagoEnabled(): boolean {
+ return true // ← 三层门控全部绕过
+}
+```
+
+**注意**：子开关（pixelValidation、mouseAnimation 等）仍然保留原始逻辑，可通过配置控制。
+
+---
+
+## 二、分层架构
+
+Computer Use 采用 **6 层架构**，每层职责清晰、边界明确：
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│ Layer 1 — MCP 工具接口层 │
+│ tools.ts: 24 个工具 schema + 参数校验 │
+│ buildComputerUseTools() → MCP Tool Definition │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 2 — 工具调度与安全控制层 │
+│ toolCalls.ts: handleToolCall() + 9 层安全关卡 │
+│ deniedApps.ts: 191 个应用分类 + 权限等级 │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 3 — MCP 服务器绑定层 │
+│ mcpServer.ts: 会话上下文 + 全局锁 + 截图缓存 │
+│ bindSessionContext() → per-call overrides │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 4 — CLI 集成层 │
+│ wrapper.tsx: 权限对话框 + 状态读写 │
+│ setup.ts: MCP 配置初始化 │
+│ gates.ts: 灰度控制（已绕过） │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 5 — Python Bridge 进程通信层 [补丁] │
+│ pythonBridge.ts: venv 管理 + JSON RPC + 错误处理 │
+│ callPythonHelper(command, payload) → T │
+├─────────────────────────────────────────────────────────────┤
+│ Layer 6 — Python 运行时执行层 [补丁] │
+│ mac_helper.py: pyautogui + mss + pyobjc │
+│ 660 行 Python 实现所有系统交互 │
+└─────────────────────────────────────────────────────────────┘
+```
+
+**标 `[补丁]` 的层**是我们替换/新增的代码，**其余层完全保留**原始 Claude Code 的逻辑。
+
+### 为什么这样分层？
+
+| 层 | 来源 | 可复用性 |
+|----|------|----------|
+| Layer 1-2 | `vendor/computer-use-mcp/` | 平台无关，可用于 Electron、Web 等宿主 |
+| Layer 3 | `vendor/computer-use-mcp/` | 平台无关，MCP 标准协议 |
+| Layer 4 | `utils/computerUse/` | CLI 专用，绑定应用状态 |
+| Layer 5-6 | `utils/computerUse/` + `runtime/` | macOS 专用，Python 实现 |
+
+---
+
+## 三、MCP 工具层
+
+### 24 个工具一览
+
+Computer Use 通过 MCP（Model Context Protocol）暴露 24 个工具给模型：
+
+| 类别 | 工具 | 权限等级 |
+|------|------|----------|
+| **权限** | `request_access`, `list_granted_applications` | 无需权限 |
+| **截屏** | `screenshot`, `zoom` | read |
+| **鼠标点击** | `left_click`, `double_click`, `triple_click` | click |
+| **鼠标高级** | `right_click`, `middle_click`, `left_click_drag` | full |
+| **鼠标移动** | `mouse_move`, `cursor_position`, `scroll` | click |
+| **鼠标底层** | `left_mouse_down`, `left_mouse_up` | full |
+| **键盘** | `type`, `key`, `hold_key` | full |
+| **应用** | `open_application`, `switch_display` | full |
+| **剪贴板** | `read_clipboard`, `write_clipboard` | full |
+| **批量** | `computer_batch` | 继承子操作 |
+| **等待** | `wait` | 无需权限 |
+
+### 坐标系统
+
+模型通过两种坐标模式与屏幕交互：
+
+```
+pixels 模式（默认）：
+ 模型看到截图尺寸 (1176 x 784)
+ 模型输出坐标 [588, 392]
+ scaleCoord() 转换：
+ x_logical = (588 * displayWidth / 1176) + originX
+ y_logical = (392 * displayHeight / 784) + originY
+
+normalized_0_100 模式：
+ 模型输出坐标 [50, 50]（百分比）
+ scaleCoord() 转换：
+ x_logical = (50 / 100) * displayWidth + originX
+ y_logical = (50 / 100) * displayHeight + originY
+```
+
+截图尺寸经过 `imageResize.ts` 计算，确保：
+- 长边 ≤ 1568 像素
+- Token 预算 ≤ 1568（视觉编码器 28px/token）
+- 保持宽高比
+
+### 应用分类系统
+
+`deniedApps.ts` 对 191 个应用进行精细分类：
+
+**浏览器类**（55 个 bundle ID）→ 权限等级 `read`
+```
+Safari, Chrome, Firefox, Arc, Edge, Opera, Brave, Vivaldi...
+理由：浏览器操作应使用 Chrome MCP，而非盲点击
+```
+
+**终端类**（102 个 bundle ID）→ 权限等级 `click`
+```
+Terminal, iTerm2, VS Code, Cursor, JetBrains 全家桶, Xcode...
+理由：终端操作应使用 Bash Tool，限制为只能点击不能输入
+```
+
+**交易类**（34 个 bundle ID）→ 权限等级 `read`
+```
+Webull, Fidelity, Interactive Brokers, Binance, Kraken...
+理由：金融操作风险极高，仅允许截图查看
+```
+
+**完全禁止**（策略拒绝名单）：
+```
+Netflix, Spotify, Apple Music, Kindle...
+理由：版权合规，不进入权限对话框直接拒绝
+```
+
+---
+
+## 四、安全关卡体系
+
+每个输入操作（点击、键盘、拖拽）在执行前需通过 **9 层安全关卡**：
+
+
+
+### 关卡详解
+
+#### Gate 1：Kill Switch
+```typescript
+if (adapter.isDisabled()) return errorResult("Computer Use is disabled")
+```
+读取 `getChicagoEnabled()` — 在补丁版中永远返回 `true`。
+
+#### Gate 2：TCC 权限检查
+```typescript
+await adapter.ensureOsPermissions()
+// → Python: check_permissions
+// → Accessibility: osascript "tell System Events..."
+// → Screen Recording: CGDisplayCaptureDisplay()
+```
+如果缺少 macOS Accessibility 或 Screen Recording 权限，直接报错。
+
+#### Gate 3：全局互斥锁
+```typescript
+await tryAcquireComputerUseLock(sessionId)
+// 文件锁: ~/.claude/computer-use.lock
+// JSON: { sessionId, pid, acquiredAt }
+```
+确保同一时间只有一个 Claude 会话可以控制电脑。支持 stale PID 恢复。
+
+#### Gate 4：隐藏非白名单应用
+```typescript
+await executor.prepareForAction(allowlistBundleIds)
+// 隐藏所有不在白名单中的应用窗口
+// 确保截图中只出现授权应用
+```
+
+#### Gate 5：前台应用检查
+```typescript
+const frontmost = await executor.getFrontmostApp()
+if (!allowlist.includes(frontmost.bundleId)) {
+ return errorResult("Application not authorized")
+}
+```
+即使通过了白名单，还需确认当前前台应用是已授权的。
+
+#### Gate 6：权限等级检查
+
+三级权限模型：
+
+| Tier | 允许的操作 | 禁止的操作 |
+|------|-----------|-----------|
+| `read` | 截图查看 | 任何输入操作 |
+| `click` | 左键点击、滚动 | 右键、拖拽、键盘输入 |
+| `full` | 全部操作 | 无限制 |
+
+```typescript
+function tierSatisfies(tier: CuAppPermTier, required: ActionKind): boolean {
+ const order = { read: 0, click: 1, full: 2 }
+ return order[tier] >= order[required]
+}
+```
+
+**反绕过机制**：如果权限不足，响应中会附加 `TIER_ANTI_SUBVERSION` 提示，防止模型通过 AppleScript 或 System Events 绕过限制。
+
+#### Gate 7：剪贴板防护
+
+**威胁模型**：
+```
+1. Agent 调用 write_clipboard("rm -rf /")
+2. 切换到 Terminal（click-tier 允许点击）
+3. 模型点击 Terminal 的粘贴按钮
+4. 恶意命令被执行
+```
+
+**防护机制**：
+```
+当 click-tier 应用成为前台时：
+ → 保存当前剪贴板内容（stash）
+ → 清空剪贴板
+ → 每次操作后再次清空
+
+当非 click-tier 应用成为前台时：
+ → 恢复原始剪贴板内容
+```
+
+#### Gate 8：像素验证（Staleness Guard）
+
+```
+上次截图 当前实际屏幕
+┌────────────┐ ┌────────────┐
+│ 按钮A │ │ 对话框 │ ← UI 已变化
+│ [756,342] │ │ 确认? │
+└────────────┘ └────────────┘
+
+像素验证：在 [756,342] 取 9×9 网格
+ → 对比上次截图 vs 实时截图的像素
+ → 不同 → 拒绝点击 + 提示重新截图
+ → 相同 → 允许点击
+```
+
+**注意**：补丁版中 `pixelValidation` 默认关闭（`hostAdapter.cropRawPatch()` 返回 null）。
+
+#### Gate 9：系统快捷键拦截
+
+`keyBlocklist.ts` 阻止危险快捷键：
+
+| 快捷键 | 危险操作 |
+|--------|---------|
+| `⌘Q` | 退出应用 |
+| `⇧⌘Q` | 登出系统 |
+| `⌥⌘⎋` | 强制退出对话框 |
+| `⌘Tab` | 应用切换器 |
+| `⌘Space` | Spotlight |
+| `⌃⌘Q` | 锁屏 |
+
+---
+
+## 五、Python Bridge 机制
+
+
+
+### 架构设计
+
+原始 Claude Code 使用编译好的 Swift 原生模块（.node NAPI 插件）直接调用 macOS API。我们用 **Python 子进程 + JSON RPC** 替代了这一层：
+
+```
+TypeScript (Bun 运行时) Python (venv)
+┌────────────────────┐ ┌────────────────────┐
+│ executor.ts │ │ mac_helper.py │
+│ │ execFile() │ │
+│ callPythonHelper │ ──────────────→ │ main() │
+│ ('click', │ command + │ ├─ 解析 argv │
+│ {x:756,y:342}) │ --payload JSON │ ├─ dispatch() │
+│ │ │ └─ click() │
+│ ← JSON.parse ──── │ ←────────────── │ pyautogui │
+│ {ok:true, │ stdout JSON │ │
+│ result:true} │ │ json_output(...) │
+└────────────────────┘ └────────────────────┘
+```
+
+### 启动引导流程
+
+首次调用 `callPythonHelper()` 时自动完成环境初始化：
+
+```
+ensureBootstrapped()
+ │
+ ├─ 检查 .runtime/venv/bin/python3 是否存在
+ │ └─ 不存在 → python3 -m venv .runtime/venv/
+ │
+ ├─ 检查 pip 是否可用
+ │ └─ 不可用 → python3 -m ensurepip --upgrade
+ │
+ ├─ 计算 runtime/requirements.txt 的 SHA256
+ │ └─ 与 .runtime/requirements.sha256 对比
+ │ └─ 不同 → pip install -r requirements.txt
+ │ 写入新的 SHA256 哈希
+ │ └─ 相同 → 跳过安装
+ │
+ └─ 就绪，返回 venv Python 路径
+```
+
+**依赖清单**（`runtime/requirements.txt`）：
+
+| 库 | 用途 |
+|----|------|
+| `mss` | 高性能屏幕截图 |
+| `Pillow` | JPEG 编码和图像处理 |
+| `pyautogui` | 鼠标点击、键盘输入 |
+| `pyobjc-core` | macOS Objective-C 桥接 |
+| `pyobjc-framework-Cocoa` | NSWorkspace（应用管理）、NSPasteboard（剪贴板） |
+| `pyobjc-framework-Quartz` | CGDisplay（显示器）、CGWindow（窗口列表） |
+
+### 命令映射表
+
+`mac_helper.py`（660 行）实现了以下命令：
+
+| 命令 | Python 实现 | 返回值 |
+|------|------------|--------|
+| `screenshot` | `mss.grab()` + `PIL.Image` JPEG 编码 | `{base64, width, height, displayWidth, displayHeight}` |
+| `zoom` | `mss.grab(region)` 区域截图 | `{base64, width, height}` |
+| `click` | `pyautogui.moveTo()` + `pyautogui.click()` | `true` |
+| `key` | `pyautogui.hotkey()` / `pyautogui.press()` | `true` |
+| `type` | `pyautogui.write(interval=0.008)` | `true` |
+| `drag` | `pyautogui.dragTo(duration=0.2)` | `true` |
+| `scroll` | `pyautogui.scroll()` / `pyautogui.hscroll()` | `true` |
+| `hold_key` | `pyautogui.keyDown()` + `sleep` + `pyautogui.keyUp()` | `true` |
+| `frontmost_app` | `NSWorkspace.frontmostApplication()` | `{bundleId, displayName}` |
+| `list_displays` | `CGGetActiveDisplayList()` + `CGDisplayBounds()` | `[DisplayGeometry...]` |
+| `find_window_displays` | `CGWindowListCopyWindowInfo()` + 交集计算 | `[{bundleId, displayIds}...]` |
+| `list_installed_apps` | 递归扫描 `/Applications` + `plistlib` | `[InstalledApp...]` |
+| `list_running_apps` | `NSWorkspace.runningApplications()` | `[RunningApp...]` |
+| `open_app` | `NSWorkspace.launchApplicationAtURL_options_` | `void` |
+| `read_clipboard` | `NSPasteboard.stringForType_()` | `string` |
+| `write_clipboard` | `NSPasteboard.setString_forType_()` | `void` |
+| `check_permissions` | `osascript` + `CGDisplayCaptureDisplay` | `{accessibility, screenRecording}` |
+
+### 错误处理
+
+```python
+# mac_helper.py 的统一错误处理
+def main():
+ try:
+ result = dispatch(command, payload)
+ json_output({"ok": True, "result": result})
+ except Exception as e:
+ error_output({"ok": False, "error": {"message": str(e)}})
+```
+
+TypeScript 端：
+```typescript
+// pythonBridge.ts
+const parsed = JSON.parse(stdout)
+if (!parsed.ok) {
+ throw new Error(parsed.error.message) // → MCP tool error
+}
+return parsed.result
+```
+
+---
+
+## 六、截图-分析-操作闭环
+
+一个完整的 Computer Use 交互由多个 **截图-分析-操作** 循环组成：
+
+### 典型交互流程
+
+```
+用户: "帮我打开网易云音乐，搜索一首歌"
+
+┌─ 循环 1：发现并打开应用 ─────────────────────────────────┐
+│ │
+│ Step 1: request_access │
+│ → 弹出权限对话框，用户授权允许操作的应用 │
+│ → 设置 allowedApps, grantFlags │
+│ │
+│ Step 2: screenshot │
+│ → 截取全屏 → JPEG 编码 → base64 │
+│ → 缓存截图尺寸 (lastScreenshotDims) │
+│ → 返回给模型 │
+│ │
+│ Step 3: 模型分析截图 │
+│ → "桌面上没有网易云音乐，需要打开它" │
+│ → 决定调用 open_application │
+│ │
+│ Step 4: open_application("com.netease.163music") │
+│ → Gate 1-9 全部通过 │
+│ → Python: NSWorkspace.launchApplicationAtURL_() │
+│ │
+└──────────────────────────────────────────────────────────┘
+
+┌─ 循环 2：定位搜索框 ────────────────────────────────────┐
+│ │
+│ Step 5: screenshot │
+│ → 截取全屏（网易云已打开） │
+│ → 更新 lastScreenshotDims │
+│ │
+│ Step 6: 模型分析截图 │
+│ → 视觉识别搜索框位置 → (756, 342) │
+│ → 决定点击搜索框 │
+│ │
+│ Step 7: left_click({coordinate: [756, 342]}) │
+│ → Gate 4: 隐藏非白名单应用 │
+│ → Gate 5: 前台是网易云 ✓ │
+│ → Gate 6: tier=full ≥ click ✓ │
+│ → Gate 7: 非 click-tier 应用，跳过 │
+│ → scaleCoord(756, 342) 转换为屏幕坐标 │
+│ → Python: pyautogui.click(x_logical, y_logical) │
+│ │
+└──────────────────────────────────────────────────────────┘
+
+┌─ 循环 3：输入搜索词 ────────────────────────────────────┐
+│ │
+│ Step 8: type({text: "喜欢你"}) │
+│ → Gate 6: tier=full ≥ keyboard ✓ │
+│ → Python: pyautogui.write("喜欢你", interval=0.008) │
+│ │
+│ Step 9: screenshot │
+│ → 确认搜索结果已出现 │
+│ → 模型分析："搜索结果列表中第一个就是目标歌曲" │
+│ │
+│ Step 10: left_click({coordinate: [...]}) │
+│ → 点击目标歌曲 │
+│ │
+└──────────────────────────────────────────────────────────┘
+```
+
+### 坐标转换的关键
+
+**模型看到的**和**屏幕实际的**是不同的坐标空间：
+
+```
+原始屏幕 (2560 x 1600, Retina 2x)
+ ├─ 逻辑尺寸: 1280 x 800
+ └─ 物理像素: 2560 x 1600
+
+imageResize 计算后：
+ ├─ 缩放后尺寸: 1176 x 735 (≤1568px 预算)
+ └─ 这就是模型"看到"的截图尺寸
+
+模型输出坐标: [588, 368]（图像空间的像素位置）
+
+scaleCoord 转换:
+ x_logical = (588 / 1176) * 1280 + originX = 640 + 0 = 640
+ y_logical = (368 / 735) * 800 + originY = 400 + 0 = 400
+
+Python 执行:
+ pyautogui.moveTo(640, 400) ← 逻辑坐标（macOS 自动处理 Retina）
+```
+
+### 截图缓存与状态同步
+
+```
+bindSessionContext 闭包
+ │
+ ├─ lastScreenshot (内存)
+ │ ├─ base64: JPEG 数据（用于 pixel validation）
+ │ ├─ width/height: 模型看到的尺寸
+ │ └─ displayWidth/displayHeight/originX/originY: 显示器几何
+ │
+ └─ AppState.computerUseMcpState (持久化)
+ ├─ allowedApps: AppGrant[] — 已授权应用列表
+ ├─ grantFlags: {...} — 剪贴板/系统快捷键权限
+ ├─ selectedDisplayId?: number — 选定的显示器
+ ├─ lastScreenshotDims?: {...} — 截图几何（跨重启恢复）
+ └─ hiddenDuringTurn?: Set — 本轮隐藏的应用
+```
+
+---
+
+## 七、关键源文件索引
+
+### vendor/computer-use-mcp/（原始代码层）
+
+| 文件 | 行数 | 职责 |
+|------|------|------|
+| `types.ts` | 622 | 权限模型、会话上下文、全部类型定义 |
+| `tools.ts` | 707 | 24 个 MCP 工具的 schema 和参数校验 |
+| `toolCalls.ts` | 1600+ | **核心**：工具派发、9 层安全关卡、权限流程 |
+| `deniedApps.ts` | 554 | 191 个应用的分类（browser/terminal/trading）和权限映射 |
+| `sentinelApps.ts` | 44 | 敏感应用预警标签（shell/filesystem/system_settings） |
+| `mcpServer.ts` | 314 | MCP 服务器工厂、会话上下文绑定、全局锁 |
+| `pixelCompare.ts` | 172 | 点击目标像素验证（staleness guard） |
+| `imageResize.ts` | 109 | 截图尺寸计算（API 图像转码算法） |
+| `keyBlocklist.ts` | 154 | 系统快捷键拦截（⌘Q、⌘Tab 等） |
+| `executor.ts` | 101 | ComputerExecutor 接口定义 |
+| `subGates.ts` | 20 | 灰度子开关预设组合 |
+
+### utils/computerUse/（CLI 适配层）
+
+| 文件 | 行数 | 职责 | 补丁? |
+|------|------|------|-------|
+| `executor.ts` | 231 | ComputerExecutor 的 Python bridge 实现 | ✅ 重写 |
+| `pythonBridge.ts` | 111 | Python 子进程管理、venv 引导、JSON RPC | ✅ 新增 |
+| `hostAdapter.ts` | 54 | HostAdapter 实现（权限检查、灰度读取） | 部分修改 |
+| `gates.ts` | 51 | GrowthBook 灰度控制（`getChicagoEnabled` 绕过） | ✅ 修改 |
+| `wrapper.tsx` | 300+ | 会话上下文构建、权限对话框、锁管理 | 未修改 |
+| `setup.ts` | 54 | MCP 配置初始化 | 未修改 |
+| `computerUseLock.ts` | 216 | 全局文件锁（`~/.claude/computer-use.lock`） | 未修改 |
+| `common.ts` | 62 | 常量定义（server name、bundle ID） | 未修改 |
+| `cleanup.ts` | — | turn-end 清理（应用恢复、剪贴板恢复） | 未修改 |
+| `toolRendering.tsx` | — | 工具结果 UI 渲染 | 未修改 |
+
+### runtime/（Python 运行时）
+
+| 文件 | 行数 | 职责 | 补丁? |
+|------|------|------|-------|
+| `mac_helper.py` | 660 | 所有系统交互的 Python 实现 | ✅ 新增 |
+| `requirements.txt` | 6 | Python 依赖声明 | ✅ 新增 |
+
+---
+
+## 八、设计权衡
+
+### 为什么选择 Python Bridge？
+
+| 维度 | 原生 Swift (.node) | Python Bridge |
+|------|-------------------|---------------|
+| **性能** | ~0ms（进程内调用） | ~50-100ms（子进程启动） |
+| **可读性** | 编译后不可读 | 660 行清晰的 Python |
+| **可修改性** | 需要 Swift 编译环境 | 直接编辑 .py 文件 |
+| **依赖** | 特定 Bun 版本的 NAPI | 任意 Python 3.8+ |
+| **跨平台** | 仅 macOS | pyautogui/mss 天然跨平台 |
+| **用户体验** | 无感知 | 无感知（模型思考时间秒级） |
+
+**结论**：50-100ms 的额外延迟在 Computer Use 的场景下完全可以忽略——模型分析截图和决策的时间通常是 2-5 秒，用户不会注意到底层操作多了 100ms。
+
+### 我们尝试过但放弃的方案
+
+**方案一：提取原生 .node 模块**
+- 从 Claude Code 二进制中成功提取了 `computer-use-swift.node`（ARM64 424KB）
+- 同步方法正常工作，但 **Swift 异步方法的 continuation 永远不会 resume**
+- 根因：.node 文件针对 Claude Code 内置 Bun 编译，与用户 Bun 版本不兼容
+
+**方案二：空 Stub 包**
+- 代码能编译但所有操作报错——没有实际执行能力
+
+---
+
+## 相关文档
+
+- [Computer Use 功能指南](./computer-use.md) — 使用方式、快速开始、环境变量
+- [源码修复记录](/reference/fixes) — 其他修复和补丁的详细记录
diff --git a/docs/features/images/01-computer-use-architecture.jpg b/docs/features/images/01-computer-use-architecture.jpg
new file mode 100644
index 00000000..65f331b8
Binary files /dev/null and b/docs/features/images/01-computer-use-architecture.jpg differ
diff --git a/docs/features/images/02-computer-use-security-gates.jpg b/docs/features/images/02-computer-use-security-gates.jpg
new file mode 100644
index 00000000..95a29b90
Binary files /dev/null and b/docs/features/images/02-computer-use-security-gates.jpg differ
diff --git a/docs/features/images/03-computer-use-python-bridge.jpg b/docs/features/images/03-computer-use-python-bridge.jpg
new file mode 100644
index 00000000..c8cbc041
Binary files /dev/null and b/docs/features/images/03-computer-use-python-bridge.jpg differ
diff --git a/docs/features/images/04-computer-use-patch.jpg b/docs/features/images/04-computer-use-patch.jpg
new file mode 100644
index 00000000..176be531
Binary files /dev/null and b/docs/features/images/04-computer-use-patch.jpg differ
diff --git a/docs/index.md b/docs/index.md
index c44b53d3..2f7b1a02 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -32,12 +32,12 @@ features:
title: Skills 系统
details: 可扩展能力插件、自定义工作流、条件激活
link: /skills/01-usage-guide
- - icon: "\U0001F4BB"
- title: Computer Use
- details: 桌面控制功能 — 截屏、鼠标、键盘操作
- link: /features/computer-use
- icon: "\U0001F310"
title: 第三方模型支持
details: 接入 OpenAI、DeepSeek、Ollama 等任意兼容模型
link: /guide/third-party-models
+ - icon: "\U0001F4BB"
+ title: Computer Use
+ details: 桌面控制功能 — 截屏、鼠标、键盘操作（Python Bridge 实现）
+ link: /features/computer-use
---
diff --git a/docs/memory/01-usage-guide.md b/docs/memory/01-usage-guide.md
index 225f7dbe..29438d55 100644
--- a/docs/memory/01-usage-guide.md
+++ b/docs/memory/01-usage-guide.md
@@ -3,7 +3,7 @@
> 让 Claude Code 跨会话记住你是谁、你偏好什么、项目正在发生什么。
-记忆系统 · 四种记忆类型 · 触发保存 · 存储位置 · 管理记忆 · 生命周期 · 快速参考
+记忆系统 · 四种记忆类型 · 触发保存 · 存储位置 · 管理记忆 · 生命周期 · 快速参考
diff --git a/docs/memory/02-implementation.md b/docs/memory/02-implementation.md
index c6e1c542..8c57f9c9 100644
--- a/docs/memory/02-implementation.md
+++ b/docs/memory/02-implementation.md
@@ -3,7 +3,7 @@
> 从系统提示词注入到后台自动提取，拆解记忆系统的每一个技术细节。
-整体架构 · 路径解析 · 提示词注入 · 自动提取 · 智能检索 · 记忆扫描 · 代理记忆 · 团队同步 · 常量速查 · 数据流总览
+整体架构 · 路径解析 · 提示词注入 · 自动提取 · 智能检索 · 记忆扫描 · 代理记忆 · 团队同步 · 常量速查 · 数据流总览
diff --git a/docs/memory/03-autodream.md b/docs/memory/03-autodream.md
index c58f817b..5d0270b2 100644
--- a/docs/memory/03-autodream.md
+++ b/docs/memory/03-autodream.md
@@ -3,7 +3,7 @@
> Claude 会"做梦"——在后台静默回顾近期会话，整合、更新、修剪记忆，就像人类睡眠中整理白天的记忆一样。
-AutoDream · 触发条件 · 整合流程 · 安全限制 · UI 展示 · 配置开关 · 对比 · 源码导航
+AutoDream · 触发条件 · 整合流程 · 安全限制 · UI 展示 · 配置开关 · 对比 · 源码导航
diff --git a/docs/skills/01-usage-guide.md b/docs/skills/01-usage-guide.md
index 17e47dd2..7225c324 100644
--- a/docs/skills/01-usage-guide.md
+++ b/docs/skills/01-usage-guide.md
@@ -3,7 +3,7 @@
> Skills 是 Claude Code 的扩展能力引擎，让你用 Markdown 文件定义专属的自动化工作流。
-Skills 是什么 · 六种来源 · 定义格式 · 调用方式 · 执行上下文 · 条件激活 · 权限控制 · 快速参考
+Skills 是什么 · 六种来源 · 定义格式 · 调用方式 · 执行上下文 · 条件激活 · 权限控制 · 快速参考
diff --git a/docs/skills/02-implementation.md b/docs/skills/02-implementation.md
index 29ad9e85..58120022 100644
--- a/docs/skills/02-implementation.md
+++ b/docs/skills/02-implementation.md
@@ -3,7 +3,7 @@
> 深度剖析 Skills 如何被发现、加载、注入、执行和管理。
-整体架构 · 发现与加载 · Frontmatter 解析 · 注入对话 · 执行引擎 · Fork 执行 · 条件激活 · Hook 集成 · 权限系统 · 完整生命周期 · 源码索引
+整体架构 · 发现与加载 · Frontmatter 解析 · 注入对话 · 执行引擎 · Fork 执行 · 条件激活 · Hook 集成 · 权限系统 · 完整生命周期 · 源码索引
