Re-add pi-web-access
This commit is contained in:
Submodule pi/files/agent/extensions/pi-web-access deleted from 4461c2bcb2
2
pi/files/agent/extensions/pi-web-access/.gitignore
vendored
Normal file
2
pi/files/agent/extensions/pi-web-access/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
node_modules/
|
||||
bun.lock
|
||||
262
pi/files/agent/extensions/pi-web-access/CHANGELOG.md
Normal file
262
pi/files/agent/extensions/pi-web-access/CHANGELOG.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# Pi Web Access - Changelog
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [0.10.2] - 2026-02-18
|
||||
|
||||
### Added
|
||||
- **Interactive search curation.** Press Ctrl+Shift+S during or after a multi-query search to open a browser-based review UI. Results stream in live via SSE. Pick which queries to keep, add new searches on the fly, switch providers — then submit to send only the curated results to the agent.
|
||||
- **Auto-condense pipeline.** When the countdown expires without manual curation, a single LLM call (Claude Haiku by default) condenses all search results into a deduplicated briefing organized by topic. Preprocessing enriches the prompt with URL overlap, answer similarity, and source quality analysis. Configure via `"autoFilter"` in `~/.pi/web-search.json`. Full uncondensed results stored and retrievable via `get_search_content`.
|
||||
- **Configurable keyboard shortcuts.** Both shortcuts (curate: Ctrl+Shift+S, activity monitor: Ctrl+Shift+W) can be remapped via `"shortcuts"` in `~/.pi/web-search.json`. Changes take effect on restart.
|
||||
- **`/websearch` command** — opens the curator directly from pi without an agent round-trip. Accepts optional comma-separated queries or opens empty.
|
||||
- **Task-aware condensation.** Optional `context` parameter on `web_search` — a brief description of the user's task. The condenser uses it to focus the briefing on what matters.
|
||||
- **Provider selection** — global dropdown in the curator UI to switch between Perplexity and Gemini. Persists to `~/.pi/web-search.json`.
|
||||
- **Live condense status in countdown.** Shows "condensing..." while the LLM is working, then "N searches condensed" once complete.
|
||||
- Markdown rendering in curator result cards via marked.js.
|
||||
- Query-level result cards with expandable answers and source lists. Check/uncheck to include or exclude.
|
||||
- SSE streaming with keepalive, socket health checks, and buffered delivery.
|
||||
- Idle-based timer (60s default, adjustable). Timeout sends all results as safe default.
|
||||
- Keyboard shortcuts: Enter (submit), Escape (skip), A (toggle all).
|
||||
- Dark/light theme via `prefers-color-scheme` with teal accent palette.
|
||||
|
||||
### Changed
|
||||
- **Curate enabled by default.** Multi-query searches show a 10-second review window; single queries send immediately. Pass `curate: false` to opt out.
|
||||
- **Curate shortcut opens browser immediately, even mid-search.** Remaining results stream in live via SSE.
|
||||
- **Tool descriptions encourage multi-query research.** The `queries` param explains how to vary phrasing and scope across 2-4 queries, with good/bad examples.
|
||||
- **Curated results instruct the LLM.** Tool output prefixed with an instruction telling the LLM to use curated results as-is.
|
||||
- Expanded view shows full answer text per query with source titles and domains.
|
||||
- Non-curated `web_search` calls now respect the saved provider preference.
|
||||
- Config helpers generalized from `loadSavedProvider`/`saveProvider` to `loadConfig`/`saveConfig`.
|
||||
|
||||
### Fixed
|
||||
- Curated `onSubmit` passed the original full query list instead of the filtered list, inflating `queryCount`.
|
||||
- Collapsed curated status mixed source URL counts with query counts.
|
||||
|
||||
### New files
|
||||
- `curator-server.ts` — ephemeral HTTP server with SSE streaming, state machine, heartbeat watchdog, and token auth.
|
||||
- `curator-page.ts` — HTML/CSS/JS for the curator UI with markdown rendering and overlay transitions.
|
||||
- `search-filter.ts` — auto-condense pipeline: preprocessing, LLM condensation via pi's model registry, and post-processing (citation verification, source list completion).
|
||||
|
||||
## [0.7.3] - 2026-02-05
|
||||
|
||||
### Added
|
||||
- Jina Reader fallback for JS-rendered pages. When Readability returns insufficient content (cookie notices, consent walls, SPA shells), the extraction chain now tries Jina Reader (`r.jina.ai`) before falling back to Gemini. Jina handles JavaScript rendering server-side and returns clean markdown. No API key required.
|
||||
- JS-render detection heuristic (`isLikelyJSRendered`) produces more specific error messages when pages appear to load content dynamically.
|
||||
- Actionable guidance when all extraction methods fail, listing steps to configure Gemini API or use `web_search` instead.
|
||||
|
||||
### Changed
|
||||
- HTTP fetch headers now mimic Chrome (realistic `User-Agent`, `Sec-Fetch-*`, `Accept-Language`) instead of the default Node.js user agent. Reduces blocks from bot-detection systems.
|
||||
- Short Readability output (< 500 chars) is now treated as a content failure, triggering the fallback chain. Previously, a 266-char cookie notice was returned as "successful" content.
|
||||
- Extraction fallback order is now: HTTP+Readability → RSC → Jina Reader → Gemini URL Context → Gemini Web → error with guidance.
|
||||
|
||||
### Fixed
|
||||
- `parseTimestamp` now rejects negative values in colon-separated format (`-1:30`, `1:-30`). Previously only the numeric path (`-90`) rejected negatives, while the colon path computed and returned negative seconds.
|
||||
|
||||
## [0.7.2] - 2026-02-03
|
||||
|
||||
### Added
|
||||
- `model` parameter on `fetch_content` to override the Gemini model per-request (e.g. `model: "gemini-2.5-flash"`)
|
||||
- Collapsed TUI results now show a 200-char text preview instead of just the status line
|
||||
- LICENSE file (MIT)
|
||||
|
||||
### Changed
|
||||
- Default Gemini model updated from `gemini-2.5-flash` to `gemini-3-flash-preview` across all API, search, URL context, YouTube, and video paths. Gemini Web gracefully falls back to `gemini-2.5-flash` when the model header isn't available.
|
||||
- README rewritten: added tagline, badges, "Why" section, Quick Start, corrected "How It Works" routing order, fixed inaccurate env var precedence claim, added missing `/v/` YouTube format, restored `/search` command docs, collapsible Files table
|
||||
|
||||
### Fixed
|
||||
- `PERPLEXITY_API_KEY` env var now takes precedence over config file value, matching `GEMINI_API_KEY` behavior and README documentation (was reversed)
|
||||
- `package.json` now includes `repository`, `homepage`, `bugs`, and `description` fields (repo link was missing from pi packages site)
|
||||
|
||||
## [0.7.0] - 2026-02-03
|
||||
|
||||
### Added
|
||||
- **Multi-provider web search**: `web_search` now supports Perplexity, Gemini API (with Google Search grounding), and Gemini Web (cookie auth) as search providers. New `provider` parameter (`auto`, `perplexity`, `gemini`) controls selection. In `auto` mode (default): Perplexity → Gemini API → Gemini Web. Backwards-compatible — existing Perplexity users see no change.
|
||||
- **Gemini API grounded search**: Structured citations via `groundingMetadata` with source URIs and text-to-source mappings. Google proxy URLs are resolved via HEAD redirects. Configured via `GEMINI_API_KEY` or `geminiApiKey` in config.
|
||||
- **Gemini Web search**: Zero-config web search for users signed into Google in Chrome. Prompt instructs Gemini to cite sources; URLs extracted from markdown response.
|
||||
- **Gemini extraction fallback**: When `fetch_content` fails (HTTP 403/429, Readability fails, network errors), automatically retries via Gemini URL Context API then Gemini Web extraction. Each has an independent 60s timeout. Handles SPAs, JS-heavy pages, and anti-bot protections.
|
||||
- **Local video file analysis**: `fetch_content` accepts file paths to video files (MP4, MOV, WebM, AVI, etc.). Detected by path prefix (`/`, `./`, `../`, `file://`), validated by extension and 50MB limit. Two-tier fallback: Gemini API (resumable upload via Files API with proper MIME types, poll-until-active and cleanup) → Gemini Web (free, cookie auth).
|
||||
- **Video prompt parameter**: `fetch_content` gains optional `prompt` parameter for asking specific questions about video content. Threads through YouTube and local video extraction. Without prompt, uses default extraction (transcript + visual descriptions).
|
||||
- **Video thumbnails**: YouTube results include the video thumbnail (fetched from `img.youtube.com`). Local video results include a frame extracted via ffmpeg (at ~1 second). Returned as image content parts alongside text — the agent sees the thumbnail as vision context.
|
||||
- **Configurable frame extraction**: `frames` parameter (1-12) on `fetch_content` for pulling visual frames from YouTube or local video. Works in five modes: frames alone (sample across entire video), single timestamp (one frame), single+frames (N frames at 5s intervals), range (default 6 frames), range+frames (N frames across the range). Endpoint-inclusive distribution with 5-second minimum spacing.
|
||||
- **Video duration in responses**: Frame extraction results include the video duration for context.
|
||||
- `searchProvider` config option in `~/.pi/web-search.json` for global provider default
|
||||
- `video` config section: `enabled`, `preferredModel`, `maxSizeMB`
|
||||
|
||||
### Changed
|
||||
- `PerplexityResponse` renamed to `SearchResponse` (shared interface for all search providers)
|
||||
- Extracted HTTP pipeline from `extractContent` into `extractViaHttp` for cleaner Gemini fallback orchestration
|
||||
- `getApiKey()`, `API_BASE`, `DEFAULT_MODEL` exported from `gemini-api.ts` for use by search and URL Context modules
|
||||
- `isPerplexityAvailable()` added to `perplexity.ts` as non-throwing API key check
|
||||
- Content-type routing in `extract.ts`: only `text/html` and `application/xhtml+xml` go through Readability; all other text types (`text/markdown`, `application/json`, `text/csv`, etc.) returned directly. Fixes the OpenAI cookbook `.md` URL that returned "Untitled (30 chars)".
|
||||
- Title extraction for non-HTML content: `extractTextTitle()` pulls from markdown `#`/`##` headings, falls back to URL filename
|
||||
- Combined `yt-dlp --print duration -g` call fetches stream URL and duration in a single invocation, reused across all frame extraction paths via `streamInfo` passthrough
|
||||
- Shared helpers in `utils.ts` (`formatSeconds`, error mapping) eliminate circular imports and duplication across youtube-extract.ts and video-extract.ts
|
||||
|
||||
### Fixed
|
||||
- `fetch_content` TUI rendered `undefined/undefined URLs` during progress updates (renderResult didn't handle `isPartial`, now shows a progress bar like `web_search` does)
|
||||
- RSC extractor produced malformed markdown for `<pre><code>` blocks (backticks inside fenced code blocks) -- extremely common on Next.js documentation pages
|
||||
- Multi-URL fetch failures rendered in green "success" color even when 0 URLs succeeded (now red)
|
||||
- `web_search` queries parameter described as "parallel" in schema but execution is sequential (changed to "batch"; `urls` correctly remains "parallel")
|
||||
- Proper error propagation for frame extraction: missing binaries (yt-dlp, ffmpeg, ffprobe), private/age-restricted/region-blocked videos, expired stream URLs (403), timestamp-exceeds-duration, and timeouts all produce specific user-facing messages instead of silent nulls
|
||||
- `isTimeoutError` now detects `execFileSync` timeouts via the `killed` flag (SIGTERM from timeout was previously unrecognized)
|
||||
- Float video durations (e.g. 15913.7s from yt-dlp) no longer produce out-of-range timestamps — durations are floored before computing frame positions
|
||||
- `parseTimestamp` consistently floors results across both bare-number ("90.5" → 90) and colon ("1:30.5" → 90) paths — previously the colon path returned floats
|
||||
- YouTube thumbnail assignment no longer sets `null` on the optional `thumbnail` field when fetch fails (was a type mismatch; now only assigned on success)
|
||||
|
||||
### New files
|
||||
- `gemini-search.ts` -- search routing + Gemini Web/API search providers with grounding
|
||||
- `gemini-url-context.ts` -- URL Context API extraction + Gemini Web extraction fallback
|
||||
- `video-extract.ts` -- local video file detection, Gemini Web/API analysis with Files API upload
|
||||
- `utils.ts` -- shared formatting and error helpers for frame extraction
|
||||
|
||||
## [0.6.0] - 2026-02-02
|
||||
|
||||
### Added
|
||||
- YouTube video understanding in `fetch_content` via three-tier fallback chain:
|
||||
- **Gemini Web** (primary): reads Chrome session cookies from macOS Keychain + SQLite, authenticates to gemini.google.com, sends YouTube URL via StreamGenerate endpoint. Full visual + audio understanding with timestamps. Zero config needed if signed into Google in Chrome.
|
||||
- **Gemini API** (secondary): direct REST calls with `GEMINI_API_KEY`. YouTube URLs passed as `file_data.file_uri`. Configure via `GEMINI_API_KEY` env var or `geminiApiKey` in `~/.pi/web-search.json`.
|
||||
- **Perplexity** (fallback): uses existing `searchWithPerplexity` for a topic summary when neither Gemini path is available. Output labeled as "Summary (via Perplexity)" so the agent knows it's not a full transcript.
|
||||
- YouTube URL detection for all common formats: `/watch?v=`, `youtu.be/`, `/shorts/`, `/live/`, `/embed/`, `/v/`, `m.youtube.com`
|
||||
- Configurable via `~/.pi/web-search.json` under `youtube` key (`enabled`, `preferredModel`)
|
||||
- Actionable error messages when extraction fails (directs user to sign into Chrome or set API key)
|
||||
- YouTube URLs no longer fall through to HTTP/Readability (which returns garbage); returns error instead
|
||||
|
||||
### New files
|
||||
- `chrome-cookies.ts` -- macOS Chrome cookie extraction using Node builtins (`node:crypto`, `node:sqlite`, `child_process`)
|
||||
- `gemini-web.ts` -- Gemini Web client ported from surf's gemini-client.cjs (cookie auth, StreamGenerate, model fallback)
|
||||
- `gemini-api.ts` -- Gemini REST API client (generateContent, file upload/processing/cleanup for Phase 2)
|
||||
- `youtube-extract.ts` -- YouTube extraction orchestrator with three-tier fallback and activity logging
|
||||
|
||||
## [0.5.1] - 2026-02-02
|
||||
|
||||
### Added
|
||||
- Bundled `librarian` skill -- structured research workflow for open-source libraries with GitHub permalinks, combining fetch_content (cloning), web_search (recent info), and git operations (blame, log, show)
|
||||
|
||||
### Fixed
|
||||
- Session fork event handler was registered as `session_branch` (non-existent event) instead of `session_fork`, meaning forks never triggered cleanup (abort pending fetches, clear clone cache, restore session data)
|
||||
- API fallback title for tree URLs with a path (e.g. `/tree/main/src`) now includes the path (`owner/repo - src`), consistent with clone-based results
|
||||
- Removed unnecessary export on `getDefaultBranch` (only used internally by `fetchViaApi`)
|
||||
|
||||
## [0.5.0] - 2026-02-01
|
||||
|
||||
### Added
|
||||
- GitHub repository clone extraction for `fetch_content` -- detects GitHub code URLs, clones repos to `/tmp/pi-github-repos/`, and returns actual file contents plus local path for further exploration with `read` and `bash`
|
||||
- Lightweight API fallback for oversized repos (>350MB) and commit SHA URLs via `gh api`
|
||||
- Clone cache with concurrent request deduplication (second request awaits first's clone)
|
||||
- `forceClone` parameter on `fetch_content` to override the size threshold
|
||||
- Configurable via `~/.pi/web-search.json` under `githubClone` key (enabled, maxRepoSizeMB, cloneTimeoutSeconds, clonePath)
|
||||
- Falls back to `git clone` when `gh` CLI is unavailable (public repos only)
|
||||
- README: GitHub clone documentation with config, flow diagram, and limitations
|
||||
|
||||
### Changed
|
||||
- Refactored `extractContent`/`fetchAllContent` signatures from positional `timeoutMs` to `ExtractOptions` object
|
||||
- Blob/tree fetch titles now include file path (e.g. `owner/repo - src/index.ts`) for better disambiguation in multi-URL results and TUI
|
||||
|
||||
### Fixed
|
||||
- README: Activity monitor keybinding corrected from `Ctrl+Shift+O` to `Ctrl+Shift+W`
|
||||
|
||||
## [0.4.5] - 2026-02-01
|
||||
|
||||
### Changed
|
||||
- Added package keywords for npm discoverability
|
||||
|
||||
## [0.4.4] - 2026-02-01
|
||||
|
||||
### Fixed
|
||||
- Adapt execute signatures to pi v0.51.0: reorder signal, onUpdate, ctx parameters across all three tools
|
||||
|
||||
## [0.4.3] - 2026-01-27
|
||||
|
||||
### Fixed
|
||||
- Google API compatibility: Use `StringEnum` for `recencyFilter` to avoid unsupported `anyOf`/`const` JSON Schema patterns
|
||||
|
||||
## [0.4.2] - 2026-01-27
|
||||
|
||||
### Fixed
|
||||
|
||||
- Single-URL fetches now store content for retrieval via `get_search_content` (previously only multi-URL)
|
||||
- Corrected `get_search_content` usage syntax in fetch_content help messages
|
||||
|
||||
### Changed
|
||||
|
||||
- Increased inline content limit from 10K to 30K chars (larger content truncated but fully retrievable)
|
||||
|
||||
### Added
|
||||
|
||||
- Banner image for README
|
||||
|
||||
## [0.4.1] - 2026-01-26
|
||||
|
||||
### Changed
|
||||
- Added `pi` manifest to package.json for pi v0.50.0 package system compliance
|
||||
- Added `pi-package` keyword for npm discoverability
|
||||
|
||||
## [0.4.0] - 2026-01-19
|
||||
|
||||
### Added
|
||||
|
||||
- PDF extraction via `unpdf` - fetches PDFs from URLs and saves as markdown to `~/Downloads/`
|
||||
- Extracts text, metadata (title, author), page count
|
||||
- Supports PDFs up to 20MB (vs 5MB for HTML)
|
||||
- Handles arxiv URLs with smart title fallback
|
||||
|
||||
### Fixed
|
||||
|
||||
- Plain text URL detection now uses hostname check instead of substring match
|
||||
|
||||
## [0.3.0] - 2026-01-19
|
||||
|
||||
### Added
|
||||
|
||||
- RSC (React Server Components) content extraction for Next.js App Router pages
|
||||
- Parses flight data from `<script>self.__next_f.push([...])</script>` tags
|
||||
- Reconstructs markdown with headings, tables, code blocks, links
|
||||
- Handles chunk references and nested components
|
||||
- Falls back to RSC extraction when Readability fails
|
||||
- Content-type validation rejects binary files (images, PDFs, audio, video, zip)
|
||||
- 5MB response size limit (checked via Content-Length header) to prevent memory issues
|
||||
|
||||
### Fixed
|
||||
|
||||
- `fetch_content` now handles plain text URLs (raw.githubusercontent.com, gist.githubusercontent.com, any text/plain response) instead of failing with "Could not extract readable content"
|
||||
|
||||
## [0.2.0] - 2026-01-11
|
||||
|
||||
### Added
|
||||
|
||||
- Activity monitor widget (`Ctrl+Shift+O`) showing live request/response activity
|
||||
- Displays last 10 API calls and URL fetches with status codes and timing
|
||||
- Shows rate limit usage and reset countdown
|
||||
- Live updates as requests complete
|
||||
- Auto-clears on session switch
|
||||
|
||||
### Changed
|
||||
|
||||
- Refactored activity tracking into dedicated `activity.ts` module
|
||||
|
||||
## [0.1.0] - 2026-01-06
|
||||
|
||||
Initial release. Designed for pi v0.37.3.
|
||||
|
||||
### Added
|
||||
|
||||
- `web_search` tool - Search via Perplexity AI with synthesized answers and citations
|
||||
- Single or batch queries (parallel execution)
|
||||
- Recency filter (day/week/month/year)
|
||||
- Domain filter (include or exclude)
|
||||
- Optional async content fetching with agent notification
|
||||
- `fetch_content` tool - Fetch and extract readable content from URLs
|
||||
- Single URL returns content directly
|
||||
- Multiple URLs store for retrieval via `get_search_content`
|
||||
- Concurrent fetching (3 max) with 30s timeout
|
||||
- `get_search_content` tool - Retrieve stored search results or fetched content
|
||||
- Access by response ID, URL, query, or index
|
||||
- `/search` command - Interactive browser for stored results
|
||||
- TUI rendering with progress bars, URL lists, and expandable previews
|
||||
- Session-aware storage with 1-hour TTL
|
||||
- Rate limiting (10 req/min for Perplexity API)
|
||||
- Config file support (`~/.pi/web-search.json`)
|
||||
- Content extraction via Readability + Turndown (max 10k chars)
|
||||
- Proper session isolation - pending fetches abort on session switch
|
||||
- URL validation before fetch attempts
|
||||
- Defensive JSON parsing for API responses
|
||||
21
pi/files/agent/extensions/pi-web-access/LICENSE
Normal file
21
pi/files/agent/extensions/pi-web-access/LICENSE
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Nico Bailon
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
334
pi/files/agent/extensions/pi-web-access/README.md
Normal file
334
pi/files/agent/extensions/pi-web-access/README.md
Normal file
@@ -0,0 +1,334 @@
|
||||
<p>
|
||||
<img src="banner.png" alt="pi-web-access" width="1100">
|
||||
</p>
|
||||
|
||||
# Pi Web Access
|
||||
|
||||
**Web search, content extraction, and video understanding for Pi agent. Zero config with Chrome, or bring your own API keys.**
|
||||
|
||||
[](https://www.npmjs.com/package/pi-web-access)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[]()
|
||||
|
||||
https://github.com/user-attachments/assets/cac6a17a-1eeb-4dde-9818-cdf85d8ea98f
|
||||
|
||||
## Why Pi Web Access
|
||||
|
||||
**Zero Config** — Signed into Google in Chrome? That's it. The extension reads your Chrome session cookies to access Gemini directly. No API keys, no setup, no subscriptions.
|
||||
|
||||
**Video Understanding** — Point it at a YouTube video or local screen recording and ask questions about what's on screen. Full transcripts, visual descriptions, and frame extraction at exact timestamps.
|
||||
|
||||
**Smart Fallbacks** — Every capability has a fallback chain. Search tries Perplexity, then Gemini API, then Gemini Web. YouTube tries Gemini Web, then API, then Perplexity. Blocked pages retry through Jina Reader and Gemini extraction. Something always works.
|
||||
|
||||
**GitHub Cloning** — GitHub URLs are cloned locally instead of scraped. The agent gets real file contents and a local path to explore, not rendered HTML.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
pi install npm:pi-web-access
|
||||
```
|
||||
|
||||
If you're not signed into Chrome, or prefer a different provider, add API keys to `~/.pi/web-search.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"perplexityApiKey": "pplx-...",
|
||||
"geminiApiKey": "AIza..."
|
||||
}
|
||||
```
|
||||
|
||||
You can configure one or both. In `auto` mode (default), `web_search` tries Perplexity first, then Gemini API, then Gemini Web.
|
||||
|
||||
Optional dependencies for video frame extraction:
|
||||
|
||||
```bash
|
||||
brew install ffmpeg # frame extraction, video thumbnails, local video duration
|
||||
brew install yt-dlp # YouTube stream URLs for frame extraction
|
||||
```
|
||||
|
||||
Without these, video content analysis (transcripts, visual descriptions via Gemini) still works. The binaries are only needed for extracting individual frames as images.
|
||||
|
||||
Requires Pi v0.37.3+.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```typescript
|
||||
// Search the web
|
||||
web_search({ query: "TypeScript best practices 2025" })
|
||||
|
||||
// Fetch a page
|
||||
fetch_content({ url: "https://docs.example.com/guide" })
|
||||
|
||||
// Clone a GitHub repo
|
||||
fetch_content({ url: "https://github.com/owner/repo" })
|
||||
|
||||
// Understand a YouTube video
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", prompt: "What libraries are shown?" })
|
||||
|
||||
// Analyze a screen recording
|
||||
fetch_content({ url: "/path/to/recording.mp4", prompt: "What error appears on screen?" })
|
||||
```
|
||||
|
||||
## Tools
|
||||
|
||||
### web_search
|
||||
|
||||
Search the web via Perplexity AI or Gemini. Returns a synthesized answer with source citations.
|
||||
|
||||
```typescript
|
||||
web_search({ query: "rust async programming" })
|
||||
web_search({ queries: ["query 1", "query 2"] })
|
||||
web_search({ query: "latest news", numResults: 10, recencyFilter: "week" })
|
||||
web_search({ query: "...", domainFilter: ["github.com"] })
|
||||
web_search({ query: "...", provider: "gemini" })
|
||||
web_search({ query: "...", includeContent: true })
|
||||
web_search({ queries: ["query 1", "query 2"], curate: true })
|
||||
```
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| `query` / `queries` | Single query or batch of queries |
|
||||
| `numResults` | Results per query (default: 5, max: 20) |
|
||||
| `recencyFilter` | `day`, `week`, `month`, or `year` |
|
||||
| `domainFilter` | Limit to domains (prefix with `-` to exclude) |
|
||||
| `provider` | `auto` (default), `perplexity`, or `gemini` |
|
||||
| `includeContent` | Fetch full page content from sources in background |
|
||||
| `curate` | Hold results for browser review (default: true for multi-query). Press Ctrl+Shift+S to open browser UI, or wait for countdown to auto-condense and send. Set to false to skip both curation and condensation. |
|
||||
|
||||
### fetch_content
|
||||
|
||||
Fetch URL(s) and extract readable content as markdown. Automatically detects and handles GitHub repos, YouTube videos, PDFs, local video files, and regular web pages.
|
||||
|
||||
```typescript
|
||||
fetch_content({ url: "https://example.com/article" })
|
||||
fetch_content({ urls: ["url1", "url2", "url3"] })
|
||||
fetch_content({ url: "https://github.com/owner/repo" })
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", prompt: "What libraries are shown?" })
|
||||
fetch_content({ url: "/path/to/recording.mp4", prompt: "What error appears on screen?" })
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", timestamp: "23:41-25:00", frames: 4 })
|
||||
```
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| `url` / `urls` | Single URL/path or multiple URLs |
|
||||
| `prompt` | Question to ask about a YouTube video or local video file |
|
||||
| `timestamp` | Extract frame(s) — single (`"23:41"`), range (`"23:41-25:00"`), or seconds (`"85"`) |
|
||||
| `frames` | Number of frames to extract (max 12) |
|
||||
| `forceClone` | Clone GitHub repos that exceed the 350MB size threshold |
|
||||
|
||||
### get_search_content
|
||||
|
||||
Retrieve stored content from previous searches or fetches. Content over 30,000 chars is truncated in tool responses but stored in full for retrieval here.
|
||||
|
||||
```typescript
|
||||
get_search_content({ responseId: "abc123", urlIndex: 0 })
|
||||
get_search_content({ responseId: "abc123", url: "https://..." })
|
||||
get_search_content({ responseId: "abc123", query: "original query" })
|
||||
```
|
||||
|
||||
## Capabilities
|
||||
|
||||
### GitHub repos
|
||||
|
||||
GitHub URLs are cloned locally instead of scraped. The agent gets real file contents and a local path to explore with `read` and `bash`. Root URLs return the repo tree + README, `/tree/` paths return directory listings, `/blob/` paths return file contents.
|
||||
|
||||
Repos over 350MB get a lightweight API-based view instead of a full clone (override with `forceClone: true`). Commit SHA URLs are handled via the API. Clones are cached for the session and wiped on session change. Private repos require the `gh` CLI.
|
||||
|
||||
### YouTube videos
|
||||
|
||||
YouTube URLs are processed via Gemini for full video understanding — visual descriptions, transcripts with timestamps, and chapter markers. Pass a `prompt` to ask specific questions about the video. Results include the video thumbnail so the agent gets visual context alongside the transcript.
|
||||
|
||||
Fallback: Gemini Web → Gemini API → Perplexity (text summary only). Handles all URL formats: `/watch?v=`, `youtu.be/`, `/shorts/`, `/live/`, `/embed/`, `/v/`.
|
||||
|
||||
### Local video files
|
||||
|
||||
Pass a file path (`/`, `./`, `../`, or `file://` prefix) to analyze video content via Gemini. Supports MP4, MOV, WebM, AVI, and other common formats up to 50MB. Pass a `prompt` to ask about specific content. If ffmpeg is installed, a thumbnail frame is included alongside the analysis.
|
||||
|
||||
Fallback: Gemini API (Files API upload) → Gemini Web.
|
||||
|
||||
### Video frame extraction
|
||||
|
||||
Use `timestamp` and/or `frames` on any YouTube URL or local video file to extract visual frames as images.
|
||||
|
||||
```typescript
|
||||
fetch_content({ url: "...", timestamp: "23:41" }) // single frame
|
||||
fetch_content({ url: "...", timestamp: "23:41-25:00" }) // range, 6 frames
|
||||
fetch_content({ url: "...", timestamp: "23:41-25:00", frames: 3 }) // range, custom count
|
||||
fetch_content({ url: "...", timestamp: "23:41", frames: 5 }) // 5 frames at 5s intervals
|
||||
fetch_content({ url: "...", frames: 6 }) // sample whole video
|
||||
```
|
||||
|
||||
Requires `ffmpeg` (and `yt-dlp` for YouTube). Timestamps accept `H:MM:SS`, `MM:SS`, or bare seconds.
|
||||
|
||||
### PDFs
|
||||
|
||||
PDF URLs are extracted as text and saved to `~/Downloads/` as markdown. The agent can then `read` specific sections without loading the full document into context. Text-based extraction only — no OCR.
|
||||
|
||||
### Blocked pages
|
||||
|
||||
When Readability fails or returns only a cookie notice, the extension retries via Jina Reader (handles JS rendering server-side, no API key needed), then Gemini URL Context API, then Gemini Web extraction. Handles SPAs, JS-heavy pages, and anti-bot protections transparently. Also parses Next.js RSC flight data when present.
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
fetch_content(url)
|
||||
→ Video file? Gemini API (Files API) → Gemini Web
|
||||
→ GitHub URL? Clone repo, return file contents + local path
|
||||
→ YouTube URL? Gemini Web → Gemini API → Perplexity
|
||||
→ HTTP fetch → PDF? Extract text, save to ~/Downloads/
|
||||
→ HTML? Readability → RSC parser → Jina Reader → Gemini fallback
|
||||
→ Text/JSON/Markdown? Return directly
|
||||
```
|
||||
|
||||
## Skills
|
||||
|
||||
### librarian
|
||||
|
||||
Bundled research workflow for investigating open-source libraries. Combines GitHub cloning, web search, and git operations (blame, log, show) to produce evidence-backed answers with permalinks. Pi loads it automatically based on your prompt. Also available via `/skill:librarian` with [pi-skill-palette](https://github.com/nicobailon/pi-skill-palette).
|
||||
|
||||
## Commands
|
||||
|
||||
### /websearch
|
||||
|
||||
Open the search curator directly in the browser. Runs searches and lets you review, add, and select results to send back to the agent — no LLM round-trip needed.
|
||||
|
||||
```
|
||||
/websearch # empty page, type your own searches
|
||||
/websearch react hooks, next.js caching # pre-fill with comma-separated queries
|
||||
```
|
||||
|
||||
Results get injected into the conversation when you click Send. The agent sees them and can use them immediately.
|
||||
|
||||
### /search
|
||||
|
||||
Browse stored search results interactively. Lists all results from the current session with their response IDs for easy retrieval.
|
||||
|
||||
## Activity Monitor
|
||||
|
||||
Toggle with **Ctrl+Shift+W** to see live request/response activity:
|
||||
|
||||
```
|
||||
─── Web Search Activity ────────────────────────────────────
|
||||
API "typescript best practices" 200 2.1s ✓
|
||||
GET docs.example.com/article 200 0.8s ✓
|
||||
GET blog.example.com/post 404 0.3s ✗
|
||||
────────────────────────────────────────────────────────────
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
All config lives in `~/.pi/web-search.json`. Every field is optional.
|
||||
|
||||
```json
|
||||
{
|
||||
"perplexityApiKey": "pplx-...",
|
||||
"geminiApiKey": "AIza...",
|
||||
"provider": "perplexity",
|
||||
"curateWindow": 10,
|
||||
"autoFilter": true,
|
||||
"githubClone": {
|
||||
"enabled": true,
|
||||
"maxRepoSizeMB": 350,
|
||||
"cloneTimeoutSeconds": 30,
|
||||
"clonePath": "/tmp/pi-github-repos"
|
||||
},
|
||||
"youtube": {
|
||||
"enabled": true,
|
||||
"preferredModel": "gemini-3-flash-preview"
|
||||
},
|
||||
"video": {
|
||||
"enabled": true,
|
||||
"preferredModel": "gemini-3-flash-preview",
|
||||
"maxSizeMB": 50
|
||||
},
|
||||
"shortcuts": {
|
||||
"curate": "ctrl+shift+s",
|
||||
"activity": "ctrl+shift+w"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`GEMINI_API_KEY` and `PERPLEXITY_API_KEY` env vars take precedence over config file values. `provider` sets the default search provider: `"perplexity"` or `"gemini"`. This is also updated automatically when you change the provider in the curator UI. `curateWindow` controls how many seconds multi-query searches wait before auto-sending results (default: 10). During the countdown, press Ctrl+Shift+S to open the browser curator. Set to 0 to always send immediately (Ctrl+Shift+S still works during the search itself).
|
||||
|
||||
### Shortcuts
|
||||
|
||||
Both shortcuts are configurable via `~/.pi/web-search.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"shortcuts": {
|
||||
"curate": "ctrl+shift+s",
|
||||
"activity": "ctrl+shift+w"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Values use the same format as pi keybindings (e.g. `ctrl+s`, `ctrl+shift+s`, `alt+r`). Changes take effect on next pi restart.
|
||||
|
||||
### Auto-Condense
|
||||
|
||||
Multi-query searches are automatically condensed into a deduplicated briefing when the countdown expires without manual curation. A single LLM call receives all search results — enriched with preprocessing analysis (URL overlap, answer similarity, source quality tiers) — and produces a concise synthesis organized by topic. Irrelevant or off-topic results are skipped automatically.
|
||||
|
||||
```json
|
||||
{
|
||||
"autoFilter": {
|
||||
"enabled": true,
|
||||
"model": "anthropic/claude-haiku-4-5",
|
||||
"prompt": "You are a research assistant..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `enabled` | `true` to enable, `false` to disable. Omit `autoFilter` entirely to enable with defaults. |
|
||||
| `model` | LLM model in `provider/model` format. Uses pi's model registry for auth. Default: `anthropic/claude-haiku-4-5`. |
|
||||
| `prompt` | System prompt for the condenser. Should instruct the model to synthesize, deduplicate, and cite sources. Omit to use the built-in prompt. |
|
||||
|
||||
Shorthand: `"autoFilter": true` or `"autoFilter": false` for enable/disable without customizing model or prompt.
|
||||
|
||||
The model uses pi's model registry, so any configured provider works — including custom gateways, OAuth tokens, and API keys. If the model isn't found in the registry or has no credentials, condensation is silently skipped.
|
||||
|
||||
The `web_search` tool also accepts an optional `context` parameter — a brief description of the user's current task or goal. When provided, the condenser uses it to focus the briefing (e.g., "building a Shopify checkout extension" helps it emphasize relevant findings over general noise).
|
||||
|
||||
Set `"enabled": false` under any feature to disable it. Config changes require a Pi restart.
|
||||
|
||||
Rate limits: Perplexity is capped at 10 requests/minute (client-side). Content fetches run 3 concurrent with a 30s timeout per URL.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Chrome cookie extraction is macOS-only — other platforms fall through to API keys. First-time access may trigger a Keychain dialog.
|
||||
- YouTube private/age-restricted videos may fail on all extraction paths.
|
||||
- Gemini can process videos up to ~1 hour; longer videos may be truncated.
|
||||
- PDFs are text-extracted only (no OCR for scanned documents).
|
||||
- GitHub branch names with slashes may misresolve file paths; the clone still works and the agent can navigate manually.
|
||||
- Non-code GitHub URLs (issues, PRs, wiki) fall through to normal web extraction.
|
||||
|
||||
<details>
|
||||
<summary>Files</summary>
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `index.ts` | Extension entry, tool definitions, commands, widget |
|
||||
| `curator-page.ts` | HTML/CSS/JS generation for the curator UI with markdown rendering |
|
||||
| `curator-server.ts` | Ephemeral HTTP server with SSE streaming and state machine |
|
||||
| `search-filter.ts` | Auto-condense pipeline — preprocessing, LLM condensation, and post-processing for multi-query results |
|
||||
| `extract.ts` | URL/file path routing, HTTP extraction, fallback orchestration |
|
||||
| `gemini-search.ts` | Search routing across Perplexity, Gemini API, Gemini Web |
|
||||
| `gemini-url-context.ts` | Gemini URL Context + Web extraction fallbacks |
|
||||
| `gemini-web.ts` | Gemini Web client (cookie auth, StreamGenerate) |
|
||||
| `gemini-api.ts` | Gemini REST API client (generateContent) |
|
||||
| `chrome-cookies.ts` | macOS Chrome cookie extraction (Keychain + SQLite) |
|
||||
| `youtube-extract.ts` | YouTube detection, three-tier extraction, frame extraction |
|
||||
| `video-extract.ts` | Local video detection, Files API upload, Gemini analysis |
|
||||
| `github-extract.ts` | GitHub URL parsing, clone cache, content generation |
|
||||
| `github-api.ts` | GitHub API fallback for large repos and commit SHAs |
|
||||
| `perplexity.ts` | Perplexity API client with rate limiting |
|
||||
| `pdf-extract.ts` | PDF text extraction, saves to markdown |
|
||||
| `rsc-extract.ts` | RSC flight data parser for Next.js pages |
|
||||
| `utils.ts` | Shared formatting and error helpers |
|
||||
| `storage.ts` | Session-aware result storage |
|
||||
| `activity.ts` | Activity tracking for the observability widget |
|
||||
| `skills/librarian/` | Bundled skill for library research |
|
||||
|
||||
</details>
|
||||
102
pi/files/agent/extensions/pi-web-access/activity.ts
Normal file
102
pi/files/agent/extensions/pi-web-access/activity.ts
Normal file
@@ -0,0 +1,102 @@
|
||||
// Types
|
||||
export interface ActivityEntry {
|
||||
id: string;
|
||||
type: "api" | "fetch";
|
||||
startTime: number;
|
||||
endTime?: number;
|
||||
|
||||
// For API calls
|
||||
query?: string;
|
||||
|
||||
// For URL fetches
|
||||
url?: string;
|
||||
|
||||
// Result - status is number (HTTP code) or null (pending/network error)
|
||||
status: number | null;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface RateLimitInfo {
|
||||
used: number;
|
||||
max: number;
|
||||
oldestTimestamp: number | null;
|
||||
windowMs: number;
|
||||
}
|
||||
|
||||
export class ActivityMonitor {
|
||||
private entries: ActivityEntry[] = [];
|
||||
private readonly maxEntries = 10;
|
||||
private listeners = new Set<() => void>();
|
||||
private rateLimitInfo: RateLimitInfo = { used: 0, max: 10, oldestTimestamp: null, windowMs: 60000 };
|
||||
private nextId = 1;
|
||||
|
||||
logStart(partial: Omit<ActivityEntry, "id" | "startTime" | "status">): string {
|
||||
const id = `act-${this.nextId++}`;
|
||||
const entry: ActivityEntry = {
|
||||
...partial,
|
||||
id,
|
||||
startTime: Date.now(),
|
||||
status: null,
|
||||
};
|
||||
this.entries.push(entry);
|
||||
if (this.entries.length > this.maxEntries) {
|
||||
this.entries.shift();
|
||||
}
|
||||
this.notify();
|
||||
return id;
|
||||
}
|
||||
|
||||
logComplete(id: string, status: number): void {
|
||||
const entry = this.entries.find((e) => e.id === id);
|
||||
if (entry) {
|
||||
entry.endTime = Date.now();
|
||||
entry.status = status;
|
||||
this.notify();
|
||||
}
|
||||
}
|
||||
|
||||
logError(id: string, error: string): void {
|
||||
const entry = this.entries.find((e) => e.id === id);
|
||||
if (entry) {
|
||||
entry.endTime = Date.now();
|
||||
entry.error = error;
|
||||
this.notify();
|
||||
}
|
||||
}
|
||||
|
||||
getEntries(): readonly ActivityEntry[] {
|
||||
return this.entries;
|
||||
}
|
||||
|
||||
getRateLimitInfo(): RateLimitInfo {
|
||||
return this.rateLimitInfo;
|
||||
}
|
||||
|
||||
updateRateLimit(info: RateLimitInfo): void {
|
||||
this.rateLimitInfo = info;
|
||||
this.notify();
|
||||
}
|
||||
|
||||
onUpdate(callback: () => void): () => void {
|
||||
this.listeners.add(callback);
|
||||
return () => this.listeners.delete(callback);
|
||||
}
|
||||
|
||||
clear(): void {
|
||||
this.entries = [];
|
||||
this.rateLimitInfo = { used: 0, max: 10, oldestTimestamp: null, windowMs: 60000 };
|
||||
this.notify();
|
||||
}
|
||||
|
||||
private notify(): void {
|
||||
for (const cb of this.listeners) {
|
||||
try {
|
||||
cb();
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const activityMonitor = new ActivityMonitor();
|
||||
BIN
pi/files/agent/extensions/pi-web-access/banner.png
Normal file
BIN
pi/files/agent/extensions/pi-web-access/banner.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.2 MiB |
321
pi/files/agent/extensions/pi-web-access/chrome-cookies.ts
Normal file
321
pi/files/agent/extensions/pi-web-access/chrome-cookies.ts
Normal file
@@ -0,0 +1,321 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { pbkdf2Sync, createDecipheriv } from "node:crypto";
|
||||
import { copyFileSync, existsSync, mkdtempSync, rmSync } from "node:fs";
|
||||
import { tmpdir, homedir, platform } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
export type CookieMap = Record<string, string>;
|
||||
|
||||
const GOOGLE_ORIGINS = [
|
||||
"https://gemini.google.com",
|
||||
"https://accounts.google.com",
|
||||
"https://www.google.com",
|
||||
];
|
||||
|
||||
const ALL_COOKIE_NAMES = new Set([
|
||||
"__Secure-1PSID",
|
||||
"__Secure-1PSIDTS",
|
||||
"__Secure-1PSIDCC",
|
||||
"__Secure-1PAPISID",
|
||||
"NID",
|
||||
"AEC",
|
||||
"SOCS",
|
||||
"__Secure-BUCKET",
|
||||
"__Secure-ENID",
|
||||
"SID",
|
||||
"HSID",
|
||||
"SSID",
|
||||
"APISID",
|
||||
"SAPISID",
|
||||
"__Secure-3PSID",
|
||||
"__Secure-3PSIDTS",
|
||||
"__Secure-3PAPISID",
|
||||
"SIDCC",
|
||||
]);
|
||||
|
||||
function getChromeCookiesPath(): string | null {
|
||||
const plat = platform();
|
||||
if (plat === "darwin") {
|
||||
return join(homedir(), "Library/Application Support/Google/Chrome/Default/Cookies");
|
||||
}
|
||||
if (plat === "linux") {
|
||||
const chromiumPath = join(homedir(), ".config/chromium/Default/Cookies");
|
||||
if (existsSync(chromiumPath)) return chromiumPath;
|
||||
|
||||
const chromePath = join(homedir(), ".config/google-chrome/Default/Cookies");
|
||||
if (existsSync(chromePath)) return chromePath;
|
||||
|
||||
return null;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function getGoogleCookies(): Promise<{ cookies: CookieMap; warnings: string[] } | null> {
|
||||
const plat = platform();
|
||||
if (plat !== "darwin" && plat !== "linux") return null;
|
||||
|
||||
const cookiesPath = getChromeCookiesPath();
|
||||
if (!cookiesPath) return null;
|
||||
|
||||
const warnings: string[] = [];
|
||||
|
||||
const password = await readEncryptionPassword();
|
||||
if (!password) {
|
||||
warnings.push("Could not read Chrome encryption password");
|
||||
return { cookies: {}, warnings };
|
||||
}
|
||||
|
||||
const macKey = plat === "darwin" ? pbkdf2Sync(password, "saltysalt", 1003, 16, "sha1") : Buffer.alloc(0);
|
||||
const tempDir = mkdtempSync(join(tmpdir(), "pi-chrome-cookies-"));
|
||||
|
||||
try {
|
||||
const tempDb = join(tempDir, "Cookies");
|
||||
copyFileSync(cookiesPath, tempDb);
|
||||
copySidecar(cookiesPath, tempDb, "-wal");
|
||||
copySidecar(cookiesPath, tempDb, "-shm");
|
||||
|
||||
const metaVersion = await readMetaVersion(tempDb);
|
||||
const stripHash = metaVersion >= 24;
|
||||
|
||||
const hosts = GOOGLE_ORIGINS.map((o) => new URL(o).hostname);
|
||||
const rows = await queryCookieRows(tempDb, hosts);
|
||||
if (!rows) {
|
||||
warnings.push("Failed to query Chrome cookie database");
|
||||
return { cookies: {}, warnings };
|
||||
}
|
||||
|
||||
const cookies: CookieMap = {};
|
||||
for (const row of rows) {
|
||||
const name = row.name as string;
|
||||
if (!ALL_COOKIE_NAMES.has(name)) continue;
|
||||
if (cookies[name]) continue;
|
||||
|
||||
let value = typeof row.value === "string" && row.value.length > 0 ? row.value : null;
|
||||
if (!value) {
|
||||
const encrypted = row.encrypted_value;
|
||||
if (encrypted instanceof Uint8Array) {
|
||||
if (plat === "linux") {
|
||||
value = decryptLinuxCookieValue(encrypted, password, stripHash);
|
||||
} else {
|
||||
value = decryptCookieValue(encrypted, macKey, stripHash);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (value) cookies[name] = value;
|
||||
}
|
||||
|
||||
return { cookies, warnings };
|
||||
} finally {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function decryptCookieValue(encrypted: Uint8Array, key: Buffer, stripHash: boolean): string | null {
|
||||
const buf = Buffer.from(encrypted);
|
||||
if (buf.length < 3) return null;
|
||||
|
||||
const prefix = buf.subarray(0, 3).toString("utf8");
|
||||
if (!/^v\d\d$/.test(prefix)) return null;
|
||||
|
||||
const ciphertext = buf.subarray(3);
|
||||
if (!ciphertext.length) return "";
|
||||
|
||||
try {
|
||||
const iv = Buffer.alloc(16, 0x20);
|
||||
const decipher = createDecipheriv("aes-128-cbc", key, iv);
|
||||
decipher.setAutoPadding(false);
|
||||
const plaintext = Buffer.concat([decipher.update(ciphertext), decipher.final()]);
|
||||
const unpadded = removePkcs7Padding(plaintext);
|
||||
const bytes = stripHash && unpadded.length >= 32 ? unpadded.subarray(32) : unpadded;
|
||||
const decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
|
||||
let i = 0;
|
||||
while (i < decoded.length && decoded.charCodeAt(i) < 0x20) i++;
|
||||
return decoded.slice(i);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function decryptLinuxCookieValue(encrypted: Uint8Array, keyringSecret: string, stripHash: boolean): string | null {
|
||||
const buf = Buffer.from(encrypted);
|
||||
if (buf.length < 3) return null;
|
||||
|
||||
const prefix = buf.subarray(0, 3).toString("utf8");
|
||||
if (prefix !== "v10" && prefix !== "v11") return null;
|
||||
|
||||
const ciphertext = buf.subarray(3);
|
||||
if (!ciphertext.length) return "";
|
||||
|
||||
try {
|
||||
const key = pbkdf2Sync(keyringSecret, "saltysalt", 1, 16, "sha1");
|
||||
const iv = Buffer.alloc(16, 0x20);
|
||||
const decipher = createDecipheriv("aes-128-cbc", key, iv);
|
||||
decipher.setAutoPadding(false);
|
||||
const plaintext = Buffer.concat([decipher.update(ciphertext), decipher.final()]);
|
||||
const unpadded = removePkcs7Padding(plaintext);
|
||||
const bytes = stripHash && unpadded.length >= 32 ? unpadded.subarray(32) : unpadded;
|
||||
const decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
|
||||
let i = 0;
|
||||
while (i < decoded.length && decoded.charCodeAt(i) < 0x20) i++;
|
||||
return decoded.slice(i);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function removePkcs7Padding(buf: Buffer): Buffer {
|
||||
if (!buf.length) return buf;
|
||||
const padding = buf[buf.length - 1];
|
||||
if (!padding || padding > 16) return buf;
|
||||
return buf.subarray(0, buf.length - padding);
|
||||
}
|
||||
|
||||
function readKeychainPassword(): Promise<string | null> {
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
"security",
|
||||
["find-generic-password", "-w", "-a", "Chrome", "-s", "Chrome Safe Storage"],
|
||||
{ timeout: 5000 },
|
||||
(err, stdout) => {
|
||||
if (err) { resolve(null); return; }
|
||||
resolve(stdout.trim() || null);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function readLinuxKeyringPassword(): Promise<string | null> {
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
"secret-tool",
|
||||
["lookup", "application", "chromium"],
|
||||
{ timeout: 5000 },
|
||||
(err, stdout) => {
|
||||
if (!err && stdout.trim()) {
|
||||
resolve(stdout.trim());
|
||||
return;
|
||||
}
|
||||
execFile(
|
||||
"secret-tool",
|
||||
["lookup", "application", "chrome"],
|
||||
{ timeout: 5000 },
|
||||
(err2, stdout2) => {
|
||||
if (!err2 && stdout2.trim()) {
|
||||
resolve(stdout2.trim());
|
||||
return;
|
||||
}
|
||||
resolve("peanuts");
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
async function readEncryptionPassword(): Promise<string | null> {
|
||||
const plat = platform();
|
||||
if (plat === "darwin") return readKeychainPassword();
|
||||
if (plat === "linux") return readLinuxKeyringPassword();
|
||||
return null;
|
||||
}
|
||||
|
||||
let sqliteModule: typeof import("node:sqlite") | null = null;
|
||||
|
||||
async function importSqlite(): Promise<typeof import("node:sqlite") | null> {
|
||||
if (sqliteModule) return sqliteModule;
|
||||
const orig = process.emitWarning.bind(process);
|
||||
process.emitWarning = ((warning: string | Error, ...args: unknown[]) => {
|
||||
const msg = typeof warning === "string" ? warning : warning?.message ?? "";
|
||||
if (msg.includes("SQLite is an experimental feature")) return;
|
||||
return (orig as Function)(warning, ...args);
|
||||
}) as typeof process.emitWarning;
|
||||
try {
|
||||
sqliteModule = await import("node:sqlite");
|
||||
return sqliteModule;
|
||||
} catch {
|
||||
return null;
|
||||
} finally {
|
||||
process.emitWarning = orig;
|
||||
}
|
||||
}
|
||||
|
||||
function supportsReadBigInts(): boolean {
|
||||
const [major, minor] = process.versions.node.split(".").map(Number);
|
||||
if (major > 24) return true;
|
||||
if (major < 24) return false;
|
||||
return minor >= 4;
|
||||
}
|
||||
|
||||
async function readMetaVersion(dbPath: string): Promise<number> {
|
||||
const sqlite = await importSqlite();
|
||||
if (!sqlite) return 0;
|
||||
const opts: Record<string, unknown> = { readOnly: true };
|
||||
if (supportsReadBigInts()) opts.readBigInts = true;
|
||||
const db = new sqlite.DatabaseSync(dbPath, opts);
|
||||
try {
|
||||
const rows = db.prepare("SELECT value FROM meta WHERE key = 'version'").all() as Array<Record<string, unknown>>;
|
||||
const val = rows[0]?.value;
|
||||
if (typeof val === "number") return Math.floor(val);
|
||||
if (typeof val === "bigint") return Number(val);
|
||||
if (typeof val === "string") return parseInt(val, 10) || 0;
|
||||
return 0;
|
||||
} catch {
|
||||
return 0;
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
async function queryCookieRows(
|
||||
dbPath: string,
|
||||
hosts: string[],
|
||||
): Promise<Array<Record<string, unknown>> | null> {
|
||||
const sqlite = await importSqlite();
|
||||
if (!sqlite) return null;
|
||||
|
||||
const clauses: string[] = [];
|
||||
for (const host of hosts) {
|
||||
for (const candidate of expandHosts(host)) {
|
||||
const esc = candidate.replaceAll("'", "''");
|
||||
clauses.push(`host_key = '${esc}'`);
|
||||
clauses.push(`host_key = '.${esc}'`);
|
||||
clauses.push(`host_key LIKE '%.${esc}'`);
|
||||
}
|
||||
}
|
||||
const where = clauses.join(" OR ");
|
||||
|
||||
const opts: Record<string, unknown> = { readOnly: true };
|
||||
if (supportsReadBigInts()) opts.readBigInts = true;
|
||||
const db = new sqlite.DatabaseSync(dbPath, opts);
|
||||
try {
|
||||
return db
|
||||
.prepare(
|
||||
`SELECT name, value, host_key, encrypted_value FROM cookies WHERE (${where}) ORDER BY expires_utc DESC`,
|
||||
)
|
||||
.all() as Array<Record<string, unknown>>;
|
||||
} catch {
|
||||
return null;
|
||||
} finally {
|
||||
db.close();
|
||||
}
|
||||
}
|
||||
|
||||
function expandHosts(host: string): string[] {
|
||||
const parts = host.split(".").filter(Boolean);
|
||||
if (parts.length <= 1) return [host];
|
||||
const candidates = new Set<string>();
|
||||
candidates.add(host);
|
||||
for (let i = 1; i <= parts.length - 2; i++) {
|
||||
const c = parts.slice(i).join(".");
|
||||
if (c) candidates.add(c);
|
||||
}
|
||||
return Array.from(candidates);
|
||||
}
|
||||
|
||||
function copySidecar(srcDb: string, targetDb: string, suffix: string): void {
|
||||
const sidecar = `${srcDb}${suffix}`;
|
||||
if (!existsSync(sidecar)) return;
|
||||
try {
|
||||
copyFileSync(sidecar, `${targetDb}${suffix}`);
|
||||
} catch {}
|
||||
}
|
||||
1209
pi/files/agent/extensions/pi-web-access/curator-page.ts
Normal file
1209
pi/files/agent/extensions/pi-web-access/curator-page.ts
Normal file
File diff suppressed because it is too large
Load Diff
325
pi/files/agent/extensions/pi-web-access/curator-server.ts
Normal file
325
pi/files/agent/extensions/pi-web-access/curator-server.ts
Normal file
@@ -0,0 +1,325 @@
|
||||
import http, { type IncomingMessage, type ServerResponse } from "node:http";
|
||||
import { generateCuratorPage } from "./curator-page.js";
|
||||
|
||||
const STALE_THRESHOLD_MS = 30000;
|
||||
const WATCHDOG_INTERVAL_MS = 5000;
|
||||
const MAX_BODY_SIZE = 64 * 1024;
|
||||
|
||||
type ServerState = "SEARCHING" | "RESULT_SELECTION" | "COMPLETED";
|
||||
|
||||
export interface CuratorServerOptions {
|
||||
queries: string[];
|
||||
sessionToken: string;
|
||||
timeout: number;
|
||||
availableProviders: { perplexity: boolean; gemini: boolean };
|
||||
defaultProvider: string;
|
||||
}
|
||||
|
||||
export interface CuratorServerCallbacks {
|
||||
onSubmit: (selectedQueryIndices: number[]) => void;
|
||||
onCancel: (reason: "user" | "timeout" | "stale") => void;
|
||||
onProviderChange: (provider: string) => void;
|
||||
onAddSearch: (query: string, queryIndex: number) => Promise<{ answer: string; results: Array<{ title: string; url: string; domain: string }> }>;
|
||||
}
|
||||
|
||||
export interface CuratorServerHandle {
|
||||
server: http.Server;
|
||||
url: string;
|
||||
close: () => void;
|
||||
pushResult: (queryIndex: number, data: { answer: string; results: Array<{ title: string; url: string; domain: string }> }) => void;
|
||||
pushError: (queryIndex: number, error: string) => void;
|
||||
searchesDone: () => void;
|
||||
}
|
||||
|
||||
function sendJson(res: ServerResponse, status: number, payload: unknown): void {
|
||||
res.writeHead(status, {
|
||||
"Content-Type": "application/json",
|
||||
"Cache-Control": "no-store",
|
||||
});
|
||||
res.end(JSON.stringify(payload));
|
||||
}
|
||||
|
||||
function parseJSONBody(req: IncomingMessage): Promise<unknown> {
|
||||
return new Promise((resolve, reject) => {
|
||||
let body = "";
|
||||
let size = 0;
|
||||
req.on("data", (chunk: Buffer) => {
|
||||
size += chunk.length;
|
||||
if (size > MAX_BODY_SIZE) {
|
||||
req.destroy();
|
||||
reject(new Error("Request body too large"));
|
||||
return;
|
||||
}
|
||||
body += chunk.toString();
|
||||
});
|
||||
req.on("end", () => {
|
||||
try { resolve(JSON.parse(body)); }
|
||||
catch { reject(new Error("Invalid JSON")); }
|
||||
});
|
||||
req.on("error", reject);
|
||||
});
|
||||
}
|
||||
|
||||
export function startCuratorServer(
|
||||
options: CuratorServerOptions,
|
||||
callbacks: CuratorServerCallbacks,
|
||||
): Promise<CuratorServerHandle> {
|
||||
const { queries, sessionToken, timeout, availableProviders, defaultProvider } = options;
|
||||
let browserConnected = false;
|
||||
let lastHeartbeatAt = Date.now();
|
||||
let completed = false;
|
||||
let watchdog: NodeJS.Timeout | null = null;
|
||||
let state: ServerState = "SEARCHING";
|
||||
let sseResponse: ServerResponse | null = null;
|
||||
const sseBuffer: string[] = [];
|
||||
let nextQueryIndex = queries.length;
|
||||
|
||||
let sseKeepalive: NodeJS.Timeout | null = null;
|
||||
|
||||
const markCompleted = (): boolean => {
|
||||
if (completed) return false;
|
||||
completed = true;
|
||||
state = "COMPLETED";
|
||||
if (watchdog) { clearInterval(watchdog); watchdog = null; }
|
||||
if (sseKeepalive) { clearInterval(sseKeepalive); sseKeepalive = null; }
|
||||
if (sseResponse) {
|
||||
try { sseResponse.end(); } catch {}
|
||||
sseResponse = null;
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
const touchHeartbeat = (): void => {
|
||||
lastHeartbeatAt = Date.now();
|
||||
browserConnected = true;
|
||||
};
|
||||
|
||||
function validateToken(body: unknown, res: ServerResponse): boolean {
|
||||
if (!body || typeof body !== "object") {
|
||||
sendJson(res, 400, { ok: false, error: "Invalid body" });
|
||||
return false;
|
||||
}
|
||||
if ((body as { token?: string }).token !== sessionToken) {
|
||||
sendJson(res, 403, { ok: false, error: "Invalid session" });
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function sendSSE(event: string, data: unknown): void {
|
||||
const payload = `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`;
|
||||
const res = sseResponse;
|
||||
if (res && !res.writableEnded && res.socket && !res.socket.destroyed) {
|
||||
try {
|
||||
const ok = res.write(payload);
|
||||
if (!ok) res.once("drain", () => {});
|
||||
} catch {
|
||||
sseBuffer.push(payload);
|
||||
}
|
||||
} else {
|
||||
sseBuffer.push(payload);
|
||||
}
|
||||
}
|
||||
|
||||
const pageHtml = generateCuratorPage(queries, sessionToken, timeout, availableProviders, defaultProvider);
|
||||
|
||||
const server = http.createServer(async (req, res) => {
|
||||
try {
|
||||
const method = req.method || "GET";
|
||||
const url = new URL(req.url || "/", `http://${req.headers.host || "127.0.0.1"}`);
|
||||
|
||||
if (method === "GET" && url.pathname === "/") {
|
||||
const token = url.searchParams.get("session");
|
||||
if (token !== sessionToken) {
|
||||
res.writeHead(403, { "Content-Type": "text/plain" });
|
||||
res.end("Invalid session");
|
||||
return;
|
||||
}
|
||||
touchHeartbeat();
|
||||
res.writeHead(200, {
|
||||
"Content-Type": "text/html; charset=utf-8",
|
||||
"Cache-Control": "no-store",
|
||||
});
|
||||
res.end(pageHtml);
|
||||
return;
|
||||
}
|
||||
|
||||
if (method === "GET" && url.pathname === "/events") {
|
||||
const token = url.searchParams.get("session");
|
||||
if (token !== sessionToken) {
|
||||
res.writeHead(403, { "Content-Type": "text/plain" });
|
||||
res.end("Invalid session");
|
||||
return;
|
||||
}
|
||||
if (state === "COMPLETED") {
|
||||
sendJson(res, 409, { ok: false, error: "No events available" });
|
||||
return;
|
||||
}
|
||||
if (sseResponse) {
|
||||
try { sseResponse.end(); } catch {}
|
||||
}
|
||||
res.writeHead(200, {
|
||||
"Content-Type": "text/event-stream",
|
||||
"Cache-Control": "no-cache",
|
||||
"Connection": "keep-alive",
|
||||
"X-Accel-Buffering": "no",
|
||||
});
|
||||
res.flushHeaders();
|
||||
if (res.socket) res.socket.setNoDelay(true);
|
||||
sseResponse = res;
|
||||
for (const msg of sseBuffer) {
|
||||
try { res.write(msg); } catch {}
|
||||
}
|
||||
sseBuffer.length = 0;
|
||||
if (sseKeepalive) clearInterval(sseKeepalive);
|
||||
sseKeepalive = setInterval(() => {
|
||||
if (sseResponse) {
|
||||
try { sseResponse.write(":keepalive\n\n"); } catch {}
|
||||
}
|
||||
}, 15000);
|
||||
req.on("close", () => {
|
||||
if (sseResponse === res) sseResponse = null;
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
if (method === "POST" && url.pathname === "/heartbeat") {
|
||||
const body = await parseJSONBody(req).catch(() => null);
|
||||
if (!body) { sendJson(res, 400, { ok: false, error: "Invalid body" }); return; }
|
||||
if (!validateToken(body, res)) return;
|
||||
touchHeartbeat();
|
||||
sendJson(res, 200, { ok: true });
|
||||
return;
|
||||
}
|
||||
|
||||
if (method === "POST" && url.pathname === "/provider") {
|
||||
const body = await parseJSONBody(req).catch(() => null);
|
||||
if (!body) { sendJson(res, 400, { ok: false, error: "Invalid body" }); return; }
|
||||
if (!validateToken(body, res)) return;
|
||||
const { provider } = body as { provider?: string };
|
||||
if (typeof provider === "string" && provider.length > 0) {
|
||||
setImmediate(() => callbacks.onProviderChange(provider));
|
||||
}
|
||||
sendJson(res, 200, { ok: true });
|
||||
return;
|
||||
}
|
||||
|
||||
if (method === "POST" && url.pathname === "/search") {
|
||||
const body = await parseJSONBody(req).catch(() => null);
|
||||
if (!body) { sendJson(res, 400, { ok: false, error: "Invalid body" }); return; }
|
||||
if (!validateToken(body, res)) return;
|
||||
if (state === "COMPLETED") {
|
||||
sendJson(res, 409, { ok: false, error: "Session closed" });
|
||||
return;
|
||||
}
|
||||
const { query } = body as { query?: string };
|
||||
if (typeof query !== "string" || query.trim().length === 0) {
|
||||
sendJson(res, 400, { ok: false, error: "Invalid query" });
|
||||
return;
|
||||
}
|
||||
const qi = nextQueryIndex++;
|
||||
touchHeartbeat();
|
||||
try {
|
||||
const result = await callbacks.onAddSearch(query.trim(), qi);
|
||||
sendJson(res, 200, { ok: true, queryIndex: qi, answer: result.answer, results: result.results });
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : "Search failed";
|
||||
sendJson(res, 200, { ok: true, queryIndex: qi, error: message });
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (method === "POST" && url.pathname === "/submit") {
|
||||
const body = await parseJSONBody(req).catch(() => null);
|
||||
if (!body) { sendJson(res, 400, { ok: false, error: "Invalid body" }); return; }
|
||||
if (!validateToken(body, res)) return;
|
||||
const { selected } = body as { selected?: number[] };
|
||||
if (!Array.isArray(selected) || !selected.every(n => typeof n === "number")) {
|
||||
sendJson(res, 400, { ok: false, error: "Invalid selection" });
|
||||
return;
|
||||
}
|
||||
if (state !== "SEARCHING" && state !== "RESULT_SELECTION") {
|
||||
sendJson(res, 409, { ok: false, error: "Cannot submit in current state" });
|
||||
return;
|
||||
}
|
||||
if (!markCompleted()) {
|
||||
sendJson(res, 409, { ok: false, error: "Session closed" });
|
||||
return;
|
||||
}
|
||||
sendJson(res, 200, { ok: true });
|
||||
setImmediate(() => callbacks.onSubmit(selected));
|
||||
return;
|
||||
}
|
||||
|
||||
if (method === "POST" && url.pathname === "/cancel") {
|
||||
const body = await parseJSONBody(req).catch(() => null);
|
||||
if (!body) { sendJson(res, 400, { ok: false, error: "Invalid body" }); return; }
|
||||
if (!validateToken(body, res)) return;
|
||||
if (!markCompleted()) {
|
||||
sendJson(res, 200, { ok: true });
|
||||
return;
|
||||
}
|
||||
const { reason } = body as { reason?: string };
|
||||
sendJson(res, 200, { ok: true });
|
||||
const cancelReason = reason === "timeout" ? "timeout" : "user";
|
||||
setImmediate(() => callbacks.onCancel(cancelReason));
|
||||
return;
|
||||
}
|
||||
|
||||
res.writeHead(404, { "Content-Type": "text/plain" });
|
||||
res.end("Not found");
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : "Server error";
|
||||
sendJson(res, 500, { ok: false, error: message });
|
||||
}
|
||||
});
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const onError = (err: Error) => {
|
||||
reject(new Error(`Curator server failed to start: ${err.message}`));
|
||||
};
|
||||
|
||||
server.once("error", onError);
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
server.off("error", onError);
|
||||
const addr = server.address();
|
||||
if (!addr || typeof addr === "string") {
|
||||
reject(new Error("Curator server: invalid address"));
|
||||
return;
|
||||
}
|
||||
const url = `http://localhost:${addr.port}/?session=${sessionToken}`;
|
||||
|
||||
watchdog = setInterval(() => {
|
||||
if (completed || !browserConnected) return;
|
||||
if (Date.now() - lastHeartbeatAt <= STALE_THRESHOLD_MS) return;
|
||||
if (!markCompleted()) return;
|
||||
setImmediate(() => callbacks.onCancel("stale"));
|
||||
}, WATCHDOG_INTERVAL_MS);
|
||||
|
||||
resolve({
|
||||
server,
|
||||
url,
|
||||
close: () => {
|
||||
const wasOpen = markCompleted();
|
||||
try { server.close(); } catch {}
|
||||
if (wasOpen) {
|
||||
setImmediate(() => callbacks.onCancel("stale"));
|
||||
}
|
||||
},
|
||||
pushResult: (queryIndex, data) => {
|
||||
if (completed) return;
|
||||
sendSSE("result", { queryIndex, query: queries[queryIndex] ?? "", ...data });
|
||||
},
|
||||
pushError: (queryIndex, error) => {
|
||||
if (completed) return;
|
||||
sendSSE("search-error", { queryIndex, query: queries[queryIndex] ?? "", error });
|
||||
},
|
||||
searchesDone: () => {
|
||||
if (completed) return;
|
||||
sendSSE("done", {});
|
||||
state = "RESULT_SELECTION";
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
187
pi/files/agent/extensions/pi-web-access/ddg-search.ts
Normal file
187
pi/files/agent/extensions/pi-web-access/ddg-search.ts
Normal file
@@ -0,0 +1,187 @@
|
||||
import { activityMonitor } from "./activity.js";
|
||||
|
||||
export interface SearchResult {
|
||||
title: string;
|
||||
url: string;
|
||||
snippet: string;
|
||||
}
|
||||
|
||||
export interface SearchResponse {
|
||||
answer: string;
|
||||
results: SearchResult[];
|
||||
}
|
||||
|
||||
export interface SearchOptions {
|
||||
numResults?: number;
|
||||
recencyFilter?: "day" | "week" | "month" | "year";
|
||||
domainFilter?: string[];
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
const DDG_HTML = "https://duckduckgo.com/html/";
|
||||
const DDG_LITE = "https://lite.duckduckgo.com/lite/";
|
||||
|
||||
function applyDomainFilter(urls: SearchResult[], domains?: string[]): SearchResult[] {
|
||||
if (!domains || domains.length === 0) return urls;
|
||||
const includes = domains.filter((d) => !d.startsWith("-")).map((d) => d.toLowerCase());
|
||||
const excludes = domains.filter((d) => d.startsWith("-")).map((d) => d.slice(1).toLowerCase());
|
||||
return urls.filter((r) => {
|
||||
try {
|
||||
const host = new URL(r.url).hostname.toLowerCase();
|
||||
if (includes.length && !includes.some((d) => host === d || host.endsWith(`.${d}`))) {
|
||||
return false;
|
||||
}
|
||||
if (excludes.some((d) => host === d || host.endsWith(`.${d}`))) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function extractResultsFromHtml(html: string): SearchResult[] {
|
||||
const results: SearchResult[] = [];
|
||||
const resultRegex = /<a[^>]+class="result__a"[^>]+href="([^"]+)"[^>]*>([\s\S]*?)<\/a>[\s\S]*?(?:<a[^>]+class="result__snippet"[^>]*>([\s\S]*?)<\/a>|<div[^>]+class="result__snippet"[^>]*>([\s\S]*?)<\/div>)/g;
|
||||
for (const match of html.matchAll(resultRegex)) {
|
||||
const url = decodeUrl(match[1]);
|
||||
const title = stripTags(match[2]);
|
||||
const snippet = stripTags(match[3] || match[4] || "");
|
||||
if (!url || !title) continue;
|
||||
results.push({ title, url, snippet });
|
||||
}
|
||||
if (results.length > 0) return results;
|
||||
|
||||
const liteRegex = /<a[^>]+href="([^"]+)"[^>]*>([\s\S]*?)<\/a>\s*<br\s*\/?>\s*<span[^>]*class="link-text"[^>]*>([\s\S]*?)<\/span>\s*<br\s*\/?>\s*<span[^>]*class="result-snippet"[^>]*>([\s\S]*?)<\/span>/g;
|
||||
for (const match of html.matchAll(liteRegex)) {
|
||||
const url = decodeUrl(match[1]);
|
||||
const title = stripTags(match[2]);
|
||||
const snippet = stripTags(match[4] || "");
|
||||
if (!url || !title) continue;
|
||||
results.push({ title, url, snippet });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
function stripTags(text: string): string {
|
||||
return text.replace(/<[^>]+>/g, "").replace(/\s+/g, " ").trim();
|
||||
}
|
||||
|
||||
function decodeUrl(url: string): string {
|
||||
try {
|
||||
const decoded = new URL(url, "https://duckduckgo.com");
|
||||
const uddg = decoded.searchParams.get("uddg");
|
||||
if (uddg) return decodeURIComponent(uddg);
|
||||
return decoded.toString();
|
||||
} catch {
|
||||
return url;
|
||||
}
|
||||
}
|
||||
|
||||
function buildQuery(query: string, options: SearchOptions): string {
|
||||
let q = query;
|
||||
if (options.recencyFilter) {
|
||||
const recency: Record<string, string> = {
|
||||
day: "d",
|
||||
week: "w",
|
||||
month: "m",
|
||||
year: "y",
|
||||
};
|
||||
q += ` time:${recency[options.recencyFilter]}`;
|
||||
}
|
||||
return q;
|
||||
}
|
||||
|
||||
export async function searchWithDuckDuckGo(
|
||||
query: string,
|
||||
options: SearchOptions = {},
|
||||
): Promise<SearchResponse> {
|
||||
const activityId = activityMonitor.logStart({ type: "api", query });
|
||||
const q = buildQuery(query, options);
|
||||
const params = new URLSearchParams({ q });
|
||||
const url = `${DDG_HTML}?${params.toString()}`;
|
||||
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(url, {
|
||||
headers: {
|
||||
"user-agent": "Mozilla/5.0",
|
||||
"accept-language": "en-US,en;q=0.9",
|
||||
},
|
||||
signal: options.signal,
|
||||
});
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
activityMonitor.logError(activityId, message);
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
throw new Error(`DuckDuckGo error ${res.status}`);
|
||||
}
|
||||
|
||||
const html = await res.text();
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
|
||||
const results = extractResultsFromHtml(html);
|
||||
let filtered = applyDomainFilter(results, options.domainFilter);
|
||||
const limit = Math.min(options.numResults ?? 5, 20);
|
||||
filtered = filtered.slice(0, limit);
|
||||
|
||||
if (filtered.length === 0) {
|
||||
// fallback to lite
|
||||
return searchWithDuckDuckGoLite(query, options);
|
||||
}
|
||||
|
||||
const answer = filtered
|
||||
.map((r, idx) => `${idx + 1}. ${r.title}\n ${r.url}`)
|
||||
.join("\n\n");
|
||||
|
||||
return { answer, results: filtered };
|
||||
}
|
||||
|
||||
async function searchWithDuckDuckGoLite(
|
||||
query: string,
|
||||
options: SearchOptions,
|
||||
): Promise<SearchResponse> {
|
||||
const activityId = activityMonitor.logStart({ type: "api", query: `${query} (lite)` });
|
||||
const q = buildQuery(query, options);
|
||||
const params = new URLSearchParams({ q });
|
||||
const url = `${DDG_LITE}?${params.toString()}`;
|
||||
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(url, {
|
||||
headers: {
|
||||
"user-agent": "Mozilla/5.0",
|
||||
"accept-language": "en-US,en;q=0.9",
|
||||
},
|
||||
signal: options.signal,
|
||||
});
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
activityMonitor.logError(activityId, message);
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
throw new Error(`DuckDuckGo lite error ${res.status}`);
|
||||
}
|
||||
|
||||
const html = await res.text();
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
|
||||
const results = extractResultsFromHtml(html);
|
||||
let filtered = applyDomainFilter(results, options.domainFilter);
|
||||
const limit = Math.min(options.numResults ?? 5, 20);
|
||||
filtered = filtered.slice(0, limit);
|
||||
|
||||
const answer = filtered
|
||||
.map((r, idx) => `${idx + 1}. ${r.title}\n ${r.url}`)
|
||||
.join("\n\n");
|
||||
|
||||
return { answer, results: filtered };
|
||||
}
|
||||
560
pi/files/agent/extensions/pi-web-access/extract.ts
Normal file
560
pi/files/agent/extensions/pi-web-access/extract.ts
Normal file
@@ -0,0 +1,560 @@
|
||||
import { Readability } from "@mozilla/readability";
|
||||
import { parseHTML } from "linkedom";
|
||||
import TurndownService from "turndown";
|
||||
import pLimit from "p-limit";
|
||||
import { activityMonitor } from "./activity.js";
|
||||
import { extractRSCContent } from "./rsc-extract.js";
|
||||
import { extractPDFToMarkdown, isPDF } from "./pdf-extract.js";
|
||||
import { extractGitHub } from "./github-extract.js";
|
||||
import { isYouTubeURL, isYouTubeEnabled, extractYouTube, extractYouTubeFrame, extractYouTubeFrames, getYouTubeStreamInfo } from "./youtube-extract.js";
|
||||
import { extractWithUrlContext, extractWithGeminiWeb } from "./gemini-url-context.js";
|
||||
import { isVideoFile, extractVideo, extractVideoFrame, getLocalVideoDuration } from "./video-extract.js";
|
||||
import { formatSeconds } from "./utils.js";
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 30000;
|
||||
const CONCURRENT_LIMIT = 3;
|
||||
|
||||
const NON_RECOVERABLE_ERRORS = ["Unsupported content type", "Response too large"];
|
||||
const MIN_USEFUL_CONTENT = 500;
|
||||
|
||||
const turndown = new TurndownService({
|
||||
headingStyle: "atx",
|
||||
codeBlockStyle: "fenced",
|
||||
});
|
||||
|
||||
const fetchLimit = pLimit(CONCURRENT_LIMIT);
|
||||
|
||||
export interface VideoFrame {
|
||||
data: string;
|
||||
mimeType: string;
|
||||
timestamp: string;
|
||||
}
|
||||
|
||||
export type FrameData = { data: string; mimeType: string };
|
||||
export type FrameResult = FrameData | { error: string };
|
||||
|
||||
export interface ExtractedContent {
|
||||
url: string;
|
||||
title: string;
|
||||
content: string;
|
||||
error: string | null;
|
||||
thumbnail?: { data: string; mimeType: string };
|
||||
frames?: VideoFrame[];
|
||||
duration?: number;
|
||||
}
|
||||
|
||||
export interface ExtractOptions {
|
||||
timeoutMs?: number;
|
||||
forceClone?: boolean;
|
||||
prompt?: string;
|
||||
timestamp?: string;
|
||||
frames?: number;
|
||||
model?: string;
|
||||
}
|
||||
|
||||
const JINA_READER_BASE = "https://r.jina.ai/";
|
||||
const JINA_TIMEOUT_MS = 30000;
|
||||
|
||||
async function extractWithJinaReader(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const jinaUrl = JINA_READER_BASE + url;
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "api", query: `jina: ${url}` });
|
||||
|
||||
try {
|
||||
const res = await fetch(jinaUrl, {
|
||||
headers: {
|
||||
"Accept": "text/markdown",
|
||||
"X-No-Cache": "true",
|
||||
},
|
||||
signal: AbortSignal.any([
|
||||
AbortSignal.timeout(JINA_TIMEOUT_MS),
|
||||
...(signal ? [signal] : []),
|
||||
]),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
return null;
|
||||
}
|
||||
|
||||
const content = await res.text();
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
|
||||
const contentStart = content.indexOf("Markdown Content:");
|
||||
if (contentStart < 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const markdownPart = content.slice(contentStart + 17).trim(); // 17 = "Markdown Content:".length
|
||||
|
||||
// Check for failed JS rendering or minimal content
|
||||
if (markdownPart.length < 100 ||
|
||||
markdownPart.startsWith("Loading...") ||
|
||||
markdownPart.startsWith("Please enable JavaScript")) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const title = extractHeadingTitle(markdownPart) ?? (new URL(url).pathname.split("/").pop() || url);
|
||||
return { url, title, content: markdownPart, error: null };
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function parseTimestamp(ts: string): number | null {
|
||||
const num = Number(ts);
|
||||
if (!isNaN(num) && num >= 0) return Math.floor(num);
|
||||
const parts = ts.split(":").map(Number);
|
||||
if (parts.some(p => isNaN(p) || p < 0)) return null;
|
||||
if (parts.length === 3) return Math.floor(parts[0] * 3600 + parts[1] * 60 + parts[2]);
|
||||
if (parts.length === 2) return Math.floor(parts[0] * 60 + parts[1]);
|
||||
return null;
|
||||
}
|
||||
|
||||
type TimestampSpec = { type: "single"; seconds: number } | { type: "range"; start: number; end: number };
|
||||
|
||||
function parseTimestampSpec(ts: string): TimestampSpec | null {
|
||||
const dashIdx = ts.indexOf("-", 1);
|
||||
if (dashIdx > 0) {
|
||||
const start = parseTimestamp(ts.slice(0, dashIdx));
|
||||
const end = parseTimestamp(ts.slice(dashIdx + 1));
|
||||
if (start !== null && end !== null && end > start) return { type: "range", start, end };
|
||||
}
|
||||
const seconds = parseTimestamp(ts);
|
||||
return seconds !== null ? { type: "single", seconds } : null;
|
||||
}
|
||||
|
||||
const DEFAULT_RANGE_FRAMES = 6;
|
||||
const MIN_FRAME_INTERVAL = 5;
|
||||
|
||||
function computeRangeTimestamps(start: number, end: number, maxFrames: number = DEFAULT_RANGE_FRAMES): number[] {
|
||||
if (maxFrames <= 1) return [start];
|
||||
const duration = end - start;
|
||||
const idealInterval = duration / (maxFrames - 1);
|
||||
if (idealInterval < MIN_FRAME_INTERVAL) {
|
||||
const timestamps: number[] = [];
|
||||
for (let t = start; t <= end && timestamps.length < maxFrames; t += MIN_FRAME_INTERVAL) {
|
||||
timestamps.push(t);
|
||||
}
|
||||
return timestamps;
|
||||
}
|
||||
return Array.from({ length: maxFrames }, (_, i) => Math.round(start + i * idealInterval));
|
||||
}
|
||||
|
||||
function buildFrameResult(
|
||||
url: string, label: string, requestedCount: number,
|
||||
frames: VideoFrame[], error: string | null, duration?: number,
|
||||
): ExtractedContent {
|
||||
if (frames.length === 0) {
|
||||
const msg = error ?? "Frame extraction failed";
|
||||
return { url, title: `Frames ${label} (0/${requestedCount})`, content: msg, error: msg };
|
||||
}
|
||||
return {
|
||||
url,
|
||||
title: `Frames ${label} (${frames.length}/${requestedCount})`,
|
||||
content: `${frames.length} frames extracted from ${label}`,
|
||||
error: null,
|
||||
frames,
|
||||
duration,
|
||||
};
|
||||
}
|
||||
|
||||
async function extractLocalFrames(
|
||||
filePath: string, timestamps: number[],
|
||||
): Promise<{ frames: VideoFrame[]; error: string | null }> {
|
||||
const results = await Promise.all(timestamps.map(async (t) => {
|
||||
const frame = await extractVideoFrame(filePath, t);
|
||||
if ("error" in frame) return { error: frame.error };
|
||||
return { ...frame, timestamp: formatSeconds(t) };
|
||||
}));
|
||||
const frames = results.filter((f): f is VideoFrame => "data" in f);
|
||||
const firstError = results.find((f): f is { error: string } => "error" in f);
|
||||
return { frames, error: frames.length === 0 && firstError ? firstError.error : null };
|
||||
}
|
||||
|
||||
export async function extractContent(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
options?: ExtractOptions,
|
||||
): Promise<ExtractedContent> {
|
||||
if (signal?.aborted) {
|
||||
return { url, title: "", content: "", error: "Aborted" };
|
||||
}
|
||||
|
||||
if (options?.frames && !options.timestamp) {
|
||||
const frameCount = options.frames;
|
||||
const ytInfo = isYouTubeURL(url);
|
||||
if (ytInfo.isYouTube && ytInfo.videoId) {
|
||||
const streamInfo = await getYouTubeStreamInfo(ytInfo.videoId);
|
||||
if ("error" in streamInfo) {
|
||||
return { url, title: "Frames", content: streamInfo.error, error: streamInfo.error };
|
||||
}
|
||||
if (streamInfo.duration === null) {
|
||||
const error = "Cannot determine video duration. Use a timestamp range instead.";
|
||||
return { url, title: "Frames", content: error, error };
|
||||
}
|
||||
const dur = Math.floor(streamInfo.duration);
|
||||
const timestamps = computeRangeTimestamps(0, dur, frameCount);
|
||||
const result = await extractYouTubeFrames(ytInfo.videoId, timestamps, streamInfo);
|
||||
const label = `${formatSeconds(0)}-${formatSeconds(dur)}`;
|
||||
return buildFrameResult(url, label, timestamps.length, result.frames, result.error, streamInfo.duration);
|
||||
}
|
||||
|
||||
const videoInfo = isVideoFile(url);
|
||||
if (videoInfo) {
|
||||
const durationResult = await getLocalVideoDuration(videoInfo.absolutePath);
|
||||
if (typeof durationResult !== "number") {
|
||||
return { url, title: "Frames", content: durationResult.error, error: durationResult.error };
|
||||
}
|
||||
const dur = Math.floor(durationResult);
|
||||
const timestamps = computeRangeTimestamps(0, dur, frameCount);
|
||||
const result = await extractLocalFrames(videoInfo.absolutePath, timestamps);
|
||||
const label = `${formatSeconds(0)}-${formatSeconds(dur)}`;
|
||||
return buildFrameResult(url, label, timestamps.length, result.frames, result.error, durationResult);
|
||||
}
|
||||
|
||||
return { url, title: "", content: "", error: "Frame extraction only works with YouTube and local video files" };
|
||||
}
|
||||
|
||||
if (options?.timestamp) {
|
||||
const spec = parseTimestampSpec(options.timestamp);
|
||||
if (spec) {
|
||||
const frameCount = options.frames;
|
||||
const ytInfo = isYouTubeURL(url);
|
||||
if (ytInfo.isYouTube && ytInfo.videoId) {
|
||||
const streamInfo = await getYouTubeStreamInfo(ytInfo.videoId);
|
||||
if ("error" in streamInfo) {
|
||||
if (spec.type === "range") {
|
||||
const label = `${formatSeconds(spec.start)}-${formatSeconds(spec.end)}`;
|
||||
return { url, title: `Frames ${label}`, content: streamInfo.error, error: streamInfo.error };
|
||||
}
|
||||
if (frameCount) {
|
||||
const end = spec.seconds + (frameCount - 1) * MIN_FRAME_INTERVAL;
|
||||
const label = `${formatSeconds(spec.seconds)}-${formatSeconds(end)}`;
|
||||
return { url, title: `Frames ${label}`, content: streamInfo.error, error: streamInfo.error };
|
||||
}
|
||||
return { url, title: `Frame at ${options.timestamp}`, content: streamInfo.error, error: streamInfo.error };
|
||||
}
|
||||
|
||||
if (spec.type === "range") {
|
||||
const label = `${formatSeconds(spec.start)}-${formatSeconds(spec.end)}`;
|
||||
if (streamInfo.duration !== null && spec.end > streamInfo.duration) {
|
||||
const error = `Timestamp ${formatSeconds(spec.end)} exceeds video duration (${formatSeconds(Math.floor(streamInfo.duration))})`;
|
||||
return { url, title: `Frames ${label}`, content: error, error };
|
||||
}
|
||||
const timestamps = frameCount
|
||||
? computeRangeTimestamps(spec.start, spec.end, frameCount)
|
||||
: computeRangeTimestamps(spec.start, spec.end);
|
||||
const result = await extractYouTubeFrames(ytInfo.videoId, timestamps, streamInfo);
|
||||
return buildFrameResult(url, label, timestamps.length, result.frames, result.error, result.duration ?? undefined);
|
||||
}
|
||||
|
||||
if (frameCount) {
|
||||
const end = spec.seconds + (frameCount - 1) * MIN_FRAME_INTERVAL;
|
||||
const label = `${formatSeconds(spec.seconds)}-${formatSeconds(end)}`;
|
||||
if (streamInfo.duration !== null && end > streamInfo.duration) {
|
||||
const error = `Timestamp ${formatSeconds(end)} exceeds video duration (${formatSeconds(Math.floor(streamInfo.duration))})`;
|
||||
return { url, title: `Frames ${label}`, content: error, error };
|
||||
}
|
||||
const timestamps = computeRangeTimestamps(spec.seconds, end, frameCount);
|
||||
const result = await extractYouTubeFrames(ytInfo.videoId, timestamps, streamInfo);
|
||||
return buildFrameResult(url, label, timestamps.length, result.frames, result.error, result.duration ?? undefined);
|
||||
}
|
||||
|
||||
if (streamInfo.duration !== null && spec.seconds > streamInfo.duration) {
|
||||
const error = `Timestamp ${formatSeconds(spec.seconds)} exceeds video duration (${formatSeconds(Math.floor(streamInfo.duration))})`;
|
||||
return { url, title: `Frame at ${options.timestamp}`, content: error, error };
|
||||
}
|
||||
const frame = await extractYouTubeFrame(ytInfo.videoId, spec.seconds, streamInfo);
|
||||
if ("error" in frame) {
|
||||
return { url, title: `Frame at ${options.timestamp}`, content: frame.error, error: frame.error };
|
||||
}
|
||||
return { url, title: `Frame at ${options.timestamp}`, content: `Video frame at ${options.timestamp}`, error: null, thumbnail: frame };
|
||||
}
|
||||
|
||||
const videoInfo = isVideoFile(url);
|
||||
if (videoInfo) {
|
||||
if (spec.type === "range") {
|
||||
const timestamps = frameCount
|
||||
? computeRangeTimestamps(spec.start, spec.end, frameCount)
|
||||
: computeRangeTimestamps(spec.start, spec.end);
|
||||
const result = await extractLocalFrames(videoInfo.absolutePath, timestamps);
|
||||
const label = `${formatSeconds(spec.start)}-${formatSeconds(spec.end)}`;
|
||||
return buildFrameResult(url, label, timestamps.length, result.frames, result.error);
|
||||
}
|
||||
|
||||
if (frameCount) {
|
||||
const end = spec.seconds + (frameCount - 1) * MIN_FRAME_INTERVAL;
|
||||
const timestamps = computeRangeTimestamps(spec.seconds, end, frameCount);
|
||||
const result = await extractLocalFrames(videoInfo.absolutePath, timestamps);
|
||||
const label = `${formatSeconds(spec.seconds)}-${formatSeconds(end)}`;
|
||||
return buildFrameResult(url, label, timestamps.length, result.frames, result.error);
|
||||
}
|
||||
|
||||
const frame = await extractVideoFrame(videoInfo.absolutePath, spec.seconds);
|
||||
if ("error" in frame) {
|
||||
return { url, title: `Frame at ${options.timestamp}`, content: frame.error, error: frame.error };
|
||||
}
|
||||
return { url, title: `Frame at ${options.timestamp}`, content: `Video frame at ${options.timestamp}`, error: null, thumbnail: frame };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const videoInfo = isVideoFile(url);
|
||||
if (videoInfo) {
|
||||
const result = await extractVideo(videoInfo, signal, options);
|
||||
return result ?? { url, title: "", content: "", error: "Video analysis requires Gemini access. Either:\n 1. Sign into gemini.google.com in Chrome (free, uses cookies)\n 2. Set GEMINI_API_KEY in ~/.pi/web-search.json" };
|
||||
}
|
||||
|
||||
try {
|
||||
new URL(url);
|
||||
} catch {
|
||||
return { url, title: "", content: "", error: "Invalid URL" };
|
||||
}
|
||||
|
||||
try {
|
||||
const ghResult = await extractGitHub(url, signal, options?.forceClone);
|
||||
if (ghResult) return ghResult;
|
||||
} catch {}
|
||||
|
||||
const ytInfo = isYouTubeURL(url);
|
||||
if (ytInfo.isYouTube && isYouTubeEnabled()) {
|
||||
try {
|
||||
const ytResult = await extractYouTube(url, signal, options?.prompt, options?.model);
|
||||
if (ytResult) return ytResult;
|
||||
} catch {}
|
||||
return {
|
||||
url,
|
||||
title: "",
|
||||
content: "",
|
||||
error: "Could not extract YouTube video content. Sign into Google in Chrome for automatic access, or set GEMINI_API_KEY.",
|
||||
};
|
||||
}
|
||||
|
||||
const httpResult = await extractViaHttp(url, signal, options);
|
||||
|
||||
if (!httpResult.error || signal?.aborted) return httpResult;
|
||||
if (NON_RECOVERABLE_ERRORS.some(prefix => httpResult.error!.startsWith(prefix))) return httpResult;
|
||||
|
||||
const jinaResult = await extractWithJinaReader(url, signal);
|
||||
if (jinaResult) return jinaResult;
|
||||
|
||||
const geminiResult = await extractWithUrlContext(url, signal)
|
||||
?? await extractWithGeminiWeb(url, signal);
|
||||
|
||||
if (geminiResult) return geminiResult;
|
||||
|
||||
const guidance = [
|
||||
httpResult.error,
|
||||
"",
|
||||
"Fallback options:",
|
||||
" \u2022 Set GEMINI_API_KEY in ~/.pi/web-search.json",
|
||||
" \u2022 Sign into gemini.google.com in Chrome",
|
||||
" \u2022 Use web_search to find content about this topic",
|
||||
].join("\n");
|
||||
return { ...httpResult, error: guidance };
|
||||
}
|
||||
|
||||
function isLikelyJSRendered(html: string): boolean {
|
||||
// Extract body content
|
||||
const bodyMatch = html.match(/<body[^>]*>([\s\S]*?)<\/body>/i);
|
||||
if (!bodyMatch) return false;
|
||||
|
||||
const bodyHtml = bodyMatch[1];
|
||||
|
||||
// Strip tags to get text content
|
||||
const textContent = bodyHtml
|
||||
.replace(/<script[\s\S]*?<\/script>/gi, "")
|
||||
.replace(/<style[\s\S]*?<\/style>/gi, "")
|
||||
.replace(/<[^>]+>/g, "")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
|
||||
// Count scripts
|
||||
const scriptCount = (html.match(/<script/gi) || []).length;
|
||||
|
||||
// Heuristic: little text content but many scripts suggests JS rendering
|
||||
return textContent.length < 500 && scriptCount > 3;
|
||||
}
|
||||
|
||||
async function extractViaHttp(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
options?: ExtractOptions,
|
||||
): Promise<ExtractedContent> {
|
||||
const timeoutMs = options?.timeoutMs ?? DEFAULT_TIMEOUT_MS;
|
||||
const activityId = activityMonitor.logStart({ type: "fetch", url });
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
const onAbort = () => controller.abort();
|
||||
signal?.addEventListener("abort", onAbort);
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
signal: controller.signal,
|
||||
headers: {
|
||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
|
||||
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
|
||||
"Accept-Language": "en-US,en;q=0.9",
|
||||
"Cache-Control": "no-cache",
|
||||
"Sec-Fetch-Dest": "document",
|
||||
"Sec-Fetch-Mode": "navigate",
|
||||
"Sec-Fetch-Site": "none",
|
||||
"Sec-Fetch-User": "?1",
|
||||
"Upgrade-Insecure-Requests": "1",
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
return {
|
||||
url,
|
||||
title: "",
|
||||
content: "",
|
||||
error: `HTTP ${response.status}: ${response.statusText}`,
|
||||
};
|
||||
}
|
||||
|
||||
const contentLengthHeader = response.headers.get("content-length");
|
||||
const contentType = response.headers.get("content-type") || "";
|
||||
const isPDFContent = isPDF(url, contentType);
|
||||
const maxResponseSize = isPDFContent ? 20 * 1024 * 1024 : 5 * 1024 * 1024;
|
||||
if (contentLengthHeader) {
|
||||
const contentLength = parseInt(contentLengthHeader, 10);
|
||||
if (contentLength > maxResponseSize) {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
return {
|
||||
url,
|
||||
title: "",
|
||||
content: "",
|
||||
error: `Response too large (${Math.round(contentLength / 1024 / 1024)}MB)`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
if (isPDFContent) {
|
||||
try {
|
||||
const buffer = await response.arrayBuffer();
|
||||
const result = await extractPDFToMarkdown(buffer, url);
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
return {
|
||||
url,
|
||||
title: result.title,
|
||||
content: `PDF extracted and saved to: ${result.outputPath}\n\nPages: ${result.pages}\nCharacters: ${result.chars}`,
|
||||
error: null,
|
||||
};
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
activityMonitor.logError(activityId, message);
|
||||
return { url, title: "", content: "", error: `PDF extraction failed: ${message}` };
|
||||
}
|
||||
}
|
||||
|
||||
if (contentType.includes("application/octet-stream") ||
|
||||
contentType.includes("image/") ||
|
||||
contentType.includes("audio/") ||
|
||||
contentType.includes("video/") ||
|
||||
contentType.includes("application/zip")) {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
return {
|
||||
url,
|
||||
title: "",
|
||||
content: "",
|
||||
error: `Unsupported content type: ${contentType.split(";")[0]}`,
|
||||
};
|
||||
}
|
||||
|
||||
const text = await response.text();
|
||||
const isHTML = contentType.includes("text/html") || contentType.includes("application/xhtml+xml");
|
||||
|
||||
if (!isHTML) {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
const title = extractTextTitle(text, url);
|
||||
return { url, title, content: text, error: null };
|
||||
}
|
||||
|
||||
const { document } = parseHTML(text);
|
||||
const reader = new Readability(document as unknown as Document);
|
||||
const article = reader.parse();
|
||||
|
||||
if (!article) {
|
||||
const rscResult = extractRSCContent(text);
|
||||
if (rscResult) {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
return { url, title: rscResult.title, content: rscResult.content, error: null };
|
||||
}
|
||||
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
|
||||
// Provide more specific error message
|
||||
const jsRendered = isLikelyJSRendered(text);
|
||||
const errorMsg = jsRendered
|
||||
? "Page appears to be JavaScript-rendered (content loads dynamically)"
|
||||
: "Could not extract readable content from HTML structure";
|
||||
|
||||
return {
|
||||
url,
|
||||
title: "",
|
||||
content: "",
|
||||
error: errorMsg,
|
||||
};
|
||||
}
|
||||
|
||||
const markdown = turndown.turndown(article.content);
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
|
||||
if (markdown.length < MIN_USEFUL_CONTENT) {
|
||||
return {
|
||||
url,
|
||||
title: article.title || "",
|
||||
content: markdown,
|
||||
error: isLikelyJSRendered(text)
|
||||
? "Page appears to be JavaScript-rendered (content loads dynamically)"
|
||||
: "Extracted content appears incomplete",
|
||||
};
|
||||
}
|
||||
|
||||
return { url, title: article.title || "", content: markdown, error: null };
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
return { url, title: "", content: "", error: message };
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
}
|
||||
}
|
||||
|
||||
export function extractHeadingTitle(text: string): string | null {
|
||||
const match = text.match(/^#{1,2}\s+(.+)/m);
|
||||
if (!match) return null;
|
||||
const cleaned = match[1].replace(/\*+/g, "").trim();
|
||||
return cleaned || null;
|
||||
}
|
||||
|
||||
function extractTextTitle(text: string, url: string): string {
|
||||
return extractHeadingTitle(text) ?? (new URL(url).pathname.split("/").pop() || url);
|
||||
}
|
||||
|
||||
export async function fetchAllContent(
|
||||
urls: string[],
|
||||
signal?: AbortSignal,
|
||||
options?: ExtractOptions,
|
||||
): Promise<ExtractedContent[]> {
|
||||
return Promise.all(urls.map((url) => fetchLimit(() => extractContent(url, signal, options))));
|
||||
}
|
||||
103
pi/files/agent/extensions/pi-web-access/gemini-api.ts
Normal file
103
pi/files/agent/extensions/pi-web-access/gemini-api.ts
Normal file
@@ -0,0 +1,103 @@
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
export const API_BASE = "https://generativelanguage.googleapis.com/v1beta";
|
||||
const CONFIG_PATH = join(homedir(), ".pi", "web-search.json");
|
||||
export const DEFAULT_MODEL = "gemini-3-flash-preview";
|
||||
|
||||
interface GeminiApiConfig {
|
||||
geminiApiKey?: string;
|
||||
}
|
||||
|
||||
let cachedConfig: GeminiApiConfig | null = null;
|
||||
|
||||
function loadConfig(): GeminiApiConfig {
|
||||
if (cachedConfig) return cachedConfig;
|
||||
if (existsSync(CONFIG_PATH)) {
|
||||
try {
|
||||
cachedConfig = JSON.parse(readFileSync(CONFIG_PATH, "utf-8")) as GeminiApiConfig;
|
||||
return cachedConfig;
|
||||
} catch {}
|
||||
}
|
||||
cachedConfig = {};
|
||||
return cachedConfig;
|
||||
}
|
||||
|
||||
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
||||
const timeout = AbortSignal.timeout(timeoutMs);
|
||||
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||
}
|
||||
|
||||
export function getApiKey(): string | null {
|
||||
const envKey = process.env.GEMINI_API_KEY;
|
||||
if (envKey) return envKey;
|
||||
return loadConfig().geminiApiKey ?? null;
|
||||
}
|
||||
|
||||
export function isGeminiApiAvailable(): boolean {
|
||||
return getApiKey() !== null;
|
||||
}
|
||||
|
||||
export interface GeminiApiOptions {
|
||||
model?: string;
|
||||
mimeType?: string;
|
||||
signal?: AbortSignal;
|
||||
timeoutMs?: number;
|
||||
}
|
||||
|
||||
export async function queryGeminiApiWithVideo(
|
||||
prompt: string,
|
||||
videoUri: string,
|
||||
options: GeminiApiOptions = {},
|
||||
): Promise<string> {
|
||||
const apiKey = getApiKey();
|
||||
if (!apiKey) throw new Error("GEMINI_API_KEY not configured");
|
||||
|
||||
const model = options.model ?? DEFAULT_MODEL;
|
||||
const signal = withTimeout(options.signal, options.timeoutMs ?? 120000);
|
||||
const url = `${API_BASE}/models/${model}:generateContent?key=${apiKey}`;
|
||||
|
||||
const fileData: Record<string, string> = { fileUri: videoUri };
|
||||
if (options.mimeType) fileData.mimeType = options.mimeType;
|
||||
|
||||
const body = {
|
||||
contents: [
|
||||
{
|
||||
parts: [
|
||||
{ fileData },
|
||||
{ text: prompt },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
signal,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errorText = await res.text();
|
||||
throw new Error(`Gemini API error ${res.status}: ${errorText.slice(0, 300)}`);
|
||||
}
|
||||
|
||||
const data = (await res.json()) as GenerateContentResponse;
|
||||
const text = data.candidates?.[0]?.content?.parts
|
||||
?.map((p) => p.text)
|
||||
.filter(Boolean)
|
||||
.join("\n");
|
||||
|
||||
if (!text) throw new Error("Gemini API returned empty response");
|
||||
return text;
|
||||
}
|
||||
|
||||
interface GenerateContentResponse {
|
||||
candidates?: Array<{
|
||||
content?: {
|
||||
parts?: Array<{ text?: string }>;
|
||||
};
|
||||
}>;
|
||||
}
|
||||
232
pi/files/agent/extensions/pi-web-access/gemini-search.ts
Normal file
232
pi/files/agent/extensions/pi-web-access/gemini-search.ts
Normal file
@@ -0,0 +1,232 @@
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { activityMonitor } from "./activity.js";
|
||||
import { getApiKey, API_BASE, DEFAULT_MODEL } from "./gemini-api.js";
|
||||
import { isGeminiWebAvailable, queryWithCookies } from "./gemini-web.js";
|
||||
import { isPerplexityAvailable, searchWithPerplexity, type SearchResult, type SearchResponse, type SearchOptions } from "./perplexity.js";
|
||||
import { searchWithDuckDuckGo } from "./ddg-search.js";
|
||||
|
||||
export type SearchProvider = "auto" | "perplexity" | "gemini" | "duckduckgo";
|
||||
|
||||
const CONFIG_PATH = join(homedir(), ".pi", "web-search.json");
|
||||
|
||||
let cachedSearchConfig: { searchProvider: SearchProvider } | null = null;
|
||||
|
||||
function getSearchConfig(): { searchProvider: SearchProvider } {
|
||||
if (cachedSearchConfig) return cachedSearchConfig;
|
||||
try {
|
||||
if (existsSync(CONFIG_PATH)) {
|
||||
const raw = JSON.parse(readFileSync(CONFIG_PATH, "utf-8"));
|
||||
cachedSearchConfig = { searchProvider: raw.searchProvider ?? "auto" };
|
||||
return cachedSearchConfig;
|
||||
}
|
||||
} catch {}
|
||||
cachedSearchConfig = { searchProvider: "auto" };
|
||||
return cachedSearchConfig;
|
||||
}
|
||||
|
||||
export interface FullSearchOptions extends SearchOptions {
|
||||
provider?: SearchProvider;
|
||||
}
|
||||
|
||||
export async function search(query: string, options: FullSearchOptions = {}): Promise<SearchResponse> {
|
||||
const config = getSearchConfig();
|
||||
const provider = options.provider ?? config.searchProvider;
|
||||
|
||||
if (provider === "perplexity") {
|
||||
return searchWithPerplexity(query, options);
|
||||
}
|
||||
|
||||
if (provider === "duckduckgo") {
|
||||
return searchWithDuckDuckGo(query, options);
|
||||
}
|
||||
|
||||
if (provider === "gemini") {
|
||||
const result = await searchWithGeminiApi(query, options)
|
||||
?? await searchWithGeminiWeb(query, options);
|
||||
if (result) return result;
|
||||
return searchWithDuckDuckGo(query, options);
|
||||
}
|
||||
|
||||
if (isPerplexityAvailable()) {
|
||||
return searchWithPerplexity(query, options);
|
||||
}
|
||||
|
||||
const geminiResult = await searchWithGeminiApi(query, options)
|
||||
?? await searchWithGeminiWeb(query, options);
|
||||
if (geminiResult) return geminiResult;
|
||||
|
||||
return searchWithDuckDuckGo(query, options);
|
||||
}
|
||||
|
||||
async function searchWithGeminiApi(query: string, options: SearchOptions = {}): Promise<SearchResponse | null> {
|
||||
const apiKey = getApiKey();
|
||||
if (!apiKey) return null;
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "api", query });
|
||||
|
||||
try {
|
||||
const model = DEFAULT_MODEL;
|
||||
const body = {
|
||||
contents: [{ parts: [{ text: query }] }],
|
||||
tools: [{ google_search: {} }],
|
||||
};
|
||||
|
||||
const res = await fetch(`${API_BASE}/models/${model}:generateContent?key=${apiKey}`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
signal: AbortSignal.any([
|
||||
AbortSignal.timeout(60000),
|
||||
...(options.signal ? [options.signal] : []),
|
||||
]),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const errorText = await res.text();
|
||||
throw new Error(`Gemini API error ${res.status}: ${errorText.slice(0, 300)}`);
|
||||
}
|
||||
|
||||
const data = await res.json() as GeminiSearchResponse;
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
|
||||
const answer = data.candidates?.[0]?.content?.parts
|
||||
?.map(p => p.text).filter(Boolean).join("\n") ?? "";
|
||||
|
||||
const metadata = data.candidates?.[0]?.groundingMetadata;
|
||||
const results = await resolveGroundingChunks(metadata?.groundingChunks, options.signal);
|
||||
|
||||
if (!answer && results.length === 0) return null;
|
||||
return { answer, results };
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function searchWithGeminiWeb(query: string, options: SearchOptions = {}): Promise<SearchResponse | null> {
|
||||
const cookies = await isGeminiWebAvailable();
|
||||
if (!cookies) return null;
|
||||
|
||||
const prompt = buildSearchPrompt(query, options);
|
||||
const activityId = activityMonitor.logStart({ type: "api", query });
|
||||
|
||||
try {
|
||||
const text = await queryWithCookies(prompt, cookies, {
|
||||
model: "gemini-3-flash-preview",
|
||||
signal: options.signal,
|
||||
timeoutMs: 60000,
|
||||
});
|
||||
|
||||
activityMonitor.logComplete(activityId, 200);
|
||||
|
||||
const results = extractSourceUrls(text);
|
||||
return { answer: text, results };
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function buildSearchPrompt(query: string, options: SearchOptions): string {
|
||||
let prompt = `Search the web and answer the following question. Include source URLs for your claims.\nFormat your response as:\n1. A direct answer to the question\n2. Cited sources as markdown links\n\nQuestion: ${query}`;
|
||||
|
||||
if (options.recencyFilter) {
|
||||
const labels: Record<string, string> = {
|
||||
day: "past 24 hours",
|
||||
week: "past week",
|
||||
month: "past month",
|
||||
year: "past year",
|
||||
};
|
||||
prompt += `\n\nOnly include results from the ${labels[options.recencyFilter]}.`;
|
||||
}
|
||||
|
||||
if (options.domainFilter?.length) {
|
||||
const includes = options.domainFilter.filter(d => !d.startsWith("-"));
|
||||
const excludes = options.domainFilter.filter(d => d.startsWith("-")).map(d => d.slice(1));
|
||||
if (includes.length) prompt += `\n\nOnly cite sources from: ${includes.join(", ")}`;
|
||||
if (excludes.length) prompt += `\n\nDo not cite sources from: ${excludes.join(", ")}`;
|
||||
}
|
||||
|
||||
return prompt;
|
||||
}
|
||||
|
||||
function extractSourceUrls(markdown: string): SearchResult[] {
|
||||
const results: SearchResult[] = [];
|
||||
const seen = new Set<string>();
|
||||
const linkRegex = /\[([^\]]+)\]\((https?:\/\/[^)]+)\)/g;
|
||||
for (const match of markdown.matchAll(linkRegex)) {
|
||||
const url = match[2];
|
||||
if (seen.has(url)) continue;
|
||||
seen.add(url);
|
||||
results.push({ title: match[1], url, snippet: "" });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
async function resolveGroundingChunks(
|
||||
chunks: GroundingChunk[] | undefined,
|
||||
signal?: AbortSignal,
|
||||
): Promise<SearchResult[]> {
|
||||
if (!chunks?.length) return [];
|
||||
|
||||
const results: SearchResult[] = [];
|
||||
for (const chunk of chunks) {
|
||||
if (!chunk.web) continue;
|
||||
const title = chunk.web.title || "";
|
||||
let url = chunk.web.uri || "";
|
||||
|
||||
if (url.includes("vertexaisearch.cloud.google.com/grounding-api-redirect")) {
|
||||
const resolved = await resolveRedirect(url, signal);
|
||||
if (resolved) url = resolved;
|
||||
}
|
||||
|
||||
if (url) results.push({ title, url, snippet: "" });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
async function resolveRedirect(proxyUrl: string, signal?: AbortSignal): Promise<string | null> {
|
||||
try {
|
||||
const res = await fetch(proxyUrl, {
|
||||
method: "HEAD",
|
||||
redirect: "manual",
|
||||
signal: AbortSignal.any([
|
||||
AbortSignal.timeout(5000),
|
||||
...(signal ? [signal] : []),
|
||||
]),
|
||||
});
|
||||
return res.headers.get("location") || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
interface GeminiSearchResponse {
|
||||
candidates?: Array<{
|
||||
content?: { parts?: Array<{ text?: string }> };
|
||||
groundingMetadata?: {
|
||||
webSearchQueries?: string[];
|
||||
groundingChunks?: GroundingChunk[];
|
||||
groundingSupports?: Array<{
|
||||
segment?: { startIndex?: number; endIndex?: number; text?: string };
|
||||
groundingChunkIndices?: number[];
|
||||
}>;
|
||||
};
|
||||
}>;
|
||||
}
|
||||
|
||||
interface GroundingChunk {
|
||||
web?: { uri?: string; title?: string };
|
||||
}
|
||||
119
pi/files/agent/extensions/pi-web-access/gemini-url-context.ts
Normal file
119
pi/files/agent/extensions/pi-web-access/gemini-url-context.ts
Normal file
@@ -0,0 +1,119 @@
|
||||
import { activityMonitor } from "./activity.js";
|
||||
import { getApiKey, API_BASE, DEFAULT_MODEL } from "./gemini-api.js";
|
||||
import { isGeminiWebAvailable, queryWithCookies } from "./gemini-web.js";
|
||||
import { extractHeadingTitle, type ExtractedContent } from "./extract.js";
|
||||
|
||||
const EXTRACTION_PROMPT = `Extract the complete readable content from this URL as clean markdown.
|
||||
Include the page title, all text content, code blocks, and tables.
|
||||
Do not summarize — extract the full content.
|
||||
|
||||
URL: `;
|
||||
|
||||
export async function extractWithUrlContext(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const apiKey = getApiKey();
|
||||
if (!apiKey) return null;
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "api", query: `url_context: ${url}` });
|
||||
|
||||
try {
|
||||
const model = DEFAULT_MODEL;
|
||||
const body = {
|
||||
contents: [{ parts: [{ text: EXTRACTION_PROMPT + url }] }],
|
||||
tools: [{ url_context: {} }],
|
||||
};
|
||||
|
||||
const res = await fetch(`${API_BASE}/models/${model}:generateContent?key=${apiKey}`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
signal: AbortSignal.any([
|
||||
AbortSignal.timeout(60000),
|
||||
...(signal ? [signal] : []),
|
||||
]),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
return null;
|
||||
}
|
||||
|
||||
const data = await res.json() as UrlContextResponse;
|
||||
activityMonitor.logComplete(activityId, res.status);
|
||||
|
||||
const metadata = data.candidates?.[0]?.url_context_metadata;
|
||||
if (metadata?.url_metadata?.length) {
|
||||
const status = metadata.url_metadata[0].url_retrieval_status;
|
||||
if (status === "URL_RETRIEVAL_STATUS_UNSAFE" || status === "URL_RETRIEVAL_STATUS_ERROR") {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const content = data.candidates?.[0]?.content?.parts
|
||||
?.map(p => p.text).filter(Boolean).join("\n") ?? "";
|
||||
|
||||
if (!content || content.length < 50) return null;
|
||||
|
||||
const title = extractTitleFromContent(content, url);
|
||||
return { url, title, content, error: null };
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function extractWithGeminiWeb(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const cookies = await isGeminiWebAvailable();
|
||||
if (!cookies) return null;
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "api", query: `gemini_web: ${url}` });
|
||||
|
||||
try {
|
||||
const text = await queryWithCookies(EXTRACTION_PROMPT + url, cookies, {
|
||||
model: "gemini-3-flash-preview",
|
||||
signal,
|
||||
timeoutMs: 60000,
|
||||
});
|
||||
|
||||
activityMonitor.logComplete(activityId, 200);
|
||||
|
||||
if (!text || text.length < 50) return null;
|
||||
|
||||
const title = extractTitleFromContent(text, url);
|
||||
return { url, title, content: text, error: null };
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function extractTitleFromContent(text: string, url: string): string {
|
||||
return extractHeadingTitle(text) ?? (new URL(url).pathname.split("/").pop() || url);
|
||||
}
|
||||
|
||||
interface UrlContextResponse {
|
||||
candidates?: Array<{
|
||||
content?: { parts?: Array<{ text?: string }> };
|
||||
url_context_metadata?: {
|
||||
url_metadata?: Array<{
|
||||
retrieved_url?: string;
|
||||
url_retrieval_status?: string;
|
||||
}>;
|
||||
};
|
||||
}>;
|
||||
}
|
||||
296
pi/files/agent/extensions/pi-web-access/gemini-web.ts
Normal file
296
pi/files/agent/extensions/pi-web-access/gemini-web.ts
Normal file
@@ -0,0 +1,296 @@
|
||||
import { type CookieMap, getGoogleCookies } from "./chrome-cookies.js";
|
||||
|
||||
const GEMINI_APP_URL = "https://gemini.google.com/app";
|
||||
const GEMINI_STREAM_GENERATE_URL =
|
||||
"https://gemini.google.com/_/BardChatUi/data/assistant.lamda.BardFrontendService/StreamGenerate";
|
||||
const GEMINI_UPLOAD_URL = "https://content-push.googleapis.com/upload";
|
||||
const GEMINI_UPLOAD_PUSH_ID = "feeds/mcudyrk2a4khkz";
|
||||
|
||||
const USER_AGENT =
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36";
|
||||
|
||||
const MODEL_HEADER_NAME = "x-goog-ext-525001261-jspb";
|
||||
const MODEL_HEADERS: Record<string, string> = {
|
||||
"gemini-3-pro": '[1,null,null,null,"9d8ca3786ebdfbea",null,null,0,[4]]',
|
||||
"gemini-2.5-pro": '[1,null,null,null,"4af6c7f5da75d65d",null,null,0,[4]]',
|
||||
"gemini-2.5-flash": '[1,null,null,null,"9ec249fc9ad08861",null,null,0,[4]]',
|
||||
};
|
||||
|
||||
const REQUIRED_COOKIES = ["__Secure-1PSID", "__Secure-1PSIDTS"];
|
||||
|
||||
export interface GeminiWebOptions {
|
||||
youtubeUrl?: string;
|
||||
model?: string;
|
||||
files?: string[];
|
||||
signal?: AbortSignal;
|
||||
timeoutMs?: number;
|
||||
}
|
||||
|
||||
function hasRequiredCookies(cookieMap: CookieMap): boolean {
|
||||
return REQUIRED_COOKIES.every((name) => Boolean(cookieMap[name]));
|
||||
}
|
||||
|
||||
export async function isGeminiWebAvailable(): Promise<CookieMap | null> {
|
||||
const result = await getGoogleCookies();
|
||||
if (!result || !hasRequiredCookies(result.cookies)) return null;
|
||||
return result.cookies;
|
||||
}
|
||||
|
||||
export async function queryWithCookies(
|
||||
prompt: string,
|
||||
cookieMap: CookieMap,
|
||||
options: GeminiWebOptions = {},
|
||||
): Promise<string> {
|
||||
const model = options.model && MODEL_HEADERS[options.model] ? options.model : "gemini-2.5-flash";
|
||||
const timeoutMs = options.timeoutMs ?? 120000;
|
||||
|
||||
let fullPrompt = prompt;
|
||||
if (options.youtubeUrl) {
|
||||
fullPrompt = `${fullPrompt}\n\nYouTube video: ${options.youtubeUrl}`;
|
||||
}
|
||||
|
||||
const result = await runGeminiWebOnce(fullPrompt, cookieMap, model, options.files, timeoutMs, options.signal);
|
||||
|
||||
if (isModelUnavailable(result.errorCode) && model !== "gemini-2.5-flash") {
|
||||
const fallback = await runGeminiWebOnce(fullPrompt, cookieMap, "gemini-2.5-flash", options.files, timeoutMs, options.signal);
|
||||
if (fallback.errorMessage) throw new Error(fallback.errorMessage);
|
||||
if (!fallback.text) throw new Error("Gemini Web returned empty response (fallback model)");
|
||||
return fallback.text;
|
||||
}
|
||||
|
||||
if (result.errorMessage) throw new Error(result.errorMessage);
|
||||
if (!result.text) throw new Error("Gemini Web returned empty response");
|
||||
return result.text;
|
||||
}
|
||||
|
||||
interface GeminiWebResult {
|
||||
text: string;
|
||||
errorCode?: number;
|
||||
errorMessage?: string;
|
||||
}
|
||||
|
||||
async function runGeminiWebOnce(
|
||||
prompt: string,
|
||||
cookieMap: CookieMap,
|
||||
model: string,
|
||||
files: string[] | undefined,
|
||||
timeoutMs: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<GeminiWebResult> {
|
||||
const effectiveSignal = withTimeout(signal, timeoutMs);
|
||||
const cookieHeader = buildCookieHeader(cookieMap);
|
||||
const accessToken = await fetchAccessToken(cookieHeader, effectiveSignal);
|
||||
|
||||
const uploaded: Array<{ id: string; name: string }> = [];
|
||||
if (files) {
|
||||
for (const filePath of files) {
|
||||
uploaded.push(await uploadFile(filePath, cookieHeader, effectiveSignal));
|
||||
}
|
||||
}
|
||||
|
||||
const fReq = buildFReqPayload(prompt, uploaded);
|
||||
const params = new URLSearchParams();
|
||||
params.set("at", accessToken);
|
||||
params.set("f.req", fReq);
|
||||
|
||||
const res = await fetch(GEMINI_STREAM_GENERATE_URL, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/x-www-form-urlencoded;charset=utf-8",
|
||||
host: "gemini.google.com",
|
||||
origin: "https://gemini.google.com",
|
||||
referer: "https://gemini.google.com/",
|
||||
"x-same-domain": "1",
|
||||
"user-agent": USER_AGENT,
|
||||
cookie: cookieHeader,
|
||||
[MODEL_HEADER_NAME]: MODEL_HEADERS[model],
|
||||
},
|
||||
body: params.toString(),
|
||||
signal: effectiveSignal,
|
||||
});
|
||||
|
||||
const rawText = await res.text();
|
||||
|
||||
if (!res.ok) {
|
||||
return { text: "", errorMessage: `Gemini request failed: ${res.status}` };
|
||||
}
|
||||
|
||||
try {
|
||||
return parseStreamGenerateResponse(rawText);
|
||||
} catch (err) {
|
||||
let errorCode: number | undefined;
|
||||
try {
|
||||
const json = JSON.parse(trimJsonEnvelope(rawText));
|
||||
errorCode = extractErrorCode(json);
|
||||
} catch {}
|
||||
return {
|
||||
text: "",
|
||||
errorCode,
|
||||
errorMessage: err instanceof Error ? err.message : String(err),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchAccessToken(
|
||||
cookieHeader: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<string> {
|
||||
const html = await fetchWithCookieRedirects(GEMINI_APP_URL, cookieHeader, 10, signal);
|
||||
|
||||
for (const key of ["SNlM0e", "thykhd"]) {
|
||||
const match = html.match(new RegExp(`"${key}":"(.*?)"`));
|
||||
if (match?.[1]) return match[1];
|
||||
}
|
||||
|
||||
throw new Error("Unable to authenticate with Gemini. Make sure you're signed into gemini.google.com in Chrome.");
|
||||
}
|
||||
|
||||
async function fetchWithCookieRedirects(
|
||||
url: string,
|
||||
cookieHeader: string,
|
||||
maxRedirects: number,
|
||||
signal: AbortSignal,
|
||||
): Promise<string> {
|
||||
let current = url;
|
||||
for (let i = 0; i <= maxRedirects; i++) {
|
||||
const res = await fetch(current, {
|
||||
headers: { "user-agent": USER_AGENT, cookie: cookieHeader },
|
||||
redirect: "manual",
|
||||
signal,
|
||||
});
|
||||
if (res.status >= 300 && res.status < 400) {
|
||||
const location = res.headers.get("location");
|
||||
if (location) {
|
||||
current = new URL(location, current).toString();
|
||||
continue;
|
||||
}
|
||||
}
|
||||
return await res.text();
|
||||
}
|
||||
throw new Error(`Too many redirects (>${maxRedirects})`);
|
||||
}
|
||||
|
||||
async function uploadFile(
|
||||
filePath: string,
|
||||
cookieHeader: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<{ id: string; name: string }> {
|
||||
const { readFileSync } = await import("node:fs");
|
||||
const { basename } = await import("node:path");
|
||||
|
||||
const data = readFileSync(filePath);
|
||||
const fileName = basename(filePath);
|
||||
const boundary = "----FormBoundary" + Math.random().toString(36).slice(2);
|
||||
const header = `--${boundary}\r\nContent-Disposition: form-data; name="file"; filename="${fileName}"\r\nContent-Type: application/octet-stream\r\n\r\n`;
|
||||
const footer = `\r\n--${boundary}--\r\n`;
|
||||
|
||||
const body = Buffer.concat([
|
||||
Buffer.from(header, "utf-8"),
|
||||
data,
|
||||
Buffer.from(footer, "utf-8"),
|
||||
]);
|
||||
|
||||
const res = await fetch(GEMINI_UPLOAD_URL, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": `multipart/form-data; boundary=${boundary}`,
|
||||
"push-id": GEMINI_UPLOAD_PUSH_ID,
|
||||
"user-agent": USER_AGENT,
|
||||
cookie: cookieHeader,
|
||||
},
|
||||
body,
|
||||
signal,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const text = await res.text();
|
||||
throw new Error(`File upload failed: ${res.status} (${text.slice(0, 200)})`);
|
||||
}
|
||||
|
||||
return { id: await res.text(), name: fileName };
|
||||
}
|
||||
|
||||
function buildFReqPayload(
|
||||
prompt: string,
|
||||
uploaded: Array<{ id: string; name: string }>,
|
||||
): string {
|
||||
const promptPayload =
|
||||
uploaded.length > 0
|
||||
? [prompt, 0, null, uploaded.map((file) => [[file.id, 1]])]
|
||||
: [prompt];
|
||||
const innerList = [promptPayload, null, null];
|
||||
return JSON.stringify([null, JSON.stringify(innerList)]);
|
||||
}
|
||||
|
||||
function withTimeout(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
|
||||
const timeout = AbortSignal.timeout(timeoutMs);
|
||||
return signal ? AbortSignal.any([signal, timeout]) : timeout;
|
||||
}
|
||||
|
||||
function buildCookieHeader(cookieMap: CookieMap): string {
|
||||
return Object.entries(cookieMap)
|
||||
.filter(([, value]) => typeof value === "string" && value.length > 0)
|
||||
.map(([name, value]) => `${name}=${value}`)
|
||||
.join("; ");
|
||||
}
|
||||
|
||||
function getNestedValue(value: unknown, pathParts: number[]): unknown {
|
||||
let current: unknown = value;
|
||||
for (const part of pathParts) {
|
||||
if (current == null) return undefined;
|
||||
if (!Array.isArray(current)) return undefined;
|
||||
current = (current as unknown[])[part];
|
||||
}
|
||||
return current;
|
||||
}
|
||||
|
||||
function trimJsonEnvelope(text: string): string {
|
||||
const start = text.indexOf("[");
|
||||
const end = text.lastIndexOf("]");
|
||||
if (start === -1 || end === -1 || end <= start) {
|
||||
throw new Error("Gemini response did not contain a JSON payload.");
|
||||
}
|
||||
return text.slice(start, end + 1);
|
||||
}
|
||||
|
||||
function extractErrorCode(responseJson: unknown): number | undefined {
|
||||
const code = getNestedValue(responseJson, [0, 5, 2, 0, 1, 0]);
|
||||
return typeof code === "number" && code >= 0 ? code : undefined;
|
||||
}
|
||||
|
||||
function isModelUnavailable(errorCode: number | undefined): boolean {
|
||||
return errorCode === 1052;
|
||||
}
|
||||
|
||||
function parseStreamGenerateResponse(rawText: string): GeminiWebResult {
|
||||
const responseJson = JSON.parse(trimJsonEnvelope(rawText));
|
||||
const errorCode = extractErrorCode(responseJson);
|
||||
|
||||
const parts = Array.isArray(responseJson) ? responseJson : [];
|
||||
let body: unknown = null;
|
||||
|
||||
for (let i = 0; i < parts.length; i++) {
|
||||
const partBody = getNestedValue(parts[i], [2]);
|
||||
if (!partBody || typeof partBody !== "string") continue;
|
||||
try {
|
||||
const parsed = JSON.parse(partBody);
|
||||
const candidateList = getNestedValue(parsed, [4]);
|
||||
if (Array.isArray(candidateList) && candidateList.length > 0) {
|
||||
body = parsed;
|
||||
break;
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
|
||||
const candidateList = getNestedValue(body, [4]);
|
||||
const firstCandidate = Array.isArray(candidateList) ? (candidateList as unknown[])[0] : undefined;
|
||||
const textRaw = getNestedValue(firstCandidate, [1, 0]) as string | undefined;
|
||||
|
||||
let text = textRaw ?? "";
|
||||
if (/^http:\/\/googleusercontent\.com\/card_content\/\d+/.test(text)) {
|
||||
const alt = getNestedValue(firstCandidate, [22, 0]) as string | undefined;
|
||||
if (alt) text = alt;
|
||||
}
|
||||
|
||||
return { text, errorCode };
|
||||
}
|
||||
196
pi/files/agent/extensions/pi-web-access/github-api.ts
Normal file
196
pi/files/agent/extensions/pi-web-access/github-api.ts
Normal file
@@ -0,0 +1,196 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import type { ExtractedContent } from "./extract.js";
|
||||
import type { GitHubUrlInfo } from "./github-extract.js";
|
||||
|
||||
const MAX_TREE_ENTRIES = 200;
|
||||
const MAX_INLINE_FILE_CHARS = 100_000;
|
||||
|
||||
let ghAvailable: boolean | null = null;
|
||||
let ghHintShown = false;
|
||||
|
||||
export async function checkGhAvailable(): Promise<boolean> {
|
||||
if (ghAvailable !== null) return ghAvailable;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
execFile("gh", ["--version"], { timeout: 5000 }, (err) => {
|
||||
ghAvailable = !err;
|
||||
resolve(ghAvailable);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export function showGhHint(): void {
|
||||
if (!ghHintShown) {
|
||||
ghHintShown = true;
|
||||
console.error("[pi-web-access] Install `gh` CLI for better GitHub repo access including private repos.");
|
||||
}
|
||||
}
|
||||
|
||||
export async function checkRepoSize(owner: string, repo: string): Promise<number | null> {
|
||||
if (!(await checkGhAvailable())) return null;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
execFile("gh", ["api", `repos/${owner}/${repo}`, "--jq", ".size"], { timeout: 10000 }, (err, stdout) => {
|
||||
if (err) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
const kb = parseInt(stdout.trim(), 10);
|
||||
resolve(Number.isNaN(kb) ? null : kb);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function getDefaultBranch(owner: string, repo: string): Promise<string | null> {
|
||||
if (!(await checkGhAvailable())) return null;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
execFile("gh", ["api", `repos/${owner}/${repo}`, "--jq", ".default_branch"], { timeout: 10000 }, (err, stdout) => {
|
||||
if (err) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
const branch = stdout.trim();
|
||||
resolve(branch || null);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchTreeViaApi(owner: string, repo: string, ref: string): Promise<string | null> {
|
||||
if (!(await checkGhAvailable())) return null;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
"gh",
|
||||
["api", `repos/${owner}/${repo}/git/trees/${ref}?recursive=1`, "--jq", ".tree[].path"],
|
||||
{ timeout: 15000, maxBuffer: 5 * 1024 * 1024 },
|
||||
(err, stdout) => {
|
||||
if (err) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
const paths = stdout.trim().split("\n").filter(Boolean);
|
||||
if (paths.length === 0) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
const truncated = paths.length > MAX_TREE_ENTRIES;
|
||||
const display = paths.slice(0, MAX_TREE_ENTRIES).join("\n");
|
||||
resolve(truncated ? display + `\n... (${paths.length} total entries)` : display);
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchReadmeViaApi(owner: string, repo: string, ref: string): Promise<string | null> {
|
||||
if (!(await checkGhAvailable())) return null;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
"gh",
|
||||
["api", `repos/${owner}/${repo}/readme?ref=${ref}`, "--jq", ".content"],
|
||||
{ timeout: 10000 },
|
||||
(err, stdout) => {
|
||||
if (err) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const decoded = Buffer.from(stdout.trim(), "base64").toString("utf-8");
|
||||
resolve(decoded.length > 8192 ? decoded.slice(0, 8192) + "\n\n[README truncated at 8K chars]" : decoded);
|
||||
} catch {
|
||||
resolve(null);
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchFileViaApi(owner: string, repo: string, path: string, ref: string): Promise<string | null> {
|
||||
if (!(await checkGhAvailable())) return null;
|
||||
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
"gh",
|
||||
["api", `repos/${owner}/${repo}/contents/${path}?ref=${ref}`, "--jq", ".content"],
|
||||
{ timeout: 10000, maxBuffer: 2 * 1024 * 1024 },
|
||||
(err, stdout) => {
|
||||
if (err) {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
resolve(Buffer.from(stdout.trim(), "base64").toString("utf-8"));
|
||||
} catch {
|
||||
resolve(null);
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
export async function fetchViaApi(
|
||||
url: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
info: GitHubUrlInfo,
|
||||
sizeNote?: string,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const ref = info.ref || (await getDefaultBranch(owner, repo));
|
||||
if (!ref) return null;
|
||||
|
||||
const lines: string[] = [];
|
||||
if (sizeNote) {
|
||||
lines.push(sizeNote);
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
if (info.type === "blob" && info.path) {
|
||||
const content = await fetchFileViaApi(owner, repo, info.path, ref);
|
||||
if (!content) return null;
|
||||
|
||||
lines.push(`## ${info.path}`);
|
||||
if (content.length > MAX_INLINE_FILE_CHARS) {
|
||||
lines.push(content.slice(0, MAX_INLINE_FILE_CHARS));
|
||||
lines.push(`\n[File truncated at 100K chars]`);
|
||||
} else {
|
||||
lines.push(content);
|
||||
}
|
||||
|
||||
return {
|
||||
url,
|
||||
title: `${owner}/${repo} - ${info.path}`,
|
||||
content: lines.join("\n"),
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
|
||||
const [tree, readme] = await Promise.all([
|
||||
fetchTreeViaApi(owner, repo, ref),
|
||||
fetchReadmeViaApi(owner, repo, ref),
|
||||
]);
|
||||
|
||||
if (!tree && !readme) return null;
|
||||
|
||||
if (tree) {
|
||||
lines.push("## Structure");
|
||||
lines.push(tree);
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
if (readme) {
|
||||
lines.push("## README.md");
|
||||
lines.push(readme);
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
lines.push("This is an API-only view. Clone the repo or use `read`/`bash` for deeper exploration.");
|
||||
|
||||
const title = info.path ? `${owner}/${repo} - ${info.path}` : `${owner}/${repo}`;
|
||||
return {
|
||||
url,
|
||||
title,
|
||||
content: lines.join("\n"),
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
505
pi/files/agent/extensions/pi-web-access/github-extract.ts
Normal file
505
pi/files/agent/extensions/pi-web-access/github-extract.ts
Normal file
@@ -0,0 +1,505 @@
|
||||
import { existsSync, readFileSync, rmSync, statSync, readdirSync, openSync, readSync, closeSync } from "node:fs";
|
||||
import { execFile } from "node:child_process";
|
||||
import { homedir } from "node:os";
|
||||
import { join, extname } from "node:path";
|
||||
import { activityMonitor } from "./activity.js";
|
||||
import type { ExtractedContent } from "./extract.js";
|
||||
import { checkGhAvailable, checkRepoSize, fetchViaApi, showGhHint } from "./github-api.js";
|
||||
|
||||
const CONFIG_PATH = join(homedir(), ".pi", "web-search.json");
|
||||
|
||||
const BINARY_EXTENSIONS = new Set([
|
||||
".png", ".jpg", ".jpeg", ".gif", ".bmp", ".ico", ".webp", ".svg", ".tiff", ".tif",
|
||||
".mp3", ".mp4", ".avi", ".mov", ".mkv", ".flv", ".wmv", ".wav", ".ogg", ".webm", ".flac", ".aac",
|
||||
".zip", ".tar", ".gz", ".bz2", ".xz", ".7z", ".rar", ".zst",
|
||||
".exe", ".dll", ".so", ".dylib", ".bin", ".o", ".a", ".lib",
|
||||
".woff", ".woff2", ".ttf", ".otf", ".eot",
|
||||
".pdf", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx",
|
||||
".sqlite", ".db", ".sqlite3",
|
||||
".pyc", ".pyo", ".class", ".jar", ".war",
|
||||
".iso", ".img", ".dmg",
|
||||
]);
|
||||
|
||||
const NOISE_DIRS = new Set([
|
||||
"node_modules", "vendor", ".next", "dist", "build", "__pycache__",
|
||||
".venv", "venv", ".tox", ".mypy_cache", ".pytest_cache",
|
||||
"target", ".gradle", ".idea", ".vscode",
|
||||
]);
|
||||
|
||||
const MAX_INLINE_FILE_CHARS = 100_000;
|
||||
const MAX_TREE_ENTRIES = 200;
|
||||
|
||||
export interface GitHubUrlInfo {
|
||||
owner: string;
|
||||
repo: string;
|
||||
ref?: string;
|
||||
refIsFullSha: boolean;
|
||||
path?: string;
|
||||
type: "root" | "blob" | "tree";
|
||||
}
|
||||
|
||||
interface CachedClone {
|
||||
localPath: string;
|
||||
clonePromise: Promise<string | null>;
|
||||
}
|
||||
|
||||
interface GitHubCloneConfig {
|
||||
enabled: boolean;
|
||||
maxRepoSizeMB: number;
|
||||
cloneTimeoutSeconds: number;
|
||||
clonePath: string;
|
||||
}
|
||||
|
||||
const cloneCache = new Map<string, CachedClone>();
|
||||
|
||||
let cachedConfig: GitHubCloneConfig | null = null;
|
||||
|
||||
function loadGitHubConfig(): GitHubCloneConfig {
|
||||
if (cachedConfig) return cachedConfig;
|
||||
|
||||
const defaults: GitHubCloneConfig = {
|
||||
enabled: true,
|
||||
maxRepoSizeMB: 350,
|
||||
cloneTimeoutSeconds: 30,
|
||||
clonePath: "/tmp/pi-github-repos",
|
||||
};
|
||||
|
||||
try {
|
||||
if (existsSync(CONFIG_PATH)) {
|
||||
const raw = JSON.parse(readFileSync(CONFIG_PATH, "utf-8"));
|
||||
const gc = raw.githubClone ?? {};
|
||||
cachedConfig = {
|
||||
enabled: gc.enabled ?? defaults.enabled,
|
||||
maxRepoSizeMB: gc.maxRepoSizeMB ?? defaults.maxRepoSizeMB,
|
||||
cloneTimeoutSeconds: gc.cloneTimeoutSeconds ?? defaults.cloneTimeoutSeconds,
|
||||
clonePath: gc.clonePath ?? defaults.clonePath,
|
||||
};
|
||||
return cachedConfig;
|
||||
}
|
||||
} catch {
|
||||
// ignore parse errors
|
||||
}
|
||||
|
||||
cachedConfig = defaults;
|
||||
return cachedConfig;
|
||||
}
|
||||
|
||||
const NON_CODE_SEGMENTS = new Set([
|
||||
"issues", "pull", "pulls", "discussions", "releases", "wiki",
|
||||
"actions", "settings", "security", "projects", "graphs",
|
||||
"compare", "commits", "tags", "branches", "stargazers",
|
||||
"watchers", "network", "forks", "milestone", "labels",
|
||||
"packages", "codespaces", "contribute", "community",
|
||||
"sponsors", "invitations", "notifications", "insights",
|
||||
]);
|
||||
|
||||
export function parseGitHubUrl(url: string): GitHubUrlInfo | null {
|
||||
let parsed: URL;
|
||||
try {
|
||||
parsed = new URL(url);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (parsed.hostname !== "github.com") return null;
|
||||
|
||||
const segments = parsed.pathname.split("/").filter(Boolean);
|
||||
if (segments.length < 2) return null;
|
||||
|
||||
const owner = segments[0];
|
||||
const repo = segments[1].replace(/\.git$/, "");
|
||||
|
||||
if (NON_CODE_SEGMENTS.has(segments[2]?.toLowerCase())) return null;
|
||||
|
||||
if (segments.length === 2) {
|
||||
return { owner, repo, refIsFullSha: false, type: "root" };
|
||||
}
|
||||
|
||||
const action = segments[2];
|
||||
if (action !== "blob" && action !== "tree") return null;
|
||||
if (segments.length < 4) return null;
|
||||
|
||||
const ref = segments[3];
|
||||
const refIsFullSha = /^[0-9a-f]{40}$/.test(ref);
|
||||
const pathParts = segments.slice(4);
|
||||
const path = pathParts.length > 0 ? pathParts.join("/") : "";
|
||||
|
||||
return {
|
||||
owner,
|
||||
repo,
|
||||
ref,
|
||||
refIsFullSha,
|
||||
path,
|
||||
type: action as "blob" | "tree",
|
||||
};
|
||||
}
|
||||
|
||||
function cacheKey(owner: string, repo: string, ref?: string): string {
|
||||
return ref ? `${owner}/${repo}@${ref}` : `${owner}/${repo}`;
|
||||
}
|
||||
|
||||
function cloneDir(config: GitHubCloneConfig, owner: string, repo: string, ref?: string): string {
|
||||
const dirName = ref ? `${repo}@${ref}` : repo;
|
||||
return join(config.clonePath, owner, dirName);
|
||||
}
|
||||
|
||||
function execClone(args: string[], localPath: string, timeoutMs: number, signal?: AbortSignal): Promise<string | null> {
|
||||
return new Promise((resolve) => {
|
||||
const child = execFile(args[0], args.slice(1), { timeout: timeoutMs }, (err) => {
|
||||
if (err) {
|
||||
try {
|
||||
rmSync(localPath, { recursive: true, force: true });
|
||||
} catch { /* ignore */ }
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
resolve(localPath);
|
||||
});
|
||||
|
||||
if (signal) {
|
||||
const onAbort = () => child.kill();
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
child.on("exit", () => signal.removeEventListener("abort", onAbort));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async function cloneRepo(
|
||||
owner: string,
|
||||
repo: string,
|
||||
ref: string | undefined,
|
||||
config: GitHubCloneConfig,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string | null> {
|
||||
const localPath = cloneDir(config, owner, repo, ref);
|
||||
|
||||
try {
|
||||
rmSync(localPath, { recursive: true, force: true });
|
||||
} catch { /* ignore */ }
|
||||
|
||||
const timeoutMs = config.cloneTimeoutSeconds * 1000;
|
||||
const hasGh = await checkGhAvailable();
|
||||
|
||||
if (hasGh) {
|
||||
const args = ["gh", "repo", "clone", `${owner}/${repo}`, localPath, "--", "--depth", "1", "--single-branch"];
|
||||
if (ref) args.push("--branch", ref);
|
||||
return execClone(args, localPath, timeoutMs, signal);
|
||||
}
|
||||
|
||||
showGhHint();
|
||||
|
||||
const gitUrl = `https://github.com/${owner}/${repo}.git`;
|
||||
const args = ["git", "clone", "--depth", "1", "--single-branch"];
|
||||
if (ref) args.push("--branch", ref);
|
||||
args.push(gitUrl, localPath);
|
||||
return execClone(args, localPath, timeoutMs, signal);
|
||||
}
|
||||
|
||||
function isBinaryFile(filePath: string): boolean {
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
if (BINARY_EXTENSIONS.has(ext)) return true;
|
||||
|
||||
let fd: number;
|
||||
try {
|
||||
fd = openSync(filePath, "r");
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
const buf = Buffer.alloc(512);
|
||||
const bytesRead = readSync(fd, buf, 0, 512, 0);
|
||||
for (let i = 0; i < bytesRead; i++) {
|
||||
if (buf[i] === 0) return true;
|
||||
}
|
||||
} catch {
|
||||
return false;
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
function formatFileSize(bytes: number): string {
|
||||
if (bytes < 1024) return `${bytes} B`;
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
|
||||
}
|
||||
|
||||
function buildTree(rootPath: string): string {
|
||||
const entries: string[] = [];
|
||||
|
||||
function walk(dir: string, relPath: string): void {
|
||||
if (entries.length >= MAX_TREE_ENTRIES) return;
|
||||
|
||||
let items: string[];
|
||||
try {
|
||||
items = readdirSync(dir).sort();
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
for (const item of items) {
|
||||
if (entries.length >= MAX_TREE_ENTRIES) return;
|
||||
if (item === ".git") continue;
|
||||
|
||||
const fullPath = join(dir, item);
|
||||
let stat;
|
||||
try {
|
||||
stat = statSync(fullPath);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
const rel = relPath ? `${relPath}/${item}` : item;
|
||||
|
||||
if (stat.isDirectory()) {
|
||||
if (NOISE_DIRS.has(item)) {
|
||||
entries.push(`${rel}/ [skipped]`);
|
||||
continue;
|
||||
}
|
||||
entries.push(`${rel}/`);
|
||||
walk(fullPath, rel);
|
||||
} else {
|
||||
entries.push(rel);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
walk(rootPath, "");
|
||||
|
||||
if (entries.length >= MAX_TREE_ENTRIES) {
|
||||
entries.push(`... (truncated at ${MAX_TREE_ENTRIES} entries)`);
|
||||
}
|
||||
|
||||
return entries.join("\n");
|
||||
}
|
||||
|
||||
function buildDirListing(rootPath: string, subPath: string): string {
|
||||
const targetPath = join(rootPath, subPath);
|
||||
const lines: string[] = [];
|
||||
|
||||
let items: string[];
|
||||
try {
|
||||
items = readdirSync(targetPath).sort();
|
||||
} catch {
|
||||
return "(directory not readable)";
|
||||
}
|
||||
|
||||
for (const item of items) {
|
||||
if (item === ".git") continue;
|
||||
const fullPath = join(targetPath, item);
|
||||
try {
|
||||
const stat = statSync(fullPath);
|
||||
if (stat.isDirectory()) {
|
||||
lines.push(` ${item}/`);
|
||||
} else {
|
||||
lines.push(` ${item} (${formatFileSize(stat.size)})`);
|
||||
}
|
||||
} catch {
|
||||
lines.push(` ${item} (unreadable)`);
|
||||
}
|
||||
}
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function readReadme(localPath: string): string | null {
|
||||
const candidates = ["README.md", "readme.md", "README", "README.txt", "README.rst"];
|
||||
for (const name of candidates) {
|
||||
const readmePath = join(localPath, name);
|
||||
if (existsSync(readmePath)) {
|
||||
try {
|
||||
const content = readFileSync(readmePath, "utf-8");
|
||||
return content.length > 8192 ? content.slice(0, 8192) + "\n\n[README truncated at 8K chars]" : content;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function generateContent(localPath: string, info: GitHubUrlInfo): string {
|
||||
const lines: string[] = [];
|
||||
lines.push(`Repository cloned to: ${localPath}`);
|
||||
lines.push("");
|
||||
|
||||
if (info.type === "root") {
|
||||
lines.push("## Structure");
|
||||
lines.push(buildTree(localPath));
|
||||
lines.push("");
|
||||
|
||||
const readme = readReadme(localPath);
|
||||
if (readme) {
|
||||
lines.push("## README.md");
|
||||
lines.push(readme);
|
||||
lines.push("");
|
||||
}
|
||||
|
||||
lines.push("Use `read` and `bash` tools at the path above to explore further.");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
if (info.type === "tree") {
|
||||
const dirPath = info.path || "";
|
||||
const fullDirPath = join(localPath, dirPath);
|
||||
|
||||
if (!existsSync(fullDirPath)) {
|
||||
lines.push(`Path \`${dirPath}\` not found in clone. Showing repository root instead.`);
|
||||
lines.push("");
|
||||
lines.push("## Structure");
|
||||
lines.push(buildTree(localPath));
|
||||
} else {
|
||||
lines.push(`## ${dirPath || "/"}`);
|
||||
lines.push(buildDirListing(localPath, dirPath));
|
||||
}
|
||||
|
||||
lines.push("");
|
||||
lines.push("Use `read` and `bash` tools at the path above to explore further.");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
if (info.type === "blob") {
|
||||
const filePath = info.path || "";
|
||||
const fullFilePath = join(localPath, filePath);
|
||||
|
||||
if (!existsSync(fullFilePath)) {
|
||||
lines.push(`Path \`${filePath}\` not found in clone. Showing repository root instead.`);
|
||||
lines.push("");
|
||||
lines.push("## Structure");
|
||||
lines.push(buildTree(localPath));
|
||||
lines.push("");
|
||||
lines.push("Use `read` and `bash` tools at the path above to explore further.");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
const stat = statSync(fullFilePath);
|
||||
|
||||
if (stat.isDirectory()) {
|
||||
lines.push(`## ${filePath || "/"}`);
|
||||
lines.push(buildDirListing(localPath, filePath));
|
||||
lines.push("");
|
||||
lines.push("Use `read` and `bash` tools at the path above to explore further.");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
if (isBinaryFile(fullFilePath)) {
|
||||
const ext = extname(filePath).replace(".", "");
|
||||
lines.push(`## ${filePath}`);
|
||||
lines.push(`Binary file (${ext}, ${formatFileSize(stat.size)}). Use \`read\` or \`bash\` tools at the path above to inspect.`);
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
const content = readFileSync(fullFilePath, "utf-8");
|
||||
lines.push(`## ${filePath}`);
|
||||
|
||||
if (content.length > MAX_INLINE_FILE_CHARS) {
|
||||
lines.push(content.slice(0, MAX_INLINE_FILE_CHARS));
|
||||
lines.push("");
|
||||
lines.push(`[File truncated at 100K chars. Full file: ${fullFilePath}]`);
|
||||
} else {
|
||||
lines.push(content);
|
||||
}
|
||||
|
||||
lines.push("");
|
||||
lines.push("Use `read` and `bash` tools at the path above to explore further.");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
async function awaitCachedClone(
|
||||
cached: CachedClone,
|
||||
url: string,
|
||||
owner: string,
|
||||
repo: string,
|
||||
info: GitHubUrlInfo,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
if (signal?.aborted) return fetchViaApi(url, owner, repo, info);
|
||||
const result = await cached.clonePromise;
|
||||
if (signal?.aborted) return fetchViaApi(url, owner, repo, info);
|
||||
if (result) {
|
||||
const content = generateContent(result, info);
|
||||
const title = info.path ? `${owner}/${repo} - ${info.path}` : `${owner}/${repo}`;
|
||||
return { url, title, content, error: null };
|
||||
}
|
||||
return fetchViaApi(url, owner, repo, info);
|
||||
}
|
||||
|
||||
export async function extractGitHub(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
forceClone?: boolean,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const info = parseGitHubUrl(url);
|
||||
if (!info) return null;
|
||||
|
||||
const config = loadGitHubConfig();
|
||||
if (!config.enabled) return null;
|
||||
|
||||
const { owner, repo } = info;
|
||||
const key = cacheKey(owner, repo, info.ref);
|
||||
|
||||
const cached = cloneCache.get(key);
|
||||
if (cached) return awaitCachedClone(cached, url, owner, repo, info, signal);
|
||||
|
||||
if (info.refIsFullSha) {
|
||||
const sizeNote = `Note: Commit SHA URLs use the GitHub API instead of cloning.`;
|
||||
return fetchViaApi(url, owner, repo, info, sizeNote);
|
||||
}
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "fetch", url: `github.com/${owner}/${repo}` });
|
||||
|
||||
if (!forceClone) {
|
||||
const sizeKB = await checkRepoSize(owner, repo);
|
||||
if (sizeKB !== null) {
|
||||
const sizeMB = sizeKB / 1024;
|
||||
if (sizeMB > config.maxRepoSizeMB) {
|
||||
activityMonitor.logComplete(activityId, 200);
|
||||
const sizeNote =
|
||||
`Note: Repository is ${Math.round(sizeMB)}MB (threshold: ${config.maxRepoSizeMB}MB). ` +
|
||||
`Showing API-fetched content instead of full clone. Ask the user if they'd like to clone the full repo -- ` +
|
||||
`if yes, call fetch_content again with the same URL and add forceClone: true to the params.`;
|
||||
return fetchViaApi(url, owner, repo, info, sizeNote);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Re-check: another concurrent caller may have started a clone while we awaited the size check
|
||||
const cachedAfterSizeCheck = cloneCache.get(key);
|
||||
if (cachedAfterSizeCheck) return awaitCachedClone(cachedAfterSizeCheck, url, owner, repo, info, signal);
|
||||
|
||||
const clonePromise = cloneRepo(owner, repo, info.ref, config, signal);
|
||||
const localPath = cloneDir(config, owner, repo, info.ref);
|
||||
cloneCache.set(key, { localPath, clonePromise });
|
||||
|
||||
const result = await clonePromise;
|
||||
|
||||
if (!result) {
|
||||
cloneCache.delete(key);
|
||||
activityMonitor.logError(activityId, "clone failed");
|
||||
|
||||
const apiFallback = await fetchViaApi(url, owner, repo, info);
|
||||
if (apiFallback) return apiFallback;
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
activityMonitor.logComplete(activityId, 200);
|
||||
const content = generateContent(result, info);
|
||||
const title = info.path ? `${owner}/${repo} - ${info.path}` : `${owner}/${repo}`;
|
||||
return { url, title, content, error: null };
|
||||
}
|
||||
|
||||
export function clearCloneCache(): void {
|
||||
for (const entry of cloneCache.values()) {
|
||||
try {
|
||||
rmSync(entry.localPath, { recursive: true, force: true });
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
cloneCache.clear();
|
||||
cachedConfig = null;
|
||||
}
|
||||
1618
pi/files/agent/extensions/pi-web-access/index.ts
Normal file
1618
pi/files/agent/extensions/pi-web-access/index.ts
Normal file
File diff suppressed because it is too large
Load Diff
248
pi/files/agent/extensions/pi-web-access/package-lock.json
generated
Normal file
248
pi/files/agent/extensions/pi-web-access/package-lock.json
generated
Normal file
@@ -0,0 +1,248 @@
|
||||
{
|
||||
"name": "pi-web-access",
|
||||
"version": "0.10.2",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "pi-web-access",
|
||||
"version": "0.10.2",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@mozilla/readability": "^0.5.0",
|
||||
"linkedom": "^0.16.0",
|
||||
"p-limit": "^6.1.0",
|
||||
"turndown": "^7.2.0",
|
||||
"unpdf": "^1.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@mixmark-io/domino": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@mixmark-io/domino/-/domino-2.2.0.tgz",
|
||||
"integrity": "sha512-Y28PR25bHXUg88kCV7nivXrP2Nj2RueZ3/l/jdx6J9f8J4nsEGcgX0Qe6lt7Pa+J79+kPiJU3LguR6O/6zrLOw==",
|
||||
"license": "BSD-2-Clause"
|
||||
},
|
||||
"node_modules/@mozilla/readability": {
|
||||
"version": "0.5.0",
|
||||
"resolved": "https://registry.npmjs.org/@mozilla/readability/-/readability-0.5.0.tgz",
|
||||
"integrity": "sha512-Z+CZ3QaosfFaTqvhQsIktyGrjFjSC0Fa4EMph4mqKnWhmyoGICsV/8QK+8HpXut6zV7zwfWwqDmEjtk1Qf6EgQ==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=14.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/boolbase": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
|
||||
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/css-select": {
|
||||
"version": "5.2.2",
|
||||
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz",
|
||||
"integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"boolbase": "^1.0.0",
|
||||
"css-what": "^6.1.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"domutils": "^3.0.1",
|
||||
"nth-check": "^2.0.1"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
},
|
||||
"node_modules/css-what": {
|
||||
"version": "6.2.2",
|
||||
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz",
|
||||
"integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
},
|
||||
"node_modules/cssom": {
|
||||
"version": "0.5.0",
|
||||
"resolved": "https://registry.npmjs.org/cssom/-/cssom-0.5.0.tgz",
|
||||
"integrity": "sha512-iKuQcq+NdHqlAcwUY0o/HL69XQrUaQdMjmStJ8JFmUaiiQErlhrmuigkg/CU4E2J0IyUKUrMAgl36TvN67MqTw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/dom-serializer": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
|
||||
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.2",
|
||||
"entities": "^4.2.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/domelementtype": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
|
||||
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
],
|
||||
"license": "BSD-2-Clause"
|
||||
},
|
||||
"node_modules/domhandler": {
|
||||
"version": "5.0.3",
|
||||
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
|
||||
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/domhandler?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/domutils": {
|
||||
"version": "3.2.2",
|
||||
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz",
|
||||
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"dom-serializer": "^2.0.0",
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/domutils?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/entities": {
|
||||
"version": "4.5.0",
|
||||
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
|
||||
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">=0.12"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/entities?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/html-escaper": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-3.0.3.tgz",
|
||||
"integrity": "sha512-RuMffC89BOWQoY0WKGpIhn5gX3iI54O6nRA0yC124NYVtzjmFWBIiFd8M0x+ZdX0P9R4lADg1mgP8C7PxGOWuQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/htmlparser2": {
|
||||
"version": "9.1.0",
|
||||
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-9.1.0.tgz",
|
||||
"integrity": "sha512-5zfg6mHUoaer/97TxnGpxmbR7zJtPwIYFMZ/H5ucTlPZhKvtum05yiPK3Mgai3a0DyVxv7qYqoweaEd2nrYQzQ==",
|
||||
"funding": [
|
||||
"https://github.com/fb55/htmlparser2?sponsor=1",
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/fb55"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"domelementtype": "^2.3.0",
|
||||
"domhandler": "^5.0.3",
|
||||
"domutils": "^3.1.0",
|
||||
"entities": "^4.5.0"
|
||||
}
|
||||
},
|
||||
"node_modules/linkedom": {
|
||||
"version": "0.16.11",
|
||||
"resolved": "https://registry.npmjs.org/linkedom/-/linkedom-0.16.11.tgz",
|
||||
"integrity": "sha512-WgaTVbj7itjyXTsCvgerpneERXShcnNJF5VIV+/4SLtyRLN+HppPre/WDHRofAr2IpEuujSNgJbCBd5lMl6lRw==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"css-select": "^5.1.0",
|
||||
"cssom": "^0.5.0",
|
||||
"html-escaper": "^3.0.3",
|
||||
"htmlparser2": "^9.1.0",
|
||||
"uhyphen": "^0.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/nth-check": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz",
|
||||
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
|
||||
"license": "BSD-2-Clause",
|
||||
"dependencies": {
|
||||
"boolbase": "^1.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/nth-check?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/p-limit": {
|
||||
"version": "6.2.0",
|
||||
"resolved": "https://registry.npmjs.org/p-limit/-/p-limit-6.2.0.tgz",
|
||||
"integrity": "sha512-kuUqqHNUqoIWp/c467RI4X6mmyuojY5jGutNU0wVTmEOOfcuwLqyMVoAi9MKi2Ak+5i9+nhmrK4ufZE8069kHA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"yocto-queue": "^1.1.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/turndown": {
|
||||
"version": "7.2.2",
|
||||
"resolved": "https://registry.npmjs.org/turndown/-/turndown-7.2.2.tgz",
|
||||
"integrity": "sha512-1F7db8BiExOKxjSMU2b7if62D/XOyQyZbPKq/nUwopfgnHlqXHqQ0lvfUTeUIr1lZJzOPFn43dODyMSIfvWRKQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@mixmark-io/domino": "^2.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/uhyphen": {
|
||||
"version": "0.2.0",
|
||||
"resolved": "https://registry.npmjs.org/uhyphen/-/uhyphen-0.2.0.tgz",
|
||||
"integrity": "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/unpdf": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/unpdf/-/unpdf-1.4.0.tgz",
|
||||
"integrity": "sha512-TahIk0xdH/4jh/MxfclzU79g40OyxtP00VnEUZdEkJoYtXAHWLiir6t3FC6z3vDqQTzc2ZHcla6uEiVTNjejuA==",
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"@napi-rs/canvas": "^0.1.69"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@napi-rs/canvas": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/yocto-queue": {
|
||||
"version": "1.2.2",
|
||||
"resolved": "https://registry.npmjs.org/yocto-queue/-/yocto-queue-1.2.2.tgz",
|
||||
"integrity": "sha512-4LCcse/U2MHZ63HAJVE+v71o7yOdIe4cZ70Wpf8D/IyjDKYQLV5GD46B+hSTjJsvV5PztjvHoU580EftxjDZFQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=12.20"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
42
pi/files/agent/extensions/pi-web-access/package.json
Normal file
42
pi/files/agent/extensions/pi-web-access/package.json
Normal file
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"name": "pi-web-access",
|
||||
"version": "0.10.2",
|
||||
"description": "Web search, URL fetching, GitHub repo cloning, PDF extraction, YouTube video understanding, and local video analysis for Pi coding agent",
|
||||
"type": "module",
|
||||
"keywords": [
|
||||
"pi-package",
|
||||
"pi",
|
||||
"pi-coding-agent",
|
||||
"extension",
|
||||
"web-search",
|
||||
"perplexity",
|
||||
"fetch",
|
||||
"scraping"
|
||||
],
|
||||
"author": "Nico Bailon",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/nicobailon/pi-web-access.git"
|
||||
},
|
||||
"bugs": {
|
||||
"url": "https://github.com/nicobailon/pi-web-access/issues"
|
||||
},
|
||||
"homepage": "https://github.com/nicobailon/pi-web-access#readme",
|
||||
"dependencies": {
|
||||
"@mozilla/readability": "^0.5.0",
|
||||
"linkedom": "^0.16.0",
|
||||
"p-limit": "^6.1.0",
|
||||
"turndown": "^7.2.0",
|
||||
"unpdf": "^1.4.0"
|
||||
},
|
||||
"pi": {
|
||||
"extensions": [
|
||||
"./index.ts"
|
||||
],
|
||||
"skills": [
|
||||
"./skills"
|
||||
],
|
||||
"video": "https://github.com/nicobailon/pi-web-access/raw/refs/heads/main/pi-web-fetch-demo.mp4"
|
||||
}
|
||||
}
|
||||
184
pi/files/agent/extensions/pi-web-access/pdf-extract.ts
Normal file
184
pi/files/agent/extensions/pi-web-access/pdf-extract.ts
Normal file
@@ -0,0 +1,184 @@
|
||||
/**
|
||||
* PDF Content Extractor
|
||||
*
|
||||
* Extracts text from PDF files and saves to markdown.
|
||||
* Uses unpdf (pdfjs-dist wrapper) for text extraction.
|
||||
*/
|
||||
|
||||
import { getDocumentProxy } from "unpdf";
|
||||
import { writeFile, mkdir } from "node:fs/promises";
|
||||
import { join, basename } from "node:path";
|
||||
import { homedir } from "node:os";
|
||||
|
||||
export interface PDFExtractResult {
|
||||
title: string;
|
||||
pages: number;
|
||||
chars: number;
|
||||
outputPath: string;
|
||||
}
|
||||
|
||||
export interface PDFExtractOptions {
|
||||
maxPages?: number;
|
||||
outputDir?: string;
|
||||
filename?: string;
|
||||
}
|
||||
|
||||
const DEFAULT_MAX_PAGES = 100;
|
||||
const DEFAULT_OUTPUT_DIR = join(homedir(), "Downloads");
|
||||
|
||||
/**
|
||||
* Extract text from a PDF buffer and save to markdown file
|
||||
*/
|
||||
export async function extractPDFToMarkdown(
|
||||
buffer: ArrayBuffer,
|
||||
url: string,
|
||||
options: PDFExtractOptions = {}
|
||||
): Promise<PDFExtractResult> {
|
||||
const {
|
||||
maxPages = DEFAULT_MAX_PAGES,
|
||||
outputDir = DEFAULT_OUTPUT_DIR,
|
||||
filename
|
||||
} = options;
|
||||
|
||||
const pdf = await getDocumentProxy(new Uint8Array(buffer));
|
||||
const metadata = await pdf.getMetadata();
|
||||
|
||||
// Extract title from metadata or URL
|
||||
const metaTitle = metadata.info?.Title as string | undefined;
|
||||
const urlTitle = extractTitleFromURL(url);
|
||||
const title = metaTitle?.trim() || urlTitle;
|
||||
|
||||
// Determine pages to extract
|
||||
const pagesToExtract = Math.min(pdf.numPages, maxPages);
|
||||
const truncated = pdf.numPages > maxPages;
|
||||
|
||||
// Extract text page by page for better structure
|
||||
const pages: { pageNum: number; text: string }[] = [];
|
||||
for (let i = 1; i <= pagesToExtract; i++) {
|
||||
const page = await pdf.getPage(i);
|
||||
const textContent = await page.getTextContent();
|
||||
const pageText = textContent.items
|
||||
.map((item: unknown) => {
|
||||
const textItem = item as { str?: string };
|
||||
return textItem.str || "";
|
||||
})
|
||||
.join(" ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
|
||||
if (pageText) {
|
||||
pages.push({ pageNum: i, text: pageText });
|
||||
}
|
||||
}
|
||||
|
||||
// Build markdown content
|
||||
const lines: string[] = [];
|
||||
|
||||
// Header with metadata
|
||||
lines.push(`# ${title}`);
|
||||
lines.push("");
|
||||
lines.push(`> Source: ${url}`);
|
||||
lines.push(`> Pages: ${pdf.numPages}${truncated ? ` (extracted first ${pagesToExtract})` : ""}`);
|
||||
if (metadata.info?.Author) {
|
||||
lines.push(`> Author: ${metadata.info.Author}`);
|
||||
}
|
||||
lines.push("");
|
||||
lines.push("---");
|
||||
lines.push("");
|
||||
|
||||
// Content with page markers
|
||||
for (let i = 0; i < pages.length; i++) {
|
||||
if (i > 0) {
|
||||
lines.push("");
|
||||
lines.push(`<!-- Page ${pages[i].pageNum} -->`);
|
||||
lines.push("");
|
||||
}
|
||||
lines.push(pages[i].text);
|
||||
}
|
||||
|
||||
if (truncated) {
|
||||
lines.push("");
|
||||
lines.push("---");
|
||||
lines.push("");
|
||||
lines.push(`*[Truncated: Only first ${pagesToExtract} of ${pdf.numPages} pages extracted]*`);
|
||||
}
|
||||
|
||||
const content = lines.join("\n");
|
||||
|
||||
// Generate output filename
|
||||
const outputFilename = filename || sanitizeFilename(title) + ".md";
|
||||
const outputPath = join(outputDir, outputFilename);
|
||||
|
||||
// Ensure output directory exists
|
||||
await mkdir(outputDir, { recursive: true });
|
||||
|
||||
// Write file
|
||||
await writeFile(outputPath, content, "utf-8");
|
||||
|
||||
return {
|
||||
title,
|
||||
pages: pdf.numPages,
|
||||
chars: content.length,
|
||||
outputPath,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract a reasonable title from URL
|
||||
*/
|
||||
function extractTitleFromURL(url: string): string {
|
||||
try {
|
||||
const urlObj = new URL(url);
|
||||
const pathname = urlObj.pathname;
|
||||
|
||||
// Get filename without extension
|
||||
let filename = basename(pathname, ".pdf");
|
||||
|
||||
// Handle arxiv URLs: /pdf/1706.03762 → "arxiv-1706.03762"
|
||||
if (urlObj.hostname.includes("arxiv.org")) {
|
||||
const match = pathname.match(/\/(?:pdf|abs)\/(\d+\.\d+)/);
|
||||
if (match) {
|
||||
filename = `arxiv-${match[1]}`;
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up filename
|
||||
filename = filename
|
||||
.replace(/[_-]+/g, " ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
|
||||
return filename || "document";
|
||||
} catch {
|
||||
return "document";
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sanitize string for use as filename
|
||||
*/
|
||||
function sanitizeFilename(name: string): string {
|
||||
return name
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9\s-]/g, "")
|
||||
.replace(/\s+/g, "-")
|
||||
.replace(/-+/g, "-")
|
||||
.slice(0, 100)
|
||||
.replace(/^-|-$/g, "")
|
||||
|| "document";
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if URL or content-type indicates a PDF
|
||||
*/
|
||||
export function isPDF(url: string, contentType?: string): boolean {
|
||||
if (contentType?.includes("application/pdf")) {
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
const urlObj = new URL(url);
|
||||
return urlObj.pathname.toLowerCase().endsWith(".pdf");
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
186
pi/files/agent/extensions/pi-web-access/perplexity.ts
Normal file
186
pi/files/agent/extensions/pi-web-access/perplexity.ts
Normal file
@@ -0,0 +1,186 @@
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { activityMonitor } from "./activity.js";
|
||||
|
||||
const PERPLEXITY_API_URL = "https://api.perplexity.ai/chat/completions";
|
||||
const CONFIG_PATH = join(homedir(), ".pi", "web-search.json");
|
||||
|
||||
const RATE_LIMIT = {
|
||||
maxRequests: 10,
|
||||
windowMs: 60 * 1000,
|
||||
};
|
||||
|
||||
const requestTimestamps: number[] = [];
|
||||
|
||||
export interface SearchResult {
|
||||
title: string;
|
||||
url: string;
|
||||
snippet: string;
|
||||
}
|
||||
|
||||
export interface SearchResponse {
|
||||
answer: string;
|
||||
results: SearchResult[];
|
||||
}
|
||||
|
||||
export interface SearchOptions {
|
||||
numResults?: number;
|
||||
recencyFilter?: "day" | "week" | "month" | "year";
|
||||
domainFilter?: string[];
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
interface WebSearchConfig {
|
||||
perplexityApiKey?: string;
|
||||
}
|
||||
|
||||
let cachedConfig: WebSearchConfig | null = null;
|
||||
|
||||
function loadConfig(): WebSearchConfig {
|
||||
if (cachedConfig) return cachedConfig;
|
||||
|
||||
if (existsSync(CONFIG_PATH)) {
|
||||
try {
|
||||
const content = readFileSync(CONFIG_PATH, "utf-8");
|
||||
cachedConfig = JSON.parse(content) as WebSearchConfig;
|
||||
return cachedConfig;
|
||||
} catch {
|
||||
cachedConfig = {};
|
||||
}
|
||||
} else {
|
||||
cachedConfig = {};
|
||||
}
|
||||
return cachedConfig;
|
||||
}
|
||||
|
||||
function getApiKey(): string {
|
||||
const config = loadConfig();
|
||||
const key = process.env.PERPLEXITY_API_KEY || config.perplexityApiKey;
|
||||
if (!key) {
|
||||
throw new Error(
|
||||
"Perplexity API key not found. Either:\n" +
|
||||
` 1. Create ${CONFIG_PATH} with { "perplexityApiKey": "your-key" }\n` +
|
||||
" 2. Set PERPLEXITY_API_KEY environment variable\n" +
|
||||
"Get a key at https://perplexity.ai/settings/api"
|
||||
);
|
||||
}
|
||||
return key;
|
||||
}
|
||||
|
||||
function checkRateLimit(): void {
|
||||
const now = Date.now();
|
||||
const windowStart = now - RATE_LIMIT.windowMs;
|
||||
|
||||
while (requestTimestamps.length > 0 && requestTimestamps[0] < windowStart) {
|
||||
requestTimestamps.shift();
|
||||
}
|
||||
|
||||
if (requestTimestamps.length >= RATE_LIMIT.maxRequests) {
|
||||
const waitMs = requestTimestamps[0] + RATE_LIMIT.windowMs - now;
|
||||
throw new Error(`Rate limited. Try again in ${Math.ceil(waitMs / 1000)}s`);
|
||||
}
|
||||
|
||||
requestTimestamps.push(now);
|
||||
}
|
||||
|
||||
function validateDomainFilter(domains: string[]): string[] {
|
||||
return domains.filter((d) => {
|
||||
const domain = d.startsWith("-") ? d.slice(1) : d;
|
||||
return /^[a-zA-Z0-9][a-zA-Z0-9-_.]*\.[a-zA-Z]{2,}$/.test(domain);
|
||||
});
|
||||
}
|
||||
|
||||
export function isPerplexityAvailable(): boolean {
|
||||
const config = loadConfig();
|
||||
return Boolean(process.env.PERPLEXITY_API_KEY || config.perplexityApiKey);
|
||||
}
|
||||
|
||||
export async function searchWithPerplexity(query: string, options: SearchOptions = {}): Promise<SearchResponse> {
|
||||
checkRateLimit();
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "api", query });
|
||||
|
||||
activityMonitor.updateRateLimit({
|
||||
used: requestTimestamps.length,
|
||||
max: RATE_LIMIT.maxRequests,
|
||||
oldestTimestamp: requestTimestamps[0] ?? null,
|
||||
windowMs: RATE_LIMIT.windowMs,
|
||||
});
|
||||
|
||||
const apiKey = getApiKey();
|
||||
const numResults = Math.min(options.numResults ?? 5, 20);
|
||||
|
||||
const requestBody: Record<string, unknown> = {
|
||||
model: "sonar",
|
||||
messages: [{ role: "user", content: query }],
|
||||
max_tokens: 1024,
|
||||
return_related_questions: false,
|
||||
};
|
||||
|
||||
if (options.recencyFilter) {
|
||||
requestBody.search_recency_filter = options.recencyFilter;
|
||||
}
|
||||
|
||||
if (options.domainFilter && options.domainFilter.length > 0) {
|
||||
const validated = validateDomainFilter(options.domainFilter);
|
||||
if (validated.length > 0) {
|
||||
requestBody.search_domain_filter = validated;
|
||||
}
|
||||
}
|
||||
|
||||
let response: Response;
|
||||
try {
|
||||
response = await fetch(PERPLEXITY_API_URL, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
Authorization: `Bearer ${apiKey}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(requestBody),
|
||||
signal: options.signal,
|
||||
});
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
if (message.toLowerCase().includes("abort")) {
|
||||
activityMonitor.logComplete(activityId, 0);
|
||||
} else {
|
||||
activityMonitor.logError(activityId, message);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
const errorText = await response.text();
|
||||
throw new Error(`Perplexity API error ${response.status}: ${errorText}`);
|
||||
}
|
||||
|
||||
let data: Record<string, unknown>;
|
||||
try {
|
||||
data = await response.json();
|
||||
} catch {
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
throw new Error("Perplexity API returned invalid JSON");
|
||||
}
|
||||
|
||||
const answer = (data.choices as Array<{ message?: { content?: string } }>)?.[0]?.message?.content || "";
|
||||
const citations = Array.isArray(data.citations) ? data.citations : [];
|
||||
|
||||
const results: SearchResult[] = [];
|
||||
for (let i = 0; i < Math.min(citations.length, numResults); i++) {
|
||||
const citation = citations[i];
|
||||
if (typeof citation === "string") {
|
||||
results.push({ title: `Source ${i + 1}`, url: citation, snippet: "" });
|
||||
} else if (citation && typeof citation === "object" && typeof citation.url === "string") {
|
||||
results.push({
|
||||
title: citation.title || `Source ${i + 1}`,
|
||||
url: citation.url,
|
||||
snippet: "",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
activityMonitor.logComplete(activityId, response.status);
|
||||
return { answer, results };
|
||||
}
|
||||
BIN
pi/files/agent/extensions/pi-web-access/pi-web-fetch-demo.mp4
Normal file
BIN
pi/files/agent/extensions/pi-web-access/pi-web-fetch-demo.mp4
Normal file
Binary file not shown.
338
pi/files/agent/extensions/pi-web-access/rsc-extract.ts
Normal file
338
pi/files/agent/extensions/pi-web-access/rsc-extract.ts
Normal file
@@ -0,0 +1,338 @@
|
||||
/**
|
||||
* RSC Content Extractor
|
||||
*
|
||||
* Extracts readable content from Next.js React Server Components (RSC) flight payloads.
|
||||
* RSC pages embed content as JSON in <script>self.__next_f.push([...])</script> tags.
|
||||
*/
|
||||
|
||||
export interface RSCExtractResult {
|
||||
title: string;
|
||||
content: string;
|
||||
}
|
||||
|
||||
export function extractRSCContent(html: string): RSCExtractResult | null {
|
||||
if (!html.includes("self.__next_f.push")) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Parse all RSC chunks into a map
|
||||
const chunkMap = new Map<string, string>();
|
||||
const scriptRegex = /<script>self\.__next_f\.push\(\[1,"([\s\S]*?)"\]\)<\/script>/g;
|
||||
|
||||
for (const match of html.matchAll(scriptRegex)) {
|
||||
let content: string;
|
||||
try {
|
||||
content = JSON.parse('"' + match[1] + '"');
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Parse each line as "id:payload"
|
||||
// Lines are separated by \n, each line is one chunk
|
||||
// Chunk IDs are hex strings, typically 1-4 chars (supports up to 65535 chunks)
|
||||
for (const line of content.split("\n")) {
|
||||
if (!line.trim()) continue;
|
||||
|
||||
const colonIdx = line.indexOf(":");
|
||||
if (colonIdx <= 0 || colonIdx > 4) continue;
|
||||
|
||||
const id = line.slice(0, colonIdx);
|
||||
if (!/^[0-9a-f]+$/i.test(id)) continue;
|
||||
|
||||
const payload = line.slice(colonIdx + 1);
|
||||
if (!payload) continue;
|
||||
|
||||
const existing = chunkMap.get(id);
|
||||
if (!existing || payload.length > existing.length) {
|
||||
chunkMap.set(id, payload);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (chunkMap.size === 0) return null;
|
||||
|
||||
// Extract title
|
||||
const titleMatch = html.match(/<title[^>]*>([^<]+)<\/title>/);
|
||||
const title = titleMatch?.[1]?.split("|")[0]?.trim() || "";
|
||||
|
||||
// Parse and cache parsed chunks
|
||||
const parsedCache = new Map<string, unknown>();
|
||||
|
||||
function getParsedChunk(id: string): unknown | null {
|
||||
if (parsedCache.has(id)) return parsedCache.get(id);
|
||||
|
||||
const chunk = chunkMap.get(id);
|
||||
if (!chunk || !chunk.startsWith("[")) {
|
||||
parsedCache.set(id, null);
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(chunk);
|
||||
parsedCache.set(id, parsed);
|
||||
return parsed;
|
||||
} catch {
|
||||
parsedCache.set(id, null);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// Extract markdown from nodes, resolving refs on the fly
|
||||
type Node = unknown;
|
||||
const visitedRefs = new Set<string>();
|
||||
|
||||
function extractNode(node: Node, ctx = { inTable: false, inCode: false }): string {
|
||||
if (node === null || node === undefined) return "";
|
||||
|
||||
if (typeof node === "string") {
|
||||
// Check if it's a reference like "$L30"
|
||||
const refMatch = node.match(/^\$L([0-9a-f]+)$/i);
|
||||
if (refMatch) {
|
||||
const refId = refMatch[1];
|
||||
if (visitedRefs.has(refId)) return ""; // Prevent cycles
|
||||
visitedRefs.add(refId);
|
||||
const refNode = getParsedChunk(refId);
|
||||
const result = refNode ? extractNode(refNode, ctx) : "";
|
||||
visitedRefs.delete(refId);
|
||||
return result;
|
||||
}
|
||||
// Filter out RSC-specific artifacts, but preserve content inside code blocks
|
||||
if (!ctx.inCode && (node === "$undefined" || node === "$" || /^\$[A-Z]/.test(node))) return "";
|
||||
return node.trim() ? node : "";
|
||||
}
|
||||
|
||||
if (typeof node === "number") return String(node);
|
||||
if (typeof node === "boolean") return "";
|
||||
if (!Array.isArray(node)) return "";
|
||||
|
||||
// RSC element: ["$", "tag", key, props]
|
||||
if (node[0] === "$" && typeof node[1] === "string") {
|
||||
const tag = node[1] as string;
|
||||
const props = (node[3] || {}) as Record<string, unknown>;
|
||||
|
||||
// Skip non-content
|
||||
const skipTags = ["script", "style", "svg", "path", "circle", "link", "meta",
|
||||
"template", "button", "input", "nav", "footer", "aside"];
|
||||
if (skipTags.includes(tag)) return "";
|
||||
|
||||
// Component ref like $L25
|
||||
if (tag.startsWith("$L")) {
|
||||
const refId = tag.slice(2);
|
||||
if (visitedRefs.has(refId)) return "";
|
||||
|
||||
// Check for heading components with baseId
|
||||
if (props.baseId && props.children) {
|
||||
return `## ${String(props.children)}\n\n`;
|
||||
}
|
||||
|
||||
visitedRefs.add(refId);
|
||||
const refNode = getParsedChunk(refId);
|
||||
let result = "";
|
||||
if (refNode) {
|
||||
result = extractNode(refNode, ctx);
|
||||
} else if (props.children) {
|
||||
result = extractNode(props.children as Node, ctx);
|
||||
}
|
||||
visitedRefs.delete(refId);
|
||||
return result;
|
||||
}
|
||||
|
||||
const children = props.children;
|
||||
const content = children ? extractNode(children as Node, ctx) : "";
|
||||
|
||||
switch (tag) {
|
||||
case "h1": return `# ${content.trim()}\n\n`;
|
||||
case "h2": return `## ${content.trim()}\n\n`;
|
||||
case "h3": return `### ${content.trim()}\n\n`;
|
||||
case "h4": return `#### ${content.trim()}\n\n`;
|
||||
case "h5": return `##### ${content.trim()}\n\n`;
|
||||
case "h6": return `###### ${content.trim()}\n\n`;
|
||||
case "p": return ctx.inTable ? content : `${content.trim()}\n\n`;
|
||||
case "code": {
|
||||
const codeContent = children ? extractNode(children as Node, { ...ctx, inCode: true }) : "";
|
||||
return ctx.inCode ? codeContent : `\`${codeContent}\``;
|
||||
}
|
||||
case "pre": {
|
||||
const preContent = children ? extractNode(children as Node, { ...ctx, inCode: true }) : "";
|
||||
return "```\n" + preContent + "\n```\n\n";
|
||||
}
|
||||
case "strong": case "b": return `**${content}**`;
|
||||
case "em": case "i": return `*${content}*`;
|
||||
case "li": return `- ${content.trim()}\n`;
|
||||
case "ul": case "ol": return content + "\n";
|
||||
case "blockquote": return `> ${content.trim()}\n\n`;
|
||||
case "table": return extractTable(node as unknown[]) + "\n";
|
||||
case "thead": case "tbody": case "tr": case "th": case "td":
|
||||
return content;
|
||||
case "div":
|
||||
if (props.role === "alert" || props["data-slot"] === "alert") {
|
||||
return `> ${content.trim()}\n\n`;
|
||||
}
|
||||
return content;
|
||||
case "a": {
|
||||
const href = props.href as string | undefined;
|
||||
return href && !href.startsWith("#") ? `[${content}](${href})` : content;
|
||||
}
|
||||
default: return content;
|
||||
}
|
||||
}
|
||||
|
||||
// Array of child nodes
|
||||
return (node as Node[]).map(n => extractNode(n, ctx)).join("");
|
||||
}
|
||||
|
||||
function extractTable(tableNode: unknown[]): string {
|
||||
const props = (tableNode[3] || {}) as Record<string, unknown>;
|
||||
const rows: string[][] = [];
|
||||
let headerRowCount = 0;
|
||||
|
||||
function walkTable(node: unknown, isHeader = false): void {
|
||||
if (node === null || node === undefined) return;
|
||||
|
||||
// Handle string refs
|
||||
if (typeof node === "string") {
|
||||
const refMatch = node.match(/^\$L([0-9a-f]+)$/i);
|
||||
if (refMatch && !visitedRefs.has(refMatch[1])) {
|
||||
visitedRefs.add(refMatch[1]);
|
||||
const refNode = getParsedChunk(refMatch[1]);
|
||||
if (refNode) walkTable(refNode, isHeader);
|
||||
visitedRefs.delete(refMatch[1]);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (!Array.isArray(node)) return;
|
||||
|
||||
if (node[0] === "$") {
|
||||
const tag = node[1] as string;
|
||||
const nodeProps = (node[3] || {}) as Record<string, unknown>;
|
||||
|
||||
// Handle component refs
|
||||
if (tag.startsWith("$L")) {
|
||||
const refId = tag.slice(2);
|
||||
if (!visitedRefs.has(refId)) {
|
||||
visitedRefs.add(refId);
|
||||
const refNode = getParsedChunk(refId);
|
||||
if (refNode) walkTable(refNode, isHeader);
|
||||
visitedRefs.delete(refId);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (tag === "thead") walkTable(nodeProps.children, true);
|
||||
else if (tag === "tbody") walkTable(nodeProps.children, false);
|
||||
else if (tag === "tr") {
|
||||
const cells: string[] = [];
|
||||
walkCells(nodeProps.children, cells);
|
||||
if (cells.length > 0) {
|
||||
rows.push(cells);
|
||||
if (isHeader) headerRowCount++;
|
||||
}
|
||||
} else walkTable(nodeProps.children, isHeader);
|
||||
} else {
|
||||
for (const child of node) walkTable(child, isHeader);
|
||||
}
|
||||
}
|
||||
|
||||
function walkCells(node: unknown, cells: string[]): void {
|
||||
if (node === null || node === undefined) return;
|
||||
|
||||
// Handle string refs
|
||||
if (typeof node === "string") {
|
||||
const refMatch = node.match(/^\$L([0-9a-f]+)$/i);
|
||||
if (refMatch && !visitedRefs.has(refMatch[1])) {
|
||||
visitedRefs.add(refMatch[1]);
|
||||
const refNode = getParsedChunk(refMatch[1]);
|
||||
if (refNode) walkCells(refNode, cells);
|
||||
visitedRefs.delete(refMatch[1]);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (!Array.isArray(node)) return;
|
||||
|
||||
if (node[0] === "$" && (node[1] === "td" || node[1] === "th")) {
|
||||
const cellProps = (node[3] || {}) as Record<string, unknown>;
|
||||
const text = extractNode(cellProps.children, { inTable: true, inCode: false })
|
||||
.trim()
|
||||
.replace(/\n/g, " ")
|
||||
.replace(/\\/g, "\\\\") // Escape backslashes first
|
||||
.replace(/\|/g, "\\|"); // Then escape pipes
|
||||
cells.push(text);
|
||||
} else if (node[0] === "$" && typeof node[1] === "string" && (node[1] as string).startsWith("$L")) {
|
||||
// Component ref for a cell
|
||||
const refId = (node[1] as string).slice(2);
|
||||
if (!visitedRefs.has(refId)) {
|
||||
visitedRefs.add(refId);
|
||||
const refNode = getParsedChunk(refId);
|
||||
if (refNode) walkCells(refNode, cells);
|
||||
visitedRefs.delete(refId);
|
||||
}
|
||||
} else {
|
||||
for (const child of node) walkCells(child, cells);
|
||||
}
|
||||
}
|
||||
|
||||
walkTable(props.children);
|
||||
if (rows.length === 0) return "";
|
||||
|
||||
const colCount = Math.max(...rows.map(r => r.length));
|
||||
let md = "";
|
||||
for (let i = 0; i < rows.length; i++) {
|
||||
const row = rows[i].concat(Array(colCount - rows[i].length).fill(""));
|
||||
md += "| " + row.join(" | ") + " |\n";
|
||||
if (i === headerRowCount - 1 || (headerRowCount === 0 && i === 0)) {
|
||||
md += "| " + Array(colCount).fill("---").join(" | ") + " |\n";
|
||||
}
|
||||
}
|
||||
return md;
|
||||
}
|
||||
|
||||
// Process main content chunk (usually "23")
|
||||
const mainChunk = getParsedChunk("23");
|
||||
|
||||
if (mainChunk) {
|
||||
const content = extractNode(mainChunk);
|
||||
if (content.trim().length > 100) {
|
||||
const cleaned = content
|
||||
.replace(/\n{3,}/g, "\n\n")
|
||||
.trim();
|
||||
return { title, content: cleaned };
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: try other chunks
|
||||
const contentParts: { order: number; text: string }[] = [];
|
||||
|
||||
for (const [id] of chunkMap) {
|
||||
if (id === "23") continue;
|
||||
const parsed = getParsedChunk(id);
|
||||
if (!parsed) continue;
|
||||
|
||||
visitedRefs.clear();
|
||||
const text = extractNode(parsed);
|
||||
|
||||
if (text.trim().length > 50 &&
|
||||
!text.includes("page was not found") &&
|
||||
!text.includes("404")) {
|
||||
contentParts.push({ order: parseInt(id, 16), text: text.trim() });
|
||||
}
|
||||
}
|
||||
|
||||
if (contentParts.length === 0) return null;
|
||||
|
||||
contentParts.sort((a, b) => a.order - b.order);
|
||||
|
||||
const seen = new Set<string>();
|
||||
const uniqueParts: string[] = [];
|
||||
for (const part of contentParts) {
|
||||
const key = part.text.slice(0, 150);
|
||||
if (!seen.has(key)) {
|
||||
seen.add(key);
|
||||
uniqueParts.push(part.text);
|
||||
}
|
||||
}
|
||||
|
||||
const content = uniqueParts.join("\n\n").replace(/\n{3,}/g, "\n\n").trim();
|
||||
return content.length > 100 ? { title, content } : null;
|
||||
}
|
||||
354
pi/files/agent/extensions/pi-web-access/search-filter.ts
Normal file
354
pi/files/agent/extensions/pi-web-access/search-filter.ts
Normal file
@@ -0,0 +1,354 @@
|
||||
import { complete, type Context } from "@mariozechner/pi-ai";
|
||||
import type { ExtensionContext } from "@mariozechner/pi-coding-agent";
|
||||
import type { SearchResult } from "./perplexity.js";
|
||||
import type { QueryResultData } from "./storage.js";
|
||||
|
||||
const DEFAULT_PROMPT = `You are a research assistant. You receive search results from multiple queries and produce a concise, deduplicated briefing.
|
||||
|
||||
Rules:
|
||||
1. Skip any query results that are clearly irrelevant or off-topic
|
||||
2. Organize by topic, not by original query
|
||||
3. Remove redundant information — if two results cover the same point, include it once with both sources cited
|
||||
4. Preserve specific facts, numbers, code examples, and recommendations
|
||||
5. Cite sources inline: "reduced by 18-29% [domain.com]"
|
||||
6. End with a full source list (domain + title + URL)
|
||||
7. Be thorough but never redundant`;
|
||||
|
||||
const DEFAULT_MODEL = "anthropic/claude-haiku-4-5";
|
||||
const TIMEOUT_MS = 15000;
|
||||
const MAX_TOKENS = 2048;
|
||||
const SKIP_THRESHOLD_TOKENS = 500;
|
||||
const REDUNDANT_SIMILARITY_THRESHOLD = 0.7;
|
||||
const QUALITY_TIERS: [RegExp, string][] = [
|
||||
[/\.(gov|edu)$/, "institutional"],
|
||||
[/^(docs\.|developer\.)/, "official-docs"],
|
||||
[/^github\.com/, "code"],
|
||||
[/^(stackoverflow|stackexchange)\.com/, "forum"],
|
||||
[/^arxiv\.org/, "paper"],
|
||||
[/^news\.ycombinator\.com|^reddit\.com/, "discussion"],
|
||||
[/^medium\.com|^dev\.to$|^substack\.com/, "blog-platform"],
|
||||
];
|
||||
const QUALITY_LABELS: Record<string, string> = {
|
||||
institutional: "institutional sources",
|
||||
"official-docs": "official docs",
|
||||
code: "code repositories",
|
||||
forum: "forum sources",
|
||||
paper: "papers",
|
||||
discussion: "discussion sources",
|
||||
"blog-platform": "blog-platform sources",
|
||||
other: "other sources",
|
||||
};
|
||||
const CITATION_RE = /\[([a-z0-9.-]+\.[a-z]{2,}(?:,\s*[a-z0-9.-]+\.[a-z]{2,})*)\](?!\()/gi;
|
||||
|
||||
export interface CondenseConfig {
|
||||
model: string;
|
||||
prompt: string;
|
||||
}
|
||||
|
||||
export interface PreprocessedData {
|
||||
overlapPairs: Array<{ q1: number; q2: number; shared: number; pct: number }>;
|
||||
similarityPairs: Array<{ q1: number; q2: number; similarity: number }>;
|
||||
totalTokens: number;
|
||||
skipCondensation: boolean;
|
||||
qualitySummary: string;
|
||||
hints: string;
|
||||
}
|
||||
|
||||
export function resolveCondenseConfig(
|
||||
raw: boolean | { enabled?: boolean; model?: string; prompt?: string } | undefined,
|
||||
): CondenseConfig | null {
|
||||
if (raw === false) return null;
|
||||
if (raw === true || raw === undefined || raw === null) {
|
||||
return { model: DEFAULT_MODEL, prompt: DEFAULT_PROMPT };
|
||||
}
|
||||
if (raw.enabled === false) return null;
|
||||
return {
|
||||
model: raw.model || DEFAULT_MODEL,
|
||||
prompt: raw.prompt || DEFAULT_PROMPT,
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeUrl(url: string): string {
|
||||
try {
|
||||
const u = new URL(url);
|
||||
const host = u.hostname.replace(/^www\./, "").toLowerCase();
|
||||
const path = u.pathname.replace(/\/$/, "");
|
||||
return `${host}${path}`;
|
||||
} catch {
|
||||
return url.trim().toLowerCase()
|
||||
.replace(/^https?:\/\//, "")
|
||||
.replace(/^www\./, "")
|
||||
.replace(/[?#].*$/, "")
|
||||
.replace(/\/$/, "");
|
||||
}
|
||||
}
|
||||
|
||||
function computeOverlapPairs(
|
||||
results: Map<number, QueryResultData>,
|
||||
): Array<{ q1: number; q2: number; shared: number; pct: number }> {
|
||||
const urlToQueries = new Map<string, Set<number>>();
|
||||
const querySourceCounts = new Map<number, number>();
|
||||
|
||||
for (const [qi, data] of results) {
|
||||
querySourceCounts.set(qi, data.results.length);
|
||||
for (const r of data.results) {
|
||||
const key = normalizeUrl(r.url);
|
||||
const set = urlToQueries.get(key) ?? new Set<number>();
|
||||
set.add(qi);
|
||||
urlToQueries.set(key, set);
|
||||
}
|
||||
}
|
||||
|
||||
const pairShared = new Map<string, number>();
|
||||
const pairKey = (a: number, b: number) => `${Math.min(a, b)}-${Math.max(a, b)}`;
|
||||
|
||||
for (const qis of urlToQueries.values()) {
|
||||
if (qis.size < 2) continue;
|
||||
const arr = [...qis];
|
||||
for (let i = 0; i < arr.length; i++) {
|
||||
for (let j = i + 1; j < arr.length; j++) {
|
||||
const key = pairKey(arr[i], arr[j]);
|
||||
pairShared.set(key, (pairShared.get(key) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const pairs: Array<{ q1: number; q2: number; shared: number; pct: number }> = [];
|
||||
for (const [key, shared] of pairShared) {
|
||||
const [a, b] = key.split("-").map(Number);
|
||||
const minSources = Math.max(1, Math.min(querySourceCounts.get(a) ?? 0, querySourceCounts.get(b) ?? 0));
|
||||
pairs.push({ q1: a, q2: b, shared, pct: Math.round((shared / minSources) * 100) });
|
||||
}
|
||||
|
||||
return pairs;
|
||||
}
|
||||
|
||||
function answerSimilarity(a: string, b: string): number {
|
||||
const words = (s: string) =>
|
||||
new Set(
|
||||
s.toLowerCase()
|
||||
.split(/\s+/)
|
||||
.filter(w => w.length > 3),
|
||||
);
|
||||
|
||||
const setA = words(a);
|
||||
const setB = words(b);
|
||||
if (setA.size === 0 || setB.size === 0) return 0;
|
||||
let intersection = 0;
|
||||
for (const word of setA) {
|
||||
if (setB.has(word)) intersection++;
|
||||
}
|
||||
return intersection / Math.max(setA.size, setB.size);
|
||||
}
|
||||
|
||||
function estimateTokens(text: string): number {
|
||||
return Math.ceil(text.length / 4);
|
||||
}
|
||||
|
||||
function extractDomain(url: string): string {
|
||||
try {
|
||||
return new URL(url).hostname.replace(/^www\./, "").toLowerCase();
|
||||
} catch {
|
||||
return normalizeUrl(url).split("/")[0] || normalizeUrl(url);
|
||||
}
|
||||
}
|
||||
|
||||
function qualityTierForDomain(domain: string): string {
|
||||
for (const [re, tier] of QUALITY_TIERS) {
|
||||
if (re.test(domain)) return tier;
|
||||
}
|
||||
return "other";
|
||||
}
|
||||
|
||||
export function preprocessSearchResults(
|
||||
results: Map<number, QueryResultData>,
|
||||
): PreprocessedData {
|
||||
const overlapPairs = computeOverlapPairs(results)
|
||||
.sort((a, b) => b.pct - a.pct || b.shared - a.shared);
|
||||
|
||||
const entries = [...results.entries()].sort((a, b) => a[0] - b[0]);
|
||||
const similarityPairs: Array<{ q1: number; q2: number; similarity: number }> = [];
|
||||
for (let i = 0; i < entries.length; i++) {
|
||||
for (let j = i + 1; j < entries.length; j++) {
|
||||
const [q1, r1] = entries[i];
|
||||
const [q2, r2] = entries[j];
|
||||
similarityPairs.push({
|
||||
q1,
|
||||
q2,
|
||||
similarity: answerSimilarity(r1.answer || "", r2.answer || ""),
|
||||
});
|
||||
}
|
||||
}
|
||||
similarityPairs.sort((a, b) => b.similarity - a.similarity);
|
||||
|
||||
let totalTokens = 0;
|
||||
const qualityCounts = new Map<string, number>();
|
||||
for (const data of results.values()) {
|
||||
totalTokens += estimateTokens(data.answer || "");
|
||||
for (const source of data.results) {
|
||||
const tier = qualityTierForDomain(extractDomain(source.url));
|
||||
qualityCounts.set(tier, (qualityCounts.get(tier) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
const skipCondensation = totalTokens < SKIP_THRESHOLD_TOKENS;
|
||||
const qualitySummary = [...qualityCounts.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.map(([tier, count]) => `${count} ${QUALITY_LABELS[tier] ?? tier}`)
|
||||
.join(", ") || "no sources";
|
||||
|
||||
const overlapLines = overlapPairs.map(p =>
|
||||
`- Q${p.q1} and Q${p.q2} share ${p.shared} sources (${p.pct}%)`,
|
||||
);
|
||||
const similarityLines = similarityPairs
|
||||
.filter(p => p.similarity >= REDUNDANT_SIMILARITY_THRESHOLD)
|
||||
.map(p => `- Q${p.q1} and Q${p.q2} answers are ${Math.round(p.similarity * 100)}% similar by word overlap`);
|
||||
|
||||
const hints = [
|
||||
"Overlap analysis:",
|
||||
...(overlapLines.length > 0 ? overlapLines : ["- No source overlap detected across queries."]),
|
||||
"",
|
||||
"Answer similarity:",
|
||||
...(similarityLines.length > 0
|
||||
? similarityLines
|
||||
: [`- No answer pairs meet the ${Math.round(REDUNDANT_SIMILARITY_THRESHOLD * 100)}% similarity threshold.`]),
|
||||
"",
|
||||
`Source quality: ${qualitySummary}.`,
|
||||
`Estimated answer tokens: ${totalTokens}. ${skipCondensation ? "Below threshold — skip condensation." : "Condensation recommended."}`,
|
||||
].join("\n");
|
||||
|
||||
return {
|
||||
overlapPairs,
|
||||
similarityPairs,
|
||||
totalTokens,
|
||||
skipCondensation,
|
||||
qualitySummary,
|
||||
hints,
|
||||
};
|
||||
}
|
||||
|
||||
export async function condenseSearchResults(
|
||||
results: Map<number, QueryResultData>,
|
||||
config: CondenseConfig,
|
||||
ctx: ExtensionContext | undefined,
|
||||
signal?: AbortSignal,
|
||||
taskContext?: string,
|
||||
preprocessed?: PreprocessedData,
|
||||
): Promise<string | null> {
|
||||
try {
|
||||
if (results.size < 2 || !ctx) return null;
|
||||
|
||||
const slashIndex = config.model.indexOf("/");
|
||||
if (slashIndex === -1) return null;
|
||||
const provider = config.model.slice(0, slashIndex);
|
||||
const modelId = config.model.slice(slashIndex + 1);
|
||||
|
||||
const model = ctx.modelRegistry.find(provider, modelId);
|
||||
if (!model) return null;
|
||||
|
||||
const apiKey = await ctx.modelRegistry.getApiKey(model);
|
||||
if (!apiKey) return null;
|
||||
|
||||
const queryData = [...results.entries()]
|
||||
.sort((a, b) => a[0] - b[0])
|
||||
.map(([qi, r]) => {
|
||||
const sources = r.results.map((s, si) => {
|
||||
const domain = extractDomain(s.url);
|
||||
const tier = qualityTierForDomain(domain);
|
||||
return `${si + 1}. ${s.title}\n ${s.url}\n quality: ${tier}`;
|
||||
}).join("\n");
|
||||
return `[${qi}] Query: "${r.query}"\n` +
|
||||
(r.error ? `Error: ${r.error}\n` : "") +
|
||||
`Answer:\n${r.answer || "(empty)"}\n` +
|
||||
`Sources:\n${sources || "(none)"}`;
|
||||
}).join("\n\n");
|
||||
|
||||
let prompt = config.prompt;
|
||||
if (taskContext) prompt += `\n\nUser's task: ${taskContext}`;
|
||||
if (preprocessed?.hints) prompt += `\n\n${preprocessed.hints}`;
|
||||
prompt += `\n\nSearch result data:\n${queryData}`;
|
||||
|
||||
const aiContext: Context = {
|
||||
messages: [{
|
||||
role: "user",
|
||||
content: [{ type: "text", text: prompt }],
|
||||
timestamp: Date.now(),
|
||||
}],
|
||||
};
|
||||
|
||||
const timeoutSignal = AbortSignal.timeout(TIMEOUT_MS);
|
||||
const combinedSignal = signal
|
||||
? AbortSignal.any([signal, timeoutSignal])
|
||||
: timeoutSignal;
|
||||
|
||||
const response = await complete(model, aiContext, {
|
||||
apiKey,
|
||||
signal: combinedSignal,
|
||||
max_tokens: MAX_TOKENS,
|
||||
} as any);
|
||||
const text = response.content.find(c => c.type === "text")?.text?.trim();
|
||||
if (!text) return null;
|
||||
return text;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function verifyCitations(condensed: string, sources: SearchResult[]): string {
|
||||
const knownDomains = new Set(sources.map(s => extractDomain(s.url)));
|
||||
return condensed.replace(CITATION_RE, (_match, inner: string) => {
|
||||
const domains = inner.split(/,\s*/).map(d => d.trim().toLowerCase());
|
||||
const verified = domains.map(d => {
|
||||
if (knownDomains.has(d)) return d;
|
||||
const closest = [...knownDomains].find(k => k.includes(d) || d.includes(k));
|
||||
return closest ?? d;
|
||||
});
|
||||
return `[${verified.join(", ")}]`;
|
||||
});
|
||||
}
|
||||
|
||||
function collectCitedDomains(text: string): string[] {
|
||||
const found: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
const re = new RegExp(CITATION_RE.source, "gi");
|
||||
for (const match of text.matchAll(re)) {
|
||||
const domains = match[1].split(/,\s*/).map(d => d.trim().toLowerCase());
|
||||
for (const domain of domains) {
|
||||
if (seen.has(domain)) continue;
|
||||
seen.add(domain);
|
||||
found.push(domain);
|
||||
}
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
function sourceLineForDomain(domain: string, sources: SearchResult[]): string {
|
||||
const source = sources.find(s => extractDomain(s.url) === domain)
|
||||
?? sources.find(s => {
|
||||
const d = extractDomain(s.url);
|
||||
return d.includes(domain) || domain.includes(d);
|
||||
});
|
||||
if (!source) return `- ${domain}`;
|
||||
const title = source.title?.trim() || source.url;
|
||||
return `- ${domain} — ${title} (${source.url})`;
|
||||
}
|
||||
|
||||
function completeSourceList(condensed: string, sources: SearchResult[]): string {
|
||||
const withoutSources = condensed.replace(/\n#{2,3}\s+Sources[\s\S]*$/i, "").trimEnd();
|
||||
const citedDomains = collectCitedDomains(withoutSources);
|
||||
if (citedDomains.length === 0) return withoutSources;
|
||||
const sourceLines = citedDomains.map(domain => sourceLineForDomain(domain, sources));
|
||||
return `${withoutSources}\n\n## Sources\n${sourceLines.join("\n")}`;
|
||||
}
|
||||
|
||||
export function postProcessCondensed(
|
||||
condensed: string,
|
||||
sources: SearchResult[],
|
||||
): string {
|
||||
const verified = verifyCitations(condensed, sources);
|
||||
const completed = completeSourceList(verified, sources);
|
||||
const outputTokens = estimateTokens(completed);
|
||||
if (outputTokens > 4000) {
|
||||
console.warn(`[pi-web-access] Condensed output length exceeded expected threshold: ~${outputTokens} tokens`);
|
||||
}
|
||||
return completed.trim();
|
||||
}
|
||||
@@ -0,0 +1,195 @@
|
||||
---
|
||||
name: librarian
|
||||
description: Research open-source libraries with evidence-backed answers and GitHub permalinks. Use when the user asks about library internals, needs implementation details with source code references, wants to understand why something was changed, or needs authoritative answers backed by actual code. Excels at navigating large open-source repos and providing citations to exact lines of code.
|
||||
---
|
||||
|
||||
# Librarian
|
||||
|
||||
Answer questions about open-source libraries by finding evidence with GitHub permalinks. Every claim backed by actual code.
|
||||
|
||||
## Execution Model
|
||||
|
||||
Pi executes tool calls sequentially, even when you emit multiple calls in one turn. But batching independent calls in a single turn still saves LLM round-trips (~5-10s each). Use these patterns:
|
||||
|
||||
| Pattern | When | Actually parallel? |
|
||||
|---------|------|-------------------|
|
||||
| Batch tool calls in one turn | Independent ops (web_search + fetch_content + read) | No, but saves round-trips |
|
||||
| `fetch_content({ urls: [...] })` | Multiple URLs to fetch | Yes (3 concurrent) |
|
||||
| Bash with `&` + `wait` | Multiple git/gh commands | Yes (OS-level) |
|
||||
|
||||
## Step 1: Classify the Request
|
||||
|
||||
Before doing anything, classify the request to pick the right research strategy.
|
||||
|
||||
| Type | Trigger | Primary Approach |
|
||||
|------|---------|-----------------|
|
||||
| **Conceptual** | "How do I use X?", "Best practice for Y?" | web_search + fetch_content (README/docs) |
|
||||
| **Implementation** | "How does X implement Y?", "Show me the source" | fetch_content (clone) + code search |
|
||||
| **Context/History** | "Why was this changed?", "History of X?" | git log + git blame + issue/PR search |
|
||||
| **Comprehensive** | Complex or ambiguous requests, "deep dive" | All of the above |
|
||||
|
||||
## Step 2: Research by Type
|
||||
|
||||
### Conceptual Questions
|
||||
|
||||
Batch these in one turn:
|
||||
|
||||
1. **web_search**: `"library-name topic"` via Perplexity for recent articles and discussions
|
||||
2. **fetch_content**: the library's GitHub repo URL to clone and check README, docs, or examples
|
||||
|
||||
Synthesize web results + repo docs. Cite official documentation and link to relevant source files.
|
||||
|
||||
### Implementation Questions
|
||||
|
||||
The core workflow -- clone, find, permalink:
|
||||
|
||||
1. **fetch_content** the GitHub repo URL -- this clones it locally and returns the file tree
|
||||
2. Use **bash** to search the cloned repo: `grep -rn "function_name"`, `find . -name "*.ts"`
|
||||
3. Use **read** to examine specific files once you've located them
|
||||
4. Get the commit SHA: `cd /tmp/pi-github-repos/owner/repo && git rev-parse HEAD`
|
||||
5. Construct permalink: `https://github.com/owner/repo/blob/<sha>/path/to/file#L10-L20`
|
||||
|
||||
Batch the initial calls: fetch_content (clone) + web_search (recent discussions) in one turn. Then dig into the clone with grep/read once it's available.
|
||||
|
||||
### Context/History Questions
|
||||
|
||||
Use git operations on the cloned repo:
|
||||
|
||||
```bash
|
||||
cd /tmp/pi-github-repos/owner/repo
|
||||
|
||||
# Recent changes to a specific file
|
||||
git log --oneline -n 20 -- path/to/file.ts
|
||||
|
||||
# Who changed what and when
|
||||
git blame -L 10,30 path/to/file.ts
|
||||
|
||||
# Full diff for a specific commit
|
||||
git show <sha> -- path/to/file.ts
|
||||
|
||||
# Search commit messages
|
||||
git log --oneline --grep="keyword" -n 10
|
||||
```
|
||||
|
||||
For issues and PRs, use bash:
|
||||
|
||||
```bash
|
||||
# Search issues
|
||||
gh search issues "keyword" --repo owner/repo --state all --limit 10
|
||||
|
||||
# Search merged PRs
|
||||
gh search prs "keyword" --repo owner/repo --state merged --limit 10
|
||||
|
||||
# View specific issue/PR with comments
|
||||
gh issue view <number> --repo owner/repo --comments
|
||||
gh pr view <number> --repo owner/repo --comments
|
||||
|
||||
# Release notes
|
||||
gh api repos/owner/repo/releases --jq '.[0:5] | .[].tag_name'
|
||||
```
|
||||
|
||||
### Comprehensive Research
|
||||
|
||||
Combine everything. Batch these in one turn:
|
||||
|
||||
1. **web_search**: recent articles and discussions
|
||||
2. **fetch_content**: clone the repo (or multiple repos if comparing)
|
||||
3. **bash**: `gh search issues "keyword" --repo owner/repo --limit 10 & gh search prs "keyword" --repo owner/repo --state merged --limit 10 & wait`
|
||||
|
||||
Then dig into the clone with grep, read, git blame, git log as needed.
|
||||
|
||||
## Step 3: Construct Permalinks
|
||||
|
||||
Permalinks are the whole point. They make your answers citable and verifiable.
|
||||
|
||||
```
|
||||
https://github.com/<owner>/<repo>/blob/<commit-sha>/<filepath>#L<start>-L<end>
|
||||
```
|
||||
|
||||
Getting the SHA from a cloned repo:
|
||||
|
||||
```bash
|
||||
cd /tmp/pi-github-repos/owner/repo && git rev-parse HEAD
|
||||
```
|
||||
|
||||
Getting the SHA from a tag:
|
||||
|
||||
```bash
|
||||
gh api repos/owner/repo/git/refs/tags/v1.0.0 --jq '.object.sha'
|
||||
```
|
||||
|
||||
Always use full commit SHAs, not branch names. Branch links break when code changes. Permalinks don't.
|
||||
|
||||
## Step 4: Cite Everything
|
||||
|
||||
Every code-related claim needs a permalink. Format:
|
||||
|
||||
```markdown
|
||||
The stale time check happens in [`notifyManager.ts`](https://github.com/TanStack/query/blob/abc123/packages/query-core/src/notifyManager.ts#L42-L50):
|
||||
|
||||
\`\`\`typescript
|
||||
function isStale(query: Query, staleTime: number): boolean {
|
||||
return query.state.dataUpdatedAt + staleTime < Date.now()
|
||||
}
|
||||
\`\`\`
|
||||
```
|
||||
|
||||
For conceptual answers, link to official docs and relevant source files. For implementation answers, every function/class reference should have a permalink.
|
||||
|
||||
## Video Analysis
|
||||
|
||||
For questions about video tutorials, conference talks, or screen recordings:
|
||||
|
||||
```typescript
|
||||
// Full extraction (transcript + visual descriptions)
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc" })
|
||||
|
||||
// Ask a specific question about a video
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", prompt: "What libraries are imported in this tutorial?" })
|
||||
|
||||
// Single frame at a known moment
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", timestamp: "23:41" })
|
||||
|
||||
// Range scan for visual discovery
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", timestamp: "23:41-25:00" })
|
||||
|
||||
// Custom density across a range
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", timestamp: "23:41-25:00", frames: 3 })
|
||||
|
||||
// Whole-video sampling
|
||||
fetch_content({ url: "https://youtube.com/watch?v=abc", frames: 6 })
|
||||
|
||||
// Analyze a local recording
|
||||
fetch_content({ url: "/path/to/demo.mp4", prompt: "What error message appears on screen?" })
|
||||
|
||||
// Batch multiple videos with the same question
|
||||
fetch_content({
|
||||
urls: ["https://youtube.com/watch?v=abc", "https://youtube.com/watch?v=def"],
|
||||
prompt: "What packages are installed?"
|
||||
})
|
||||
```
|
||||
|
||||
Use single timestamps for known moments, ranges for visual scanning, and frames-alone for a quick overview of the whole video.
|
||||
|
||||
The `prompt` parameter only applies to video content (YouTube URLs and local video files). For non-video URLs, it's ignored.
|
||||
|
||||
## Failure Recovery
|
||||
|
||||
| Failure | Recovery |
|
||||
|---------|----------|
|
||||
| grep finds nothing | Broaden the query, try concept names instead of exact function names |
|
||||
| gh CLI rate limited | Use the already-cloned repo in /tmp/pi-github-repos/ for git operations |
|
||||
| Repo too large to clone | fetch_content returns an API-only view automatically; use that or add `forceClone: true` |
|
||||
| File not found in clone | Branch name with slashes may have misresolved; list the repo tree and navigate manually |
|
||||
| Uncertain about implementation | State your uncertainty explicitly, propose a hypothesis, show what evidence you did find |
|
||||
| Video extraction fails | Ensure Chrome is signed into gemini.google.com (free) or set GEMINI_API_KEY |
|
||||
| Page returns 403/bot block | Gemini fallback triggers automatically; no action needed if Gemini is configured |
|
||||
| web_search fails | Check provider config; try explicit `provider: "gemini"` if Perplexity key is missing |
|
||||
|
||||
## Guidelines
|
||||
|
||||
- Vary search queries when running multiple searches -- different angles, not the same pattern repeated
|
||||
- Prefer recent sources; filter out outdated results when they conflict with newer information
|
||||
- For version-specific questions, clone the tagged version: `fetch_content("https://github.com/owner/repo/tree/v1.0.0")`
|
||||
- When the repo is already cloned from a previous fetch_content call, reuse it -- check the path before cloning again
|
||||
- Answer directly. Skip preamble like "I'll help you with..." -- go straight to findings
|
||||
71
pi/files/agent/extensions/pi-web-access/storage.ts
Normal file
71
pi/files/agent/extensions/pi-web-access/storage.ts
Normal file
@@ -0,0 +1,71 @@
|
||||
import type { ExtensionContext } from "@mariozechner/pi-coding-agent";
|
||||
import type { ExtractedContent } from "./extract.js";
|
||||
import type { SearchResult } from "./perplexity.js";
|
||||
|
||||
const CACHE_TTL_MS = 60 * 60 * 1000;
|
||||
|
||||
export interface QueryResultData {
|
||||
query: string;
|
||||
answer: string;
|
||||
results: SearchResult[];
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
export interface StoredSearchData {
|
||||
id: string;
|
||||
type: "search" | "fetch";
|
||||
timestamp: number;
|
||||
queries?: QueryResultData[];
|
||||
urls?: ExtractedContent[];
|
||||
}
|
||||
|
||||
const storedResults = new Map<string, StoredSearchData>();
|
||||
|
||||
export function generateId(): string {
|
||||
return Date.now().toString(36) + Math.random().toString(36).slice(2, 8);
|
||||
}
|
||||
|
||||
export function storeResult(id: string, data: StoredSearchData): void {
|
||||
storedResults.set(id, data);
|
||||
}
|
||||
|
||||
export function getResult(id: string): StoredSearchData | null {
|
||||
return storedResults.get(id) ?? null;
|
||||
}
|
||||
|
||||
export function getAllResults(): StoredSearchData[] {
|
||||
return Array.from(storedResults.values());
|
||||
}
|
||||
|
||||
export function deleteResult(id: string): boolean {
|
||||
return storedResults.delete(id);
|
||||
}
|
||||
|
||||
export function clearResults(): void {
|
||||
storedResults.clear();
|
||||
}
|
||||
|
||||
function isValidStoredData(data: unknown): data is StoredSearchData {
|
||||
if (!data || typeof data !== "object") return false;
|
||||
const d = data as Record<string, unknown>;
|
||||
if (typeof d.id !== "string" || !d.id) return false;
|
||||
if (d.type !== "search" && d.type !== "fetch") return false;
|
||||
if (typeof d.timestamp !== "number") return false;
|
||||
if (d.type === "search" && !Array.isArray(d.queries)) return false;
|
||||
if (d.type === "fetch" && !Array.isArray(d.urls)) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
export function restoreFromSession(ctx: ExtensionContext): void {
|
||||
storedResults.clear();
|
||||
const now = Date.now();
|
||||
|
||||
for (const entry of ctx.sessionManager.getBranch()) {
|
||||
if (entry.type === "custom" && entry.customType === "web-search-results") {
|
||||
const data = entry.data;
|
||||
if (isValidStoredData(data) && now - data.timestamp < CACHE_TTL_MS) {
|
||||
storedResults.set(data.id, data);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
44
pi/files/agent/extensions/pi-web-access/utils.ts
Normal file
44
pi/files/agent/extensions/pi-web-access/utils.ts
Normal file
@@ -0,0 +1,44 @@
|
||||
export function formatSeconds(s: number): string {
|
||||
const h = Math.floor(s / 3600);
|
||||
const m = Math.floor((s % 3600) / 60);
|
||||
const sec = s % 60;
|
||||
if (h > 0) return `${h}:${String(m).padStart(2, "0")}:${String(sec).padStart(2, "0")}`;
|
||||
return `${m}:${String(sec).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
export function readExecError(err: unknown): { code?: string; stderr: string; message: string } {
|
||||
if (!err || typeof err !== "object") {
|
||||
return { stderr: "", message: String(err) };
|
||||
}
|
||||
const code = (err as { code?: string }).code;
|
||||
const message = (err as { message?: string }).message ?? "";
|
||||
const stderrRaw = (err as { stderr?: Buffer | string }).stderr;
|
||||
const stderr = Buffer.isBuffer(stderrRaw)
|
||||
? stderrRaw.toString("utf-8")
|
||||
: typeof stderrRaw === "string"
|
||||
? stderrRaw
|
||||
: "";
|
||||
return { code, stderr, message };
|
||||
}
|
||||
|
||||
export function isTimeoutError(err: unknown): boolean {
|
||||
if (!err || typeof err !== "object") return false;
|
||||
if ((err as { killed?: boolean }).killed) return true;
|
||||
const name = (err as { name?: string }).name;
|
||||
const code = (err as { code?: string }).code;
|
||||
const message = (err as { message?: string }).message ?? "";
|
||||
return name === "AbortError" || code === "ETIMEDOUT" || message.toLowerCase().includes("timed out");
|
||||
}
|
||||
|
||||
export function trimErrorText(text: string): string {
|
||||
return text.replace(/\s+/g, " ").trim().slice(0, 200);
|
||||
}
|
||||
|
||||
export function mapFfmpegError(err: unknown): string {
|
||||
const { code, stderr, message } = readExecError(err);
|
||||
if (code === "ENOENT") return "ffmpeg is not installed. Install with: brew install ffmpeg";
|
||||
if (isTimeoutError(err)) return "ffmpeg timed out extracting frame";
|
||||
if (stderr.includes("403")) return "Stream URL returned 403 — may have expired, try again";
|
||||
const snippet = trimErrorText(stderr || message);
|
||||
return snippet ? `ffmpeg failed: ${snippet}` : "ffmpeg failed";
|
||||
}
|
||||
330
pi/files/agent/extensions/pi-web-access/video-extract.ts
Normal file
330
pi/files/agent/extensions/pi-web-access/video-extract.ts
Normal file
@@ -0,0 +1,330 @@
|
||||
import { existsSync, readFileSync, readdirSync, statSync } from "node:fs";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { resolve, extname, basename, join, dirname } from "node:path";
|
||||
import { homedir } from "node:os";
|
||||
import { activityMonitor } from "./activity.js";
|
||||
import { isGeminiWebAvailable, queryWithCookies } from "./gemini-web.js";
|
||||
import { queryGeminiApiWithVideo, getApiKey, API_BASE } from "./gemini-api.js";
|
||||
import { extractHeadingTitle, type ExtractedContent, type ExtractOptions, type FrameResult } from "./extract.js";
|
||||
import { readExecError, trimErrorText, mapFfmpegError } from "./utils.js";
|
||||
|
||||
const CONFIG_PATH = join(homedir(), ".pi", "web-search.json");
|
||||
const UPLOAD_BASE = "https://generativelanguage.googleapis.com/upload/v1beta";
|
||||
|
||||
const DEFAULT_VIDEO_PROMPT = `Extract the complete content of this video. Include:
|
||||
1. Video title (infer from content if not explicit), duration
|
||||
2. A brief summary (2-3 sentences)
|
||||
3. Full transcript with timestamps
|
||||
4. Descriptions of any code, terminal commands, diagrams, slides, or UI shown on screen
|
||||
|
||||
Format as markdown.`;
|
||||
|
||||
const VIDEO_EXTENSIONS: Record<string, string> = {
|
||||
".mp4": "video/mp4",
|
||||
".mov": "video/quicktime",
|
||||
".webm": "video/webm",
|
||||
".avi": "video/x-msvideo",
|
||||
".mpeg": "video/mpeg",
|
||||
".mpg": "video/mpeg",
|
||||
".wmv": "video/x-ms-wmv",
|
||||
".flv": "video/x-flv",
|
||||
".3gp": "video/3gpp",
|
||||
".3gpp": "video/3gpp",
|
||||
};
|
||||
|
||||
interface VideoFileInfo {
|
||||
absolutePath: string;
|
||||
mimeType: string;
|
||||
sizeBytes: number;
|
||||
}
|
||||
|
||||
interface VideoConfig {
|
||||
enabled: boolean;
|
||||
preferredModel: string;
|
||||
maxSizeMB: number;
|
||||
}
|
||||
|
||||
const VIDEO_CONFIG_DEFAULTS: VideoConfig = {
|
||||
enabled: true,
|
||||
preferredModel: "gemini-3-flash-preview",
|
||||
maxSizeMB: 50,
|
||||
};
|
||||
|
||||
let cachedVideoConfig: VideoConfig | null = null;
|
||||
|
||||
function loadVideoConfig(): VideoConfig {
|
||||
if (cachedVideoConfig) return cachedVideoConfig;
|
||||
try {
|
||||
if (existsSync(CONFIG_PATH)) {
|
||||
const raw = JSON.parse(readFileSync(CONFIG_PATH, "utf-8"));
|
||||
const v = raw.video ?? {};
|
||||
cachedVideoConfig = {
|
||||
enabled: v.enabled ?? VIDEO_CONFIG_DEFAULTS.enabled,
|
||||
preferredModel: v.preferredModel ?? VIDEO_CONFIG_DEFAULTS.preferredModel,
|
||||
maxSizeMB: v.maxSizeMB ?? VIDEO_CONFIG_DEFAULTS.maxSizeMB,
|
||||
};
|
||||
return cachedVideoConfig;
|
||||
}
|
||||
} catch {}
|
||||
cachedVideoConfig = { ...VIDEO_CONFIG_DEFAULTS };
|
||||
return cachedVideoConfig;
|
||||
}
|
||||
|
||||
export function isVideoFile(input: string): VideoFileInfo | null {
|
||||
const config = loadVideoConfig();
|
||||
if (!config.enabled) return null;
|
||||
|
||||
const isFilePath = input.startsWith("/") || input.startsWith("./") || input.startsWith("../") || input.startsWith("file://");
|
||||
if (!isFilePath) return null;
|
||||
|
||||
const filePath = input.startsWith("file://") ? new URL(input).pathname : input;
|
||||
|
||||
const ext = extname(filePath).toLowerCase();
|
||||
const mimeType = VIDEO_EXTENSIONS[ext];
|
||||
if (!mimeType) return null;
|
||||
|
||||
const absolutePath = resolveFilePath(filePath);
|
||||
if (!absolutePath) return null;
|
||||
|
||||
const stat = statSync(absolutePath);
|
||||
if (!stat.isFile()) return null;
|
||||
|
||||
const maxBytes = config.maxSizeMB * 1024 * 1024;
|
||||
if (stat.size > maxBytes) return null;
|
||||
|
||||
return { absolutePath, mimeType, sizeBytes: stat.size };
|
||||
}
|
||||
|
||||
function resolveFilePath(filePath: string): string | null {
|
||||
const absolutePath = resolve(filePath);
|
||||
if (existsSync(absolutePath)) return absolutePath;
|
||||
|
||||
const dir = dirname(absolutePath);
|
||||
const base = basename(absolutePath);
|
||||
if (!existsSync(dir)) return null;
|
||||
|
||||
try {
|
||||
const normalizedBase = normalizeSpaces(base);
|
||||
const match = readdirSync(dir).find(f => normalizeSpaces(f) === normalizedBase);
|
||||
return match ? join(dir, match) : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeSpaces(s: string): string {
|
||||
return s.replace(/[\u00A0\u2000-\u200B\u202F\u205F\u3000\uFEFF]/g, " ");
|
||||
}
|
||||
|
||||
export async function extractVideo(
|
||||
info: VideoFileInfo,
|
||||
signal?: AbortSignal,
|
||||
options?: ExtractOptions,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const config = loadVideoConfig();
|
||||
const effectivePrompt = options?.prompt ?? DEFAULT_VIDEO_PROMPT;
|
||||
const effectiveModel = options?.model ?? config.preferredModel;
|
||||
const displayName = basename(info.absolutePath);
|
||||
const activityId = activityMonitor.logStart({ type: "fetch", url: `video:${displayName}` });
|
||||
|
||||
const result = await tryVideoGeminiApi(info, effectivePrompt, effectiveModel, signal)
|
||||
?? await tryVideoGeminiWeb(info, effectivePrompt, effectiveModel, signal);
|
||||
|
||||
if (result) {
|
||||
const thumbnail = await extractVideoFrame(info.absolutePath);
|
||||
if (!("error" in thumbnail)) {
|
||||
result.thumbnail = thumbnail;
|
||||
}
|
||||
activityMonitor.logComplete(activityId, 200);
|
||||
return result;
|
||||
}
|
||||
|
||||
activityMonitor.logError(activityId, "all video extraction paths failed");
|
||||
return null;
|
||||
}
|
||||
|
||||
function mapFfprobeError(err: unknown): string {
|
||||
const { code, stderr, message } = readExecError(err);
|
||||
if (code === "ENOENT") return "ffprobe is not installed. Install ffmpeg which includes ffprobe";
|
||||
const snippet = trimErrorText(stderr || message);
|
||||
return snippet ? `ffprobe failed: ${snippet}` : "ffprobe failed";
|
||||
}
|
||||
|
||||
export async function extractVideoFrame(filePath: string, seconds: number = 1): Promise<FrameResult> {
|
||||
try {
|
||||
const { execFileSync } = await import("node:child_process");
|
||||
const buffer = execFileSync("ffmpeg", [
|
||||
"-ss", String(seconds), "-i", filePath,
|
||||
"-frames:v", "1", "-f", "image2pipe", "-vcodec", "mjpeg", "pipe:1",
|
||||
], { maxBuffer: 5 * 1024 * 1024, timeout: 10000, stdio: ["pipe", "pipe", "pipe"] });
|
||||
if (buffer.length === 0) return { error: "ffmpeg failed: empty output" };
|
||||
return { data: buffer.toString("base64"), mimeType: "image/jpeg" };
|
||||
} catch (err) {
|
||||
return { error: mapFfmpegError(err) };
|
||||
}
|
||||
}
|
||||
|
||||
export async function getLocalVideoDuration(filePath: string): Promise<number | { error: string }> {
|
||||
try {
|
||||
const { execFileSync } = await import("node:child_process");
|
||||
const output = execFileSync("ffprobe", [
|
||||
"-v", "quiet",
|
||||
"-show_entries", "format=duration",
|
||||
"-of", "csv=p=0",
|
||||
filePath,
|
||||
], { timeout: 10000, encoding: "utf-8", stdio: ["pipe", "pipe", "pipe"] }).trim();
|
||||
const duration = Number.parseFloat(output);
|
||||
if (!Number.isFinite(duration)) return { error: "ffprobe failed: invalid duration output" };
|
||||
return duration;
|
||||
} catch (err) {
|
||||
return { error: mapFfprobeError(err) };
|
||||
}
|
||||
}
|
||||
|
||||
async function tryVideoGeminiWeb(
|
||||
info: VideoFileInfo,
|
||||
prompt: string,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
try {
|
||||
const cookies = await isGeminiWebAvailable();
|
||||
if (!cookies) return null;
|
||||
if (signal?.aborted) return null;
|
||||
|
||||
const text = await queryWithCookies(prompt, cookies, {
|
||||
files: [info.absolutePath],
|
||||
model,
|
||||
signal,
|
||||
timeoutMs: 180000,
|
||||
});
|
||||
|
||||
return {
|
||||
url: info.absolutePath,
|
||||
title: extractVideoTitle(text, info.absolutePath),
|
||||
content: text,
|
||||
error: null,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function tryVideoGeminiApi(
|
||||
info: VideoFileInfo,
|
||||
prompt: string,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const apiKey = getApiKey();
|
||||
if (!apiKey) return null;
|
||||
if (signal?.aborted) return null;
|
||||
|
||||
let fileName: string | null = null;
|
||||
try {
|
||||
const uploaded = await uploadToFilesApi(info, apiKey, signal);
|
||||
fileName = uploaded.name;
|
||||
|
||||
await pollFileState(fileName, apiKey, signal, 120000);
|
||||
|
||||
const text = await queryGeminiApiWithVideo(prompt, uploaded.uri, {
|
||||
model,
|
||||
mimeType: info.mimeType,
|
||||
signal,
|
||||
timeoutMs: 120000,
|
||||
});
|
||||
|
||||
return {
|
||||
url: info.absolutePath,
|
||||
title: extractVideoTitle(text, info.absolutePath),
|
||||
content: text,
|
||||
error: null,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
} finally {
|
||||
if (fileName) deleteGeminiFile(fileName, apiKey);
|
||||
}
|
||||
}
|
||||
|
||||
async function uploadToFilesApi(
|
||||
info: VideoFileInfo,
|
||||
apiKey: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ name: string; uri: string }> {
|
||||
const displayName = basename(info.absolutePath);
|
||||
|
||||
const initRes = await fetch(`${UPLOAD_BASE}/files`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"x-goog-api-key": apiKey,
|
||||
"X-Goog-Upload-Protocol": "resumable",
|
||||
"X-Goog-Upload-Command": "start",
|
||||
"X-Goog-Upload-Header-Content-Length": String(info.sizeBytes),
|
||||
"X-Goog-Upload-Header-Content-Type": info.mimeType,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({ file: { display_name: displayName } }),
|
||||
signal,
|
||||
});
|
||||
|
||||
if (!initRes.ok) {
|
||||
const text = await initRes.text();
|
||||
throw new Error(`File upload init failed: ${initRes.status} (${text.slice(0, 200)})`);
|
||||
}
|
||||
|
||||
const uploadUrl = initRes.headers.get("x-goog-upload-url");
|
||||
if (!uploadUrl) throw new Error("No upload URL in response headers");
|
||||
|
||||
const fileData = await readFile(info.absolutePath);
|
||||
const uploadRes = await fetch(uploadUrl, {
|
||||
method: "PUT",
|
||||
headers: {
|
||||
"Content-Length": String(info.sizeBytes),
|
||||
"X-Goog-Upload-Offset": "0",
|
||||
"X-Goog-Upload-Command": "upload, finalize",
|
||||
},
|
||||
body: fileData,
|
||||
signal,
|
||||
});
|
||||
|
||||
if (!uploadRes.ok) {
|
||||
const text = await uploadRes.text();
|
||||
throw new Error(`File upload failed: ${uploadRes.status} (${text.slice(0, 200)})`);
|
||||
}
|
||||
|
||||
const result = await uploadRes.json() as { file: { name: string; uri: string } };
|
||||
return result.file;
|
||||
}
|
||||
|
||||
async function pollFileState(
|
||||
fileName: string,
|
||||
apiKey: string,
|
||||
signal?: AbortSignal,
|
||||
timeoutMs: number = 120000,
|
||||
): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
|
||||
while (Date.now() < deadline) {
|
||||
if (signal?.aborted) throw new Error("Aborted");
|
||||
|
||||
const res = await fetch(`${API_BASE}/${fileName}?key=${apiKey}`, { signal });
|
||||
if (!res.ok) throw new Error(`File state check failed: ${res.status}`);
|
||||
|
||||
const data = await res.json() as { state: string };
|
||||
if (data.state === "ACTIVE") return;
|
||||
if (data.state === "FAILED") throw new Error("File processing failed");
|
||||
|
||||
await new Promise(r => setTimeout(r, 5000));
|
||||
}
|
||||
|
||||
throw new Error("File processing timed out");
|
||||
}
|
||||
|
||||
function deleteGeminiFile(fileName: string, apiKey: string): void {
|
||||
fetch(`${API_BASE}/${fileName}?key=${apiKey}`, { method: "DELETE" }).catch(() => {});
|
||||
}
|
||||
|
||||
function extractVideoTitle(text: string, filePath: string): string {
|
||||
return extractHeadingTitle(text) ?? basename(filePath, extname(filePath));
|
||||
}
|
||||
282
pi/files/agent/extensions/pi-web-access/youtube-extract.ts
Normal file
282
pi/files/agent/extensions/pi-web-access/youtube-extract.ts
Normal file
@@ -0,0 +1,282 @@
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { homedir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { activityMonitor } from "./activity.js";
|
||||
import { isGeminiWebAvailable, queryWithCookies } from "./gemini-web.js";
|
||||
import { isGeminiApiAvailable, queryGeminiApiWithVideo } from "./gemini-api.js";
|
||||
import { searchWithPerplexity } from "./perplexity.js";
|
||||
import { extractHeadingTitle, type ExtractedContent, type FrameResult, type VideoFrame } from "./extract.js";
|
||||
import { formatSeconds, readExecError, isTimeoutError, trimErrorText, mapFfmpegError } from "./utils.js";
|
||||
|
||||
const CONFIG_PATH = join(homedir(), ".pi", "web-search.json");
|
||||
|
||||
const YOUTUBE_PROMPT = `Extract the complete content of this YouTube video. Include:
|
||||
1. Video title, channel name, and duration
|
||||
2. A brief summary (2-3 sentences)
|
||||
3. Full transcript with timestamps
|
||||
4. Descriptions of any code, terminal commands, diagrams, slides, or UI shown on screen
|
||||
|
||||
Format as markdown.`;
|
||||
|
||||
const YOUTUBE_REGEX =
|
||||
/(?:(?:www\.|m\.)?youtube\.com\/(?:watch\?.*v=|shorts\/|live\/|embed\/|v\/)|youtu\.be\/)([a-zA-Z0-9_-]{11})/;
|
||||
|
||||
interface YouTubeConfig {
|
||||
enabled: boolean;
|
||||
preferredModel: string;
|
||||
}
|
||||
|
||||
const defaults: YouTubeConfig = { enabled: true, preferredModel: "gemini-3-flash-preview" };
|
||||
let cachedConfig: YouTubeConfig | null = null;
|
||||
|
||||
function loadYouTubeConfig(): YouTubeConfig {
|
||||
if (cachedConfig) return cachedConfig;
|
||||
try {
|
||||
if (existsSync(CONFIG_PATH)) {
|
||||
const raw = JSON.parse(readFileSync(CONFIG_PATH, "utf-8"));
|
||||
const yt = raw.youtube ?? {};
|
||||
cachedConfig = {
|
||||
enabled: yt.enabled ?? defaults.enabled,
|
||||
preferredModel: yt.preferredModel ?? defaults.preferredModel,
|
||||
};
|
||||
return cachedConfig;
|
||||
}
|
||||
} catch {}
|
||||
cachedConfig = { ...defaults };
|
||||
return cachedConfig;
|
||||
}
|
||||
|
||||
export function isYouTubeURL(url: string): { isYouTube: boolean; videoId: string | null } {
|
||||
try {
|
||||
const parsed = new URL(url);
|
||||
if (parsed.pathname === "/playlist") {
|
||||
return { isYouTube: false, videoId: null };
|
||||
}
|
||||
} catch {
|
||||
return { isYouTube: false, videoId: null };
|
||||
}
|
||||
|
||||
const match = url.match(YOUTUBE_REGEX);
|
||||
if (!match) return { isYouTube: false, videoId: null };
|
||||
return { isYouTube: true, videoId: match[1] };
|
||||
}
|
||||
|
||||
export function isYouTubeEnabled(): boolean {
|
||||
return loadYouTubeConfig().enabled;
|
||||
}
|
||||
|
||||
export async function extractYouTube(
|
||||
url: string,
|
||||
signal?: AbortSignal,
|
||||
prompt?: string,
|
||||
model?: string,
|
||||
): Promise<ExtractedContent | null> {
|
||||
const config = loadYouTubeConfig();
|
||||
const { videoId } = isYouTubeURL(url);
|
||||
const canonicalUrl = videoId
|
||||
? `https://www.youtube.com/watch?v=${videoId}`
|
||||
: url;
|
||||
const effectivePrompt = prompt ?? YOUTUBE_PROMPT;
|
||||
const effectiveModel = model ?? config.preferredModel;
|
||||
|
||||
const activityId = activityMonitor.logStart({ type: "fetch", url: `youtube.com/${videoId ?? "video"}` });
|
||||
|
||||
const result = await tryGeminiWeb(canonicalUrl, effectivePrompt, effectiveModel, signal)
|
||||
?? await tryGeminiApi(canonicalUrl, effectivePrompt, effectiveModel, signal)
|
||||
?? await tryPerplexity(url, effectivePrompt, signal);
|
||||
|
||||
if (result) {
|
||||
result.url = url;
|
||||
if (videoId) {
|
||||
const thumb = await fetchYouTubeThumbnail(videoId);
|
||||
if (thumb) result.thumbnail = thumb;
|
||||
}
|
||||
activityMonitor.logComplete(activityId, 200);
|
||||
return result;
|
||||
}
|
||||
|
||||
activityMonitor.logError(activityId, "all extraction paths failed");
|
||||
return null;
|
||||
}
|
||||
|
||||
type StreamInfo = { streamUrl: string; duration: number | null };
|
||||
type StreamResult = StreamInfo | { error: string };
|
||||
|
||||
function mapYtDlpError(err: unknown): string {
|
||||
const { code, stderr, message } = readExecError(err);
|
||||
if (code === "ENOENT") return "yt-dlp is not installed. Install with: brew install yt-dlp";
|
||||
if (isTimeoutError(err)) return "yt-dlp timed out fetching video info";
|
||||
const lower = stderr.toLowerCase();
|
||||
if (lower.includes("private")) return "Video is private or unavailable";
|
||||
if (lower.includes("sign in")) return "Video is age-restricted and requires authentication";
|
||||
if (lower.includes("not available")) return "Video is unavailable in your region or has been removed";
|
||||
if (lower.includes("live")) return "Cannot extract frames from a live stream";
|
||||
const snippet = trimErrorText(stderr || message);
|
||||
return snippet ? `yt-dlp failed: ${snippet}` : "yt-dlp failed";
|
||||
}
|
||||
|
||||
export async function getYouTubeStreamInfo(videoId: string): Promise<StreamResult> {
|
||||
try {
|
||||
const { execFileSync } = await import("node:child_process");
|
||||
const output = execFileSync("yt-dlp", [
|
||||
"--print", "duration",
|
||||
"-g", `https://www.youtube.com/watch?v=${videoId}`,
|
||||
], { timeout: 15000, encoding: "utf-8", stdio: ["pipe", "pipe", "pipe"] }).trim();
|
||||
const lines = output.split(/\r?\n/);
|
||||
const rawDuration = lines[0]?.trim();
|
||||
const streamUrl = lines[1]?.trim();
|
||||
if (!streamUrl) return { error: "yt-dlp failed: missing stream URL" };
|
||||
const parsedDuration = rawDuration && rawDuration !== "NA" ? Number.parseFloat(rawDuration) : NaN;
|
||||
const duration = Number.isFinite(parsedDuration) ? parsedDuration : null;
|
||||
return { streamUrl, duration };
|
||||
} catch (err) {
|
||||
return { error: mapYtDlpError(err) };
|
||||
}
|
||||
}
|
||||
|
||||
async function extractFrameFromStream(streamUrl: string, seconds: number): Promise<FrameResult> {
|
||||
try {
|
||||
const { execFileSync } = await import("node:child_process");
|
||||
const buffer = execFileSync("ffmpeg", [
|
||||
"-ss", String(seconds), "-i", streamUrl,
|
||||
"-frames:v", "1", "-f", "image2pipe", "-vcodec", "mjpeg", "pipe:1",
|
||||
], { maxBuffer: 5 * 1024 * 1024, timeout: 30000, stdio: ["pipe", "pipe", "pipe"] });
|
||||
if (buffer.length === 0) return { error: "ffmpeg failed: empty output" };
|
||||
return { data: buffer.toString("base64"), mimeType: "image/jpeg" };
|
||||
} catch (err) {
|
||||
return { error: mapFfmpegError(err) };
|
||||
}
|
||||
}
|
||||
|
||||
export async function extractYouTubeFrame(
|
||||
videoId: string,
|
||||
seconds: number,
|
||||
streamInfo?: StreamInfo,
|
||||
): Promise<FrameResult> {
|
||||
const info = streamInfo ?? await getYouTubeStreamInfo(videoId);
|
||||
if ("error" in info) return info;
|
||||
return extractFrameFromStream(info.streamUrl, seconds);
|
||||
}
|
||||
|
||||
export async function extractYouTubeFrames(
|
||||
videoId: string,
|
||||
timestamps: number[],
|
||||
streamInfo?: StreamInfo,
|
||||
): Promise<{ frames: VideoFrame[]; duration: number | null; error: string | null }> {
|
||||
const info = streamInfo ?? await getYouTubeStreamInfo(videoId);
|
||||
if ("error" in info) return { frames: [], duration: null, error: info.error };
|
||||
const results = await Promise.all(timestamps.map(async (t) => {
|
||||
const frame = await extractFrameFromStream(info.streamUrl, t);
|
||||
if ("error" in frame) return { error: frame.error };
|
||||
return { ...frame, timestamp: formatSeconds(t) };
|
||||
}));
|
||||
const frames = results.filter((f): f is VideoFrame => "data" in f);
|
||||
const errorResult = results.find((f): f is { error: string } => "error" in f);
|
||||
return { frames, duration: info.duration, error: frames.length === 0 && errorResult ? errorResult.error : null };
|
||||
}
|
||||
|
||||
export async function fetchYouTubeThumbnail(videoId: string): Promise<{ data: string; mimeType: string } | null> {
|
||||
try {
|
||||
const res = await fetch(`https://img.youtube.com/vi/${videoId}/hqdefault.jpg`, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
const buffer = Buffer.from(await res.arrayBuffer());
|
||||
if (buffer.length === 0) return null;
|
||||
return { data: buffer.toString("base64"), mimeType: "image/jpeg" };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function tryGeminiWeb(
|
||||
url: string,
|
||||
prompt: string,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
try {
|
||||
const cookies = await isGeminiWebAvailable();
|
||||
if (!cookies) return null;
|
||||
|
||||
if (signal?.aborted) return null;
|
||||
|
||||
const text = await queryWithCookies(prompt, cookies, {
|
||||
youtubeUrl: url,
|
||||
model,
|
||||
signal,
|
||||
timeoutMs: 120000,
|
||||
});
|
||||
|
||||
return {
|
||||
url,
|
||||
title: extractHeadingTitle(text) ?? "YouTube Video",
|
||||
content: text,
|
||||
error: null,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function tryGeminiApi(
|
||||
url: string,
|
||||
prompt: string,
|
||||
model: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
try {
|
||||
if (!isGeminiApiAvailable()) return null;
|
||||
|
||||
if (signal?.aborted) return null;
|
||||
|
||||
const text = await queryGeminiApiWithVideo(prompt, url, {
|
||||
model,
|
||||
signal,
|
||||
timeoutMs: 120000,
|
||||
});
|
||||
|
||||
return {
|
||||
url,
|
||||
title: extractHeadingTitle(text) ?? "YouTube Video",
|
||||
content: text,
|
||||
error: null,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function tryPerplexity(
|
||||
url: string,
|
||||
prompt: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ExtractedContent | null> {
|
||||
try {
|
||||
if (signal?.aborted) return null;
|
||||
|
||||
const perplexityQuery = prompt === YOUTUBE_PROMPT
|
||||
? `Summarize this YouTube video in detail: ${url}`
|
||||
: `${prompt} YouTube video: ${url}`;
|
||||
|
||||
const { answer } = await searchWithPerplexity(
|
||||
perplexityQuery,
|
||||
{ signal },
|
||||
);
|
||||
|
||||
if (!answer) return null;
|
||||
|
||||
const content =
|
||||
`# Video Summary (via Perplexity)\n\n${answer}\n\n` +
|
||||
`*Full video understanding requires Gemini access. Set GEMINI_API_KEY or sign into Google in Chrome.*`;
|
||||
|
||||
return {
|
||||
url,
|
||||
title: "Video Summary (via Perplexity)",
|
||||
content,
|
||||
error: null,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user