Form intelligence.
engineered.

Overview

FormHarvester is available as a Windows desktop app, a Python CLI, and a library. All three interfaces share the same Selenium-based engine, campaign profiles, settings, email extraction, form handling, and local technology detector.

Install

Windows desktop app

Download the latest executable from GitHub Releases. The desktop app contains the GUI and browser engine; no configuration files need to be edited manually.

Python package

pip install formharvester

# Optional desktop GUI support
pip install "formharvester[gui]"

The base package provides the CLI and library. The gui extra adds pywebview.

Quick start

Run the desktop GUI

formharvester gui
# Optional developer-only controls
formharvester gui --dev

Run a configured campaign

formharvester profile create roofing
formharvester profile use roofing
formharvester run --headless --max-pages 3

Campaign Search queries are sent to Google. Optional URL filters are matched against the URLs returned by Google; they do not search page content or specifically identify contact pages. Leave them empty to keep all Google result links, subject to normal deduplication and the visited-site filter. This does not mean every hyperlink on Google or every link inside each website.

Inspect lead intelligence

formharvester leads list
formharvester leads metrics
formharvester leads export --output leads.csv

Discover URLs without submitting forms

formharvester discover "roofing companies austin"

LLM form content

FormHarvester can generate the Subject and Message fields with OpenAI, Anthropic, or DeepSeek. Enable Use LLM-generated content beside the Subject and Message fields in the Campaigns tab. Configure the provider, model, and API key in Settings.

When the toggle is off, Subject and Message contain the exact text that will be submitted. When it is on, the GUI shows separate Subject prompt and Message prompt fields; the direct submission text is preserved, so switching between modes is reversible. The provider returns the final values as a structured JSON object. Other campaign fields continue using the deterministic form-filling rules.

If the toggle is enabled without an API key for the selected provider, the GUI shows a warning and directs you to Settings before a run can start.

Safety gates

Providers and configuration

The base package uses the existing HTTP dependency and provider-native endpoints: OpenAI Responses, Anthropic Messages, and DeepSeek Chat Completions. No provider SDK is required. Model names are editable because provider catalogs change.

"llm": {
  "enabled": true,
  "provider": "openai",
  "model": "gpt-5",
  "openai_api_key": "YOUR_API_KEY",
  "review_before_submit": true,
  "request_timeout": 60
}

API keys are stored in the local settings file, masked in the GUI, and excluded from logs and exports. Provider usage may incur charges. Configured campaign identity fields are sent to the selected provider when generation is enabled.

Prompt context

The shared prompt builder sends the Subject and Message prompts plus structured context: target URL, detected technologies, public emails, configured identity fields, and visible form-field metadata. It does not send raw HTML, cookies, scripts, or page source.

Lead intelligence

Each discovered root domain is persisted in the local data/leads.sqlite3 database. A lead combines discovery context, contact signals, technology evidence, deterministic qualification, generated drafts, policy decisions, and submission outcomes.

Qualification

The score is explainable and uses observable signals: a contact form contributes 45 points, public email signals contribute 20, and technology evidence contributes up to 20. The Leads tab shows the score and the reasons behind it so a user can judge quality before outreach.

Selective autopilot

Autopilot is disabled by default and must be enabled per campaign. It requires LLM-generated content and can require review before submission. A submission is allowed only when the score threshold, technology include/exclude rules, contact-form requirement, domain cooldown, daily limit, and per-run limit pass.

Suppressed domains remain blocked until restored. Every submission attempt, generated draft, policy result, and suppression action is recorded in the local audit history.

CLI reporting

formharvester leads list
formharvester leads list --status QUALIFIED
formharvester leads metrics
formharvester leads export --output leads.csv

Technology detection

Every harvest runs a local, passive scan by default. It reuses the Selenium session already opened for the site, so it does not require a separate Wappalyzer extension, a hosted API, or another crawl.

What is inspected

Detected result

result = fh.harvest("https://example.com")

for technology in result.technologies:
    print(technology.name)
    print(technology.category)
    print(technology.version)
    print(technology.confidence)
    print(technology.evidence)

Each match contains a name, category, optional version, confidence score, and evidence records. Evidence values are designed for auditing; cookie values are redacted and are never exported.

Inference and confidence

Technology detection is fingerprinting, not access to a site's source repository or server. A public page can expose the frontend and infrastructure while hiding the backend behind a reverse proxy.

Live smoke-test example

A read-only smoke scan of https://mory.dev returned the following at test time:

Stacks change. This is an example of the result format, not a permanent assertion about the site.

Library API

The library API avoids settings files and returns structured Python objects.

from formharvester import FormFillDetails, FormHarvester, HarvesterOptions

with FormHarvester(
    FormFillDetails(email="jane@example.com", message="I would like a quote."),
    HarvesterOptions(headless=True, send_form=False, detect_technologies=True),
) as fh:
    result = fh.harvest("https://example.com")
    print(result.status)
    print(result.emails)
    print([technology.name for technology in result.technologies])

result.status is one of SUBMITTED, FORM_NOT_FOUND, BUTTON_NOT_FOUND, VISITED, LLM_ERROR, REVIEW_SKIPPED, POLICY_BLOCKED, NOT_QUALIFIED, DRY_RUN, or ERROR. When generation succeeds, result.generated contains the generated Subject and Message plus provider/model metadata.

Convenience functions harvest_site(), harvest_sites(), and discover_sites() are also available.

LLM generation

from formharvester import FormFillDetails, FormHarvester, HarvesterOptions

details = FormFillDetails(
    email="jane@example.com",
    subject="Ask about a website rebuild",
    message="Write a concise introduction using our frontend experience.",
)
options = HarvesterOptions(
    send_form=True,
    llm_enabled=True,
    llm_provider="openai",
    llm_model="gpt-5",
    llm_api_key="YOUR_API_KEY",
)

with FormHarvester(details, options) as fh:
    result = fh.harvest("https://example.com")
    print(result.status, result.generated)

Library callers can inject an LlmClient for tests or custom providers. Manual review is desktop-GUI-only; enabling llm_review_before_submit in a library run raises a configuration error rather than prompting interactively.

CLI and GUI

CLI

formharvester settings show
formharvester settings set engine.detect_technologies false
formharvester profile list
formharvester run --profile roofing --headless

The CLI prints a short technology summary after each processed site. Run flags override saved settings for that run. Saved LLM settings are used for normal CLI runs, but CLI execution refuses the GUI-only review mode when submission is enabled.

formharvester leads list
formharvester leads list --status QUALIFIED
formharvester leads metrics
formharvester leads export --output leads.csv

Lead reports are emitted from the local SQLite database. Use the Campaigns tab to configure score thresholds, technology rules, cooldowns, limits, dry runs, and selective autopilot. The Leads tab shows the same records with suppression and audit actions.

Desktop GUI

Open the Campaigns tab to toggle LLM-generated Subject and Message content and edit either direct text or prompts. Use Qualification & safety for campaign-level autopilot rules. Use Settings for technology detection, provider credentials, model selection, and manual review. The Run tab prints per-site progress and pauses with editable generated content when review is enabled; the Leads tab shows the persistent funnel. Run formharvester gui --dev to reveal advanced developer-only controls, including form-fill preview mode.

External API health

The top-right header contains separate health indicators for the configured LLM and CAPTCHA services. Green means the provider accepted the configured credentials and returned a valid response. Yellow means configuration is missing, the endpoint is unreachable, or the provider rejected the credentials. Gray means that integration is disabled. Checks run asynchronously, are briefly cached, and use non-generation/non-solving requests so they do not create LLM content or spend CAPTCHA credits.

Output files

Runtime data lives beside the active configuration. A campaign named roofing can produce:

Technology JSONL record

{
  "url": "https://example.com",
  "scanned_at": "2026-08-29T00:00:00+00:00",
  "technologies": [{
    "name": "Next.js",
    "category": "Web framework",
    "version": null,
    "confidence": 0.98,
    "evidence": [{
      "source": "dom_or_url",
      "value": "__NEXT_DATA__ or /_next/",
      "detail": "Next.js runtime marker"
    }]
  }]
}

Configuration

Settings are stored in formharvester.json. Campaign form details and search queries are stored in profiles/<name>.json. Use formharvester settings path to locate the active configuration directory.

Campaign policy is stored in each profiles/<name>.json under policy. Important fields include autopilot_enabled, minimum_score, require_review, cooldown_days, max_submissions_per_run, max_submissions_per_day, dry_run, include_technologies, and exclude_technologies. Autopilot defaults to disabled and review defaults to required.

Set FORMHARVESTER_HOME to move configuration, profiles, data, and logs to another directory.

Architecture

Google discovery
      ↓
Selenium Chrome session
      ├── contact-page discovery
      ├── email extraction
      ├── technology signal capture
      │     └── local fingerprint rules
      ├── optional LLM Subject/Message generation
      └── optional form fill/submit
      ↓
HarvestResult + campaign output files

The detector is intentionally separate from the form handler. If a page exposes no technology signal, or if browser performance logs are unavailable, normal harvesting continues with HTML, DOM, and resource signals.

FormHarvester does not require Wappalyzer, a hosted technology API, or a browser extension. LLM providers are isolated behind a normalized client interface, and lead persistence is local SQLite, so an additional provider or reporting client can be added without changing the browser workflow.

Development

uv sync
uv run pytest -q -m "not smoke"
uv run pytest -q -m smoke
uv run ruff check src tests

The smoke test needs Chrome, a matching driver, and network access. The test suite includes browser-free detector and API tests as well as the live smoke test.