Skip to content

SpiderFoot

Automated OSINT collection with a reviewable module set

CommunityOSINT· Relationships & Link AnalysisIntermediateOpen sourceEntry revised 21 Dec 2025
  • Linux
  • macOS
  • Windows
  • Docker
  • Source
Official siteSource

Community. Contributor-submitted. Not every claim has been re-checked. Cross-check against the upstream documentation before relying on a command.

This entry is thinner than the rest of the directory — missing sections are shown as such rather than filled with filler.Improve this pageContribute

Overview

3 commands documented

SpiderFoot orchestrates many public data sources behind one interface: run a scan against a domain or IP, choose modules, and read the correlations it draws. The web UI exists so a human can judge each element rather than trusting a summary.

Module quality depends on the API keys you provide, so results are only as complete as your configuration. Every element carries the source and the raw data — use it when you cite anything.

Supported platforms

5

Documented install or usage guidance

Learning curve

intermediate

Difficulty of becoming productive, not of the underlying theory

Tags

osint, modules, correlation, domain, automation

Dataset entry

spiderfoot.ts

Reviewed 2025-12-21

Installation

Grouped by platform. Elevation requirements are marked per method.

From source in a venv

Alternativesource
git clone https://github.com/smicallef/spiderfoot && cd spiderfoot
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt && python3 sf.py -l 127.0.0.1:5001

Package availability follows your distribution and enabled repositories. Entry revised 21 Dec 2025 — confirm the current release on the project's own download page.

Commands

3 entries

Every command carries its purpose, an example where useful, and the limitations that change how you should read the output.

01Start a scan from the CLI listener

bash

The web interface drives scans; the module set and options are chosen per scan.

python3 sf.py -l 127.0.0.1:5001

02Headless scan with selected modules

bash

The sfl-cli client runs a scan without the UI, which is what a scheduled job needs.

python3 sfl-cli -d 'DOMAIN' -m sfp_dns,sfp_whois,sfp_certtrans -s -t 5

Notes

  • Omit `-m` to run every enabled module; that is slower and noisier than a chosen set.

03Query results programmatically

bash

Scan results are available through the HTTP API so exports can go into a notebook.

curl -s http://127.0.0.1:5001/scans | head -c 400

Common errors

Symptoms you will actually hit, with the cause and the legitimate fix.

Most modules return 'no data' or report missing API keysCause 1/2

Possible causes

  • Several sources require credentials, and some deprecated sources return nothing at all.

Usual fix

Configure keys in Settings, disable modules you cannot feed, and note which sources were unavailable in the report.

UI unreachable from another machineCause 2/2

Possible causes

  • Intentional: the listener binds to loopback.

Usual fix

Use an SSH tunnel (`ssh -L 5001:127.0.0.1:5001 host`) rather than exposing the UI on a network interface.

Tips

  • Start each new case with a fresh scan; SpiderFoot merges elements within a scan and history confuses attribution.
  • Export the scan (CSV) with the raw element data before writing conclusions.

Alternatives & comparisons

Not documented yet: maltego. Request an entry and it will link up automatically.Request a tool

Side-by-side

References

Where to verify anything on this page. External links open in a new tab.

Found something wrong?Suggest an editMore in OSINT

Looking for alternatives?

theHarvester cover adjacent parts of the same job.

Compare side by side