61  hakrawler(1) - Kailash Tool Reference

61.1 hakrawler

hakrawler - Web crawler for endpoint and asset discovery.

Layer CLASSIC
Category c-01-information-gathering
Safety class safe
Licence GPL-3.0-only
Upstream github.com/hakluke/hakrawler

61.1.1 NAME

hakrawler - Web crawler for endpoint and asset discovery.

61.1.2 SYNOPSIS & usage

Derived from the upstream README at generation time; verify against --help where the tool is installed.

echo https://google.com | hakrawler
cat urls.txt | hakrawler

61.1.3 DESCRIPTION (upstream)

Fast golang web crawler for gathering URLs and JavaScript file locations. This is basically a simple implementation of the awesome Gocolly library. Single URL: echo https://google.com | hakrawler Multiple URLs: cat urls.txt | hakrawler Timeout for each line of stdin after 5 seconds: cat urls.txt | hakrawler -timeout 5 Send all requests through a proxy: cat urls.txt | hakrawler -proxy http://localhost:8080 Include subdomains: echo https://google.com | hakrawler -subs > Note: a common issue is that the tool returns no URLs. This usually happens when a domain is specified (https://example.com), but it redirects to a subdomain (https://www.example.com). The subdomain is not included in the scope

61.1.4 SEE ALSO

ref: pin - verified 2026-10-14