Offensive Security Tool: URL Hunter
Reading Time: 3 Minutes
Offensive Security Tool: URL Hunter
URL Hunter
URL Hunter by utkusen, is a recon tool that allows searching on URLs that are exposed via shortener services such as bit.ly and goo.gl. The project is written in Go.
How?
A group named URLTeam (kudos to them) are brute forcing the URL shortener services and publishing matched results on a daily basis. URL Hunter downloads their collections and lets you analyze them.
This is a great and quick tool recommended for Bug Bounty Hunting, Pentesting & Red Team, aimed at quick discovery of potential URLs that need to be analyzed, and using different patterns to find several vulnerabilities based on the patterns used especially when combined with ‘GF’ another regex tool to show interesting patterns such as finding XSS, LFI, SQL, Exposed Panels and more.
See Also: Not all Phishing attack types can be protected using software solutions
Installation
From Binary
You can download the pre-built binaries from the releases page and run. For example:
tar xzvf urlhunter_0.1.0_Linux_amd64.tar.gz
./urlhunter –help
From Source
- Install Go on your system
- Run: go get -u github.com/utkusen/urlhunter
Note For The Windows Users: urlhunter uses XZ Utils which is pre-installed on Linux and macOS systems. For Windows systems, you need to download it from https://tukaani.org/xz/
See Also: Hacking stories – Operation Aurora: When China hacked Google
Usage
urlhunter requires 3 parameters to run: -keywords, -date and -o.
For example: urlhunter -keywords keywords.txt -date 2020-11-20 -o out.txt
-keywords
You need to specify the txt file that contains keywords to search on URLs. Keywords must be written line by line. You have three different ways to specify keywords:
Single Keyword: urlhunter will search the given keyword as a substring. For example:
acme.com keyword will both match https://acme.com/blabla and https://another.com/?referrer=acme.com
Multiple Keywords: urlhunter will search the given keywords with an AND logic. Which means, a URL must include all the provided keywords. Keywords must be separated with , character. For example:
acme.com,admin will match https://acme.com/secret/adminpanel but won’t match https://acme.com/somethingelse
Regex Values: urlhunter will search for the given regex value. In the keyword file, the line that contains a regular expression formula must start with regex string. The format is: regex REGEXFORMULA. For example:
regex 1\d{10} will match https://example.com/index.php?id=12938454312 but won’t match https://example.com/index.php?id=abc223
-date
urlhunter downloads the archive files of the given date(s). You have three different ways to specify the date:
Latest: urlhunter will download the latest archive. -date latest
Single Date: urlhunter will download the archive of the given date. Date format is YYYY-MM-DD.
For example: -date 2020-11-20
Date Range: urlhunter will download all the archives between given start and end dates.
For example: -date 2020-11-10:2020-11-20
-o
You can specify the output file with -o parameter. For example -o out.txt
Demonstration Video
The Speed Problem
Archive.org throttles the speed when downloading files. Therefore, downloading an archive takes more time than usual. As a workaround, you can download the archives via Torrent and put them under the archive/ folder which is located in the same directory with the urlhunter’s binary. The directory tree will look like:
|-urlhunter
|—urlhunter(binary)
|—archive
|—–urlteam_2020-11-20-11-17-04
|—–urlteam_2020-11-17-11-17-04
Example Use Cases
urlhunter might be useful for cyber intelligence and bug bounty purposes. For example:
docs.google.com/a/acme.com drive.google.com/a/acme.com keywords allow you to find public Google Docs&Drive share links of Acme company.
acme.com,password_reset_token keyword may allow you to find the working password reset tokens of acme.com
trello.com allows you to find public Trello addresses.
See Also: Offensive Security Tool: Discover
Explore our Store: You can find Apparel & Mugs about Hacking and especially for Offensive Security.
Click here ➡ Store
