- A list of targets is provided
- A TCP connect scan is done on the target port to test if it's open
- If it's open, HTTP and HTTPS requests are sent to determine if the service is HTTP-based and whether it requires TLS
- If the service is HTTP, a check is done to determine if a previous report file is in the same directory
- Dirbuster is run using Python's subprocess.Popen(). If a timeout is specified, then after the timeout period, a SIGINT signal is sent to Dirbuster so it can safely shut down and write results to disk. A note is added to the report indicating that the scan timed out.
- The next IP:port goes through the same process (TCP connect, HTTP service query, dirbust)
What's really useful about this workflow is that a target with a closed port or non-HTTP based services running can still be provided to AutoDirbuster. The advantage of this is that Nmap scan results can be directly provided to AutoDirbuster. In fact, there's an option just for that: provide an Nmap Gnmap results file as a list of targets.
Installation
The installation process is straightforward:
- Clone the repository with git
- Navigate to the repository on your machine and install dependencies
- Run AutoDirbuster. If you see the usage output, the installation was a success and you're ready to use AutoDirbuster.
Copy and paste the commands below to install AutoDirbuster:
git clone https://github.com/NetSPI/AutoDirbuster.git cd AutoDirbuster && pip3 install -r requirements.txt python AutoDirbuster.py
If the script isn't working as intended, check the GitHub repository for common issues here.
Features
A number of features were added to make AutoDirbuster customizable.
These features include:
- Target timeout
- Automatically end a scan after a given amount of time
- Useful for targets that respond with the same status code for every request or for an unresponsive or slow target
- Automatic DNS reverse lookup
- The reverse lookup hostname result will be used instead of just the IP
- Useful for targets that are using virtual hosting
- Gnmap mode
- Directly provide an Nmap Gnmap results file as the list of targets
- Port scan and then immediately start directory busting
- Custom wordlist
- AutoDirbuster uses OWASP's directory-list-2.3-small.txt by default but any list can be used
- Single target mode
- Quickly launch DirBuster from the terminal against a single target without having to spend time configuring its parameters
- Recursive mode
- Custom file extension list
- Number of connection threads
- Start point of the scan
Recommended Workflow
- Run Nmap and find open ports, outputting the results with "-oG" or "-oA"
- Run AutoDirbuster in a terminal multiplexer, such as tmux, with the Nmap results and a timeout
- Example: python AutoDirbuster.py -g Nmap_results.gnmap -to 15
- As the pentest progresses, periodically review the dirbust results using the included DirBuster pretty printing script dirbust_read.py, which will ignore all DirBuster error lines and only print the found directories and files
Conclusion
Directory busting is an important part of a penetration test but can be a painful manual process on its own. Using AutoDirbuster makes directory busting painless, efficient, and very fast. Give it a shot and see if you find it useful.
https://github.com/NetSPI/AutoDirbuster
[post_title] => AutoDirbuster - Automatically Run and Save DirBuster Scans for Multiple IPs [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => autodirbuster [to_ping] => [pinged] => [post_modified] => 2023-05-18 12:51:48 [post_modified_gmt] => 2023-05-18 17:51:48 [post_content_filtered] => [post_parent] => 0 [guid] => https://blog.netspi.com/?p=11763 [menu_order] => 469 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) ) [post_count] => 2 [current_post] => -1 [before_loop] => 1 [in_the_loop] => [post] => WP_Post Object ( [ID] => 11783 [post_author] => 55 [post_date] => 2020-10-29 07:00:52 [post_date_gmt] => 2020-10-29 07:00:52 [post_content] =>TL;DR
Use NetblockTool to easily dump a unique list of IP addresses belonging to a company and its subsidiaries.
Download the tool here: https://github.com/NetSPI/NetblockTool
The Problem
A problem that I was frequently running into for both offensive and defensive roles is determining the IP addresses that a company owns and uses. Traditionally, gathering a list of IP addresses a company owns is a long and very manual process. Various sources need to be used like Google, ARIN, WHOIS, IPinfo, Censys, and Shodan. The list goes on.
Thankfully, there are some automated tools that exist that make this process a bit easier. Recon-ng is one of these tools but it isn't perfect, and while it does a lot of things well, easily gathering a complete list of netblocks for a company is not one of those things. This is where NetblockTool comes in.
The Solution: NetblockTool
Written as a standalone Python script, NetblockTool is designed to fill in this tooling gap.
For blue team users, simply provide the name of your company and receive a list of unique netblocks, ranked by the likelihood that the returned netblock belongs to your company.
For red team users, use NetblockTool to gather IP ranges, points of contact, and even netblocks belonging to your target's subsidiaries.
Getting Started
Getting started is easy. Simply clone the repository, install the requirements, and you're ready to start using NetblockTool.
git clone https://github.com/NetSPI/NetblockTool.git cd NetblockTool && pip3 install -r requirements.txt python3 NetblockTool.py -v Company
How does it work?
NetblockTool uses several data sources to gather netblocks that a company may own, which include Google dorking, the ARIN database, the ARIN API website, and IPinfo. Since public websites are being scraped, there is no API key needed for any site when using NetblockTool.
First, the user provides a target company. NetblockTool then scrapes Google using a Google dork to retrieve networks that IPinfo knows about.
Next, the ARIN database is queried by sending the same traffic a normal user would send by visiting their website and manually searching for a company. The results are then scraped for ARIN objects (like networks and company contacts) and the objects are visited and further scraped. The advantage of this method is that more results are provided than just directly querying the database using their APIs.
After all sources have been scraped, each discovered netblock is deduplicated and assigned a confidence score that it belongs to the company. The score is largely based on the name of the netblock, the type of ARIN object it is (either ASN, network, or a leased range known as a customer), and the address linked to the netblock.
From here, further operations are then performed that are based on the user's arguments, such a retrieving geolocation data for each IP.
Finally, the total number of addresses is printed and the results are written to a CSV. The first 15 rows for Google are shown below.
Subsidiaries
What if a company has subsidiaries and has netblocks registered to them? NetblockTool has you covered. It's able to automatically query the Securities and Exchange Commission's public database to retrieve a list of possible subsidiaries and then enumerate the subsidiaries' netblocks.
Common Use Cases
There are many different ways of getting the data you desire from NetblockTool, but the easiest way of running the tool is simply:
python3 NetblockTool.py -v Company
If you want to extract netblocks owned by your target company's subsidiaries, specify that flag:
python3 NetblockTool.py -v Company -s
Extracting point of contact information can also be helpful:
python3 NetblockTool.py -v Company -p
Or, if you want to get as much information as possible, including netblocks found using wildcard queries, points of contact, geolocation data, and physical addresses:
python3 NetblockTool.py -wpgav Company -so
Conclusion
Whether you need to find the netblocks your employer owns or find the netblocks for your next red team engagement, NetblockTool is your quick and easy solution. Give it a shot and see if you find it useful.
https://github.com/NetSPI/NetblockTool
[post_title] => NetblockTool: The Easy Way to Find IP Addresses Owned by a Company [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => netblocktool [to_ping] => [pinged] => [post_modified] => 2023-05-18 12:51:05 [post_modified_gmt] => 2023-05-18 17:51:05 [post_content_filtered] => [post_parent] => 0 [guid] => https://blog.netspi.com/?p=11783 [menu_order] => 458 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [comment_count] => 0 [current_comment] => -1 [found_posts] => 2 [max_num_pages] => 0 [max_num_comment_pages] => 0 [is_single] => [is_preview] => [is_page] => [is_archive] => [is_date] => [is_year] => [is_month] => [is_day] => [is_time] => [is_author] => [is_category] => [is_tag] => [is_tax] => [is_search] => [is_feed] => [is_comment_feed] => [is_trackback] => [is_home] => 1 [is_privacy_policy] => [is_404] => [is_embed] => [is_paged] => [is_admin] => [is_attachment] => [is_singular] => [is_robots] => [is_favicon] => [is_posts_page] => [is_post_type_archive] => [query_vars_hash:WP_Query:private] => 90571db7c37130b86a524a194c3089ec [query_vars_changed:WP_Query:private] => [thumbnails_cached] => [allow_query_attachment_by_filename:protected] => [stopwords:WP_Query:private] => [compat_fields:WP_Query:private] => Array ( [0] => query_vars_hash [1] => query_vars_changed ) [compat_methods:WP_Query:private] => Array ( [0] => init_query_flags [1] => parse_tax_query ) )