When pentesting a website, it is important to find hidden information which could possibly be attack vectors but are not visible on the public. Finding hidden files can be considered the first step in searching for potential attacks in a website. There are different tools created for this purpose but they each have their way of usage. Websploit, for example, is one of those tools.

In this guide, however, we are going to look at how to install and use dirsearch in Ubuntu. Dirsearch is written in Python and is used in brute-forcing hidden web directories and files. It is a simple to use, yet powerful command line tool, and is available for Linux, Windows and MacOS. Some of the features that make dirsearch stand out include: multithreading, proxy support, request delaying, user agent randomization and support for multiple extensions. Being Python, it is easy to integrate into scripts and existing projects.

Dirsearch works best when it comes to recursive scanning. When it finds any file, it goes back through it and crawl for any additional directories. Due to this recursive scanning, its simplicity and speed that comes with the use of command line, dirsearch remains a powerful tool that every pentester should know.

Install Dirsearch on Linux

For my installation, I am going to be using Ubuntu. We are going to be installing from github. Update your system repository and install git.

sudo apt-get update
sudo apt-get install git

Next, use git command to clone the directory where the dirsearch will be

$ git clone https://github.com/maurosoria/dirsearch
Cloning into 'dirsearch'...
remote: Enumerating objects: 12757, done.
remote: Counting objects: 100% (448/448), done.
remote: Compressing objects: 100% (200/200), done.
remote: Total 12757 (delta 347), reused 248 (delta 248), pack-reused 12309 (from 3)
Receiving objects: 100% (12757/12757), 21.87 MiB | 9.65 MiB/s, done.
Resolving deltas: 100% (8356/8356), done.

Change to the dirsearch created directory

cd dirsearch

List the content to ensure that is is properly installed.

$ ls
CHANGELOG.md  CONTRIBUTING.md  db  default.conf  dirsearch.py  Dockerfile  lib  logs  README.md  reports  thirdparty

Configure Dirsearch

Once installed, dirsearch can be run in different ways which we are going to discuss below:

But before that, install defusedxml:

sudo python3 -m pip install defusedxml

Run Dirsearch Using Python

To run dirsearch with Python, ensure that you have python3 installed in your system. The syntax to use is:

python3 dirsearch.py -u <target-url>

Run dirsearch using bash

To dirsearch with bash, we simply run the .py executable file as below:

./dirsearch.py

Run dirsearch using symbolic link

Here, we will create a symbolic link in the /bin directory which then allows us to run dirsearch from anywhere as opposed to just running it from its directory. Run the below commands to create the symbolic link:

cd /bin/ 
sudo ln -s ~/dirsearch/dirsearch.py dirsearch

After that simply run the command ‘dirsearch’ from anywhere

$ dirsearch
URL target is missing, try using -u <url>

How to use Dirsearch to Scan files and directories

Note that using -h flag gives more information on how to use dirsearch.

$ dirsearch -h
Mandatory: 
   -u URL, --url=URL   URL target 
   -l URLLIST, --url-list=URLLIST 
                       URL list target 
   -e EXTENSIONS, --extensions=EXTENSIONS 
                       Extensions list separated by comma (Example: php,asp) 
   -E, --extensions-list 
                       Use predefined list of common extensions 
   -X EXCLUDEEXTENSIONS, --exclude-extensions=EXCLUDEEXTENSIONS 
                       Exclude extensions list, separated by comma (Example: 
                       asp,jsp) 

 Dictionary Settings: 
   -w WORDLIST, --wordlist=WORDLIST 
                       Customize wordlist (separated by comma) 
   --prefixes=PREFIXES 
                       Add custom prefixes to all entries (separated by 
                       comma) 
   --suffixes=SUFFIXES 
                       Add custom suffixes to all entries, ignores 
                       directories (separated by comma) 
   -f, --force-extensions 
                       Force extensions for every wordlist entry. Add 
                       %NOFORCE% at the end of the entry in the wordlist that 
                       you do not want to force 
   --no-extension      Remove extensions in all wordlist entries (Example: 
                       admin.php -> admin) 
   --no-dot-extensions 
                       Remove the "." character before extensions 
   -C, --capitalization 
                       Capital wordlist 
   -U, --uppercase     Uppercase wordlist 
   -L, --lowercase     Lowercase wordlist 

 General Settings: 
   -d DATA, --data=DATA 
                       HTTP request data (POST, PUT, ... body) 
   -r, --recursive     Bruteforce recursively 
   -R RECURSIVE_LEVEL_MAX, --recursive-level-max=RECURSIVE_LEVEL_MAX 
                       Max recursion level (subdirs) (Default: 1 [only 
                       rootdir + 1 dir]) 
   --suppress-empty    Suppress empty responses 
   --minimal=MINIMUMRESPONSESIZE 
                       Minimal response length 
   --maximal=MAXIMUMRESPONSESIZE 
                       Maximal response length 
   --scan-subdir=SCANSUBDIRS, --scan-subdirs=SCANSUBDIRS 
                       Scan subdirectories of the given URL (separated by 
                       comma) 
   --exclude-subdir=EXCLUDESUBDIRS, --exclude-subdirs=EXCLUDESUBDIRS 
                       Exclude the following subdirectories during recursive 
                       scan (separated by comma) 
   -t THREADSCOUNT, --threads=THREADSCOUNT

At the very least, dirsearch requires a URL and at least one file extension to run. For example, we can give a valid URL with the -u flag, and a file extension to search for using the -e flag:

dirsearch -u http://10.10.2.15/site1 -e php

The above command is suppose to provide information about the extensions, http methods used, threads number and the size of current word list. It then crawls the directories and returns the findings including status code, size and name of directory.

We also pass -x flag to exclude certain HTTP status codes.

dirsearch -u http://10.10.2.15/site1 -e php -x 403

We can tell dirsearch to use a wordlist of our choice by setting the -w flag:

 dirsearch -u http://10.10.2.15/site1 -e php -x 403,301,302 -w /usr/share/wordlists/wfuzz/general/common.txt

To run the recursive search, simply tack on the -r flag.The command completes the initial scan then go back through and scan each directory it found recursively.

dirsearch -u http://10.10.2.15/dvwa -e php -x 403,301,302 -r

It is possible to pause the scan at any time with a keyboard interrupt. Pressing e completely exits the scan c will continue where it left off, and n will make it to move on to the next directory. These give us some control over the results since recursive scanning can often take quite some time.

Note that to set the recursion level to a deeper value, use the -R flag and indicate how many levels deep to go.

dirsearch -u http://10.10.2.15/site1 -e php -x 403,301,302 -r -R 3

That’s it about how to install and use Dirsearch on Ubuntu. Practice more and enjoy! Check below more interesting guides on Linux:

• How To Install Wireshark on Linux Mint 22
• How To Ping IP or Host with Graph using gping
• How to Install WordPress with Nginx on Amazon Linux 2023