Efficient brute-forcing
A brute-force attack typically involves a barrage of requests, or guesses, to gain access or reveal information that may be otherwise hidden. We may brute-force a login form on an administrative panel in order to look for commonly used passwords or usernames. We may also brute-force a web application's root directory looking for common misconfiguration and misplaced sensitive files.
Many successful engagements were made so by weak credentials or application misconfiguration. Brute-forcing can help to reveal information that may have been obscured, or can grant access to a database because the developer forgot to change the default credentials.
There are obvious challenges to brute-forcing. Primarily, it is time-consuming and can be very noisy. Brute-forcing a web service, for example, with the infamous rockyou.txt wordlist will no doubt wake up your friendly neighborhood security operations center (SOC) analyst and may put an end to your activities early. The rockyou.txt list has over 14 million entries and could eventually result in a successful credential guess, but it may be better to limit the flood of traffic to the target with a smaller, more efficient list.
One of the better collections of common keywords, credentials, directories, payloads, and even webshells is the SecLists repository: https://github.com/danielmiessler/SecLists.
Note
An alternative, or supplement, to SecLists is FuzzDB. It is a similar collection of files containing various payloads that can help with brute-forcing, and it can also be downloaded from the GitHub repository at https://github.com/fuzzdb-project/fuzzdb.
Grabbing the latest copy of SecLists is easy using git, a popular version control system tool. We can pull down the repository using the git clone command:
root@kali:~/tools# git clone https://github.com/danielmiessler/SecLists
SecLists contains an ever-evolving database of compiled wordlists that can be used in discovery scans, brute-force attacks, and much more:
The security community is a frequent contributor to SecLists, and it is good practice to pull the latest changes from GitHub before starting an engagement.
Hopefully, target mapping has already provided a few key pieces of information that can help you to brute-force more efficiently. While Nikto and Nmap may not always find a quick and easy remote code execution vulnerability, they do return data that can be useful when deciding what wordlist to use for discovery.
Useful information can include the following:
- The webserver software: Apache, NGINX, or IIS
- Server-side development language: ASP.NET, PHP, or Java
- Underlying operating system: Linux, Windows, or embedded
robots.txt- Interesting response headers
- WAF detection: F5 or Akamai
You can make assumptions about the application based on the very simple information shown in the preceding list. For example, an IIS web server is more likely to have an application developed in ASP.NET as opposed to PHP. While PHP is still available on Windows (via XAMPP), it is not as commonly encountered in production environments. In contrast, while there are Active Server Pages (ASP) processors on Linux systems, PHP or Node.js are much more common these days. While brute-forcing for files, you can take this into account when attaching the extension to the payload: .asp and .aspx for Windows targets, and .php for Linux targets is a good start.
The robots.txt file is generally interesting, as it can provide "hidden" directories or files, and can be a good starting point when brute-forcing for directories or files. The robots.txt file essentially provides instructions for legitimate crawler bots on what they're allowed to index and what they should ignore. This is a convenient way to implement this protocol, but it has the implication that this file must be readable by anonymous users, including yourself.
A sample robots.txt file will look something like this:
User-agent: * Disallow: /cgi-bin/ Disallow: /test/ Disallow: /~admin/
Google's crawlers will ignore the subdirectories, but you cannot. This is valuable information for the upcoming scans.
Content discovery
We have already mentioned two tools that are very useful for initial discovery scans: OWASP ZAP and Burp Suite. Burp's Intruder module is throttled in the free version but can still be useful for quick checks. Both of these attack proxies are available in Kali Linux and can be easily downloaded for other distributions. There are other command-line alternatives, such as Gobuster, which can be used to automate the process a bit more.
Burp Suite
As mentioned, Burp Suite comes bundled with the Intruder module, which allows us to easily perform content discovery. We can leverage it to look for hidden directories and files, and even guess credentials. It supports payload processing and encoding, which enables us to customize our scanning to better interface with the target application.
In the Intruder module, you can leverage the same wordlists provided by SecLists and can even combine multiple lists into one attack. This is a powerful module with lots of features, including, but not limited to, the following:
- Cluster bomb attack, which is well suited for multiple payloads, such as usernames and passwords, which we will showcase later
- Payload processing for highly customized attacks
- Attack throttling and variable delays for low and slow attacks
- …and much more!
We will cover these and others in later chapters.
Figure 2.5: The Burp Suite Intruder module Payloads screen
The free version of Burp Suite is readily available in Kali Linux but, as we've noted in the preceding chapter, it is a bit limited. There are some restrictions in the Intruder module, notably the time-throttling of attack connections. For large payload counts, this may become a hindrance.
The professional version of Burp Suite is highly recommended for those who test applications regularly. Burp Suite is also valuable when reverse engineering applications or protocols. It is quite common for modern applications or malware to communicate with external servers via HTTP. Intercepting, modifying, and replaying this traffic can be valuable.
OWASP ZAP
The free alternative to Burp Suite is ZAP, a powerful tool in its own right, and it provides some of the discovery capabilities of Burp Suite.
The ZAP equivalent for Burp's Intruder is the Fuzzer module, and it has similar functionality, as show in the following figure:
Figure 2.6: OWASP ZAP's Fuzzer module configuration. As ZAP is open-source, there are no usage restrictions. If the goal is to perform a quick content discovery scan or credential brute-force, it may be a better alternative to the free version of Burp Suite.
Gobuster
Gobuster is an efficient command-line utility for content discovery. Gobuster does not come preinstalled on Kali Linux, but it is available on GitHub. As its name implies, Gobuster was written in the Go language and will require the golang compiler to be installed before it can be used for an attack.
The steps to configure Gobuster are fairly easy on Kali Linux. We can start by issuing the following command:
root@kali:~# apt-get install golang
The preceding command will globally install the Go compiler. This is required to build the latest version of Gobuster.
Next, you need to make sure that the GOPATH and GOBIN environment variables are set properly. We will point GOPATH to a go directory in our home path and set GOBIN to the newly defined GOPATH value:
root@kali:~# export GOPATH=~/go root@kali:~# export GOBIN=$GOPATH
We can now pull the latest version of Gobuster from GitHub using the git clone command:
root@kali:~/tools# git clone https://github.com/OJ/gobuster Cloning into 'gobuster'... [...]
We can then get dependencies, and compile the Gobuster application. The go get and go build commands will generate the Gobuster binary in the local directory:
root@kali:~/tools/gobuster# go get && go build
If the commands don't produce output, the tool was compiled and is ready for use:
root@kali:~/tools/gobuster# ./gobuster Gobuster v1.3 OJ Reeves (@TheColonial) ===================================================== [!] WordList (-w): Must be specified [!] Url/Domain (-u): Must be specified ===================================================== root@kali:~/tools/gobuster#
Gobuster has many useful features, including attacking through a proxy (such as a local Burp Suite instance), outputting to a file for further processing, or even brute-forcing subdirectories for a target domain.
The following figure shows Gobuster performing a discovery scan on the http://10.0.5.181 using a common web content file from the SecLists repository:
Figure 2.7: Sample Gobuster running on the 10.0.5.181 server
A command-line URL discovery tool may prove useful on systems where we cannot run a full-blown graphical user interface (GUI) application, such as Burp or ZAP.
Persistent content discovery
The results of a particular scan can reveal interesting directories, but they're not always accessible, and directory indexing is increasingly rare in applications. Thankfully, by using content discovery scans we can look into directories for other misconfigured sensitive information. Consider a scenario where the application hosted on http://10.0.5.181/ contains a particular directory that may be password protected. A common misconfiguration in applications is to protect the parent directory but incorrectly assume all subdirectories are also protected. This leads developers to drop more sensitive directories in the parent and leave them be.
Earlier inspection of the robots.txt file revealed a few interesting directories:
Disallow: /cgi-bin/ Disallow: /test/ Disallow: /~admin/
The admin directory catches the eye, but attempting to access /~admin/ returns an HTTP 403 Forbidden error:
Figure 2.8: Access to the directory is forbidden
This may be discouraging, but we can't stop here. The target directory is too attractive to give up now. Using OWASP ZAP, we can start a new Fuzzer activity on this directory and see if we can find anything of interest that is not protected.
Make sure that the cursor is placed at the end of the URL in the left-most pane. Click the Add button next to Fuzz Locations in the right-most pane:
Figure 2.9: Fuzzer configuration, adding Fuzz Locations
On the next screen, we can add a new payload to feed the Fuzzer. We will select the raft-small-files.txt wordlist from the SecLists repository:
Figure 2.10: Fuzzer configuration – the Add Payload screen
Since we want to treat the /~admin URI as a directory and look for files within, we will have to use a string processor for the selected payload. This will be a simple Prefix String processor, which will prepend a forward-slash to each entry in our list.
Figure 2.11: Fuzzer configuration – the Add Processor screen
The Fuzzer task may take a while to complete, and it will produce lots of 403 or 404 errors. In this case, we were able to locate a somewhat hidden administration file.
Figure 2.12: The completed Fuzzer scan shows an accessible hidden file
The HTTP 200 response indicates that we have access to this file, even though the parent directory /~admin/ was inaccessible. It appears we have access to the admin.html file contained within the enticing admin directory.
Application security is hard to implement correctly, and it is even harder to maintain that initial security baseline as the application ages and evolves, and staff rotate. Access is granted and not removed; files are added with broken permissions; and underlying operating systems and frameworks become outdated, and remotely exploitable.
When running initial content discovery scans, it is important to remember not to stop at the first error message we see. Access control deficiencies are very common, and we could uncover various unprotected subdirectories or files if we are persistent.
Payload processing
Burp Suite's Intruder module is a powerful ally to an attacker when targeting web applications. Earlier discovery scans have identified the secretive, but enticing, /~admin/ directory. A subsequent scan of the directory itself uncovered an unprotected admin.html file.
Before we proceed, we will switch to the Burp Suite attack proxy and configure the Target Scope to the vuln.app.local domain:
Figure 2.13: The Burp Suite Target Scope configuration screen
The Target Scope allows us to define hosts, ports, or URLs that are to be included in the scope of the attack. This helps to filter out traffic that may not be related to our target. With Burp Suite configured as our attack proxy, we can visit the hidden admin.html URL and record the traffic in our proxy's history:
Figure 2.14: Accessing the hidden file through the browser succeeds
Following the Server Connectivity Test link, we are greeted with a basic authentication realm Admin Tools, as shown here:
Figure 2.15: Authentication popup when attempting to follow the link
Our pentester reflexes kick in and we automatically type in the unfortunately common admin/admin credentials, but with no luck this time.
Since all of the interactions with the target are being recorded by the Burp proxy, we can simply pass the failed request on to the Intruder module, as shown in the following figure. Intruder will let us attack the basic authentication mechanism with little effort:
Figure 2.16: The HTTP history screen
In the Intruder module, the defaults are good for the most part—we just have to select the Base64-encoded credentials portion of the Authorization header and click the Add button on the right-hand side. This will identify this position in the HTTP request as the payload location.
The following shows the payload position selected in the Authorization header:
Figure 2.17: Specifying a payload position in the Authorization header
In the Payloads tab, we will select the Custom iterator payload type from the dropdown, as seen in the following figure:
Figure 2.18: Configuring the Payload type
The Authorization header contains the Base64-encoded plaintext values of the colon-separated username and password. To brute-force the application effectively, the payload will have to be in the same format. We will need to submit a payload that follows the same format that the Authorization header expects. For each brute-force request that the attack proxy will make, the payload will have to be the username and password separated by a colon, and wrapped by Base64 encoding: base64([user_payload]:[password_payload]).
We can grab the already captured value in the Authorization header and pass it to Burp Suite's Decoder module. Decoder allows us to quickly process strings to and from various encoding schemes, such as Base64, URL encoding, GZip, and others.
This figure shows how we can leverage Decoder to convert the value YWRtaW46YWRtaW4= from Base64 using the Decode as... dropdown. The result is listed in the bottom pane as admin:admin:
Figure 2.19: The Burp Decoder screen
Back in the Intruder module, for payload position 1, we will once again use a small wordlist from the SecLists Usernames collection called top-usernames-shortlist.txt. Our goal is to find low-hanging fruit, while minimizing the flood of requests that will hit the application. Using a short list of common high-value usernames is a good first step.
This figure shows that the list was loaded in payload position 1 using the Load... button in the Payload Options:
Figure 2.20: Payload position 1 configuration screen
The separator for position 1 should be colon (:). For payload position 2, you can use the 500-worst-passwords.txt list from the SecLists passwords directory.
The following figure shows payload position 2 containing the loaded 500-worst-passwords.txt contents:
Figure 2.21: Payload position 2 configuration screen
The separator for position 2 should be left blank.
At this point, each request sent to the application will contain an Authorization header in the following format:
Authorization: Basic admin:admin Authorization: Basic admin:test [...] Authorization: Basic root:secret Authorization: Basic root:password
To complete the payload, we also have to instruct Intruder to Base64-encode the payload before sending it over the wire. We can use a payload processor to force Base64 encoding for every request.
In the Payloads tab, under Payload Processing, click Add and select the Base64-encode processor from the Encode category. We will also disable automatic URL encoding, as it may break the Authorization header.
The following URL shows the enabled Base64-encode processor:
Figure 2.22: Payload processing rule – Base64-encode
Once the payload has been configured, we can begin the brute-force using the Start Attack button in the top-right corner of the Intruder module, as shown in the following figure:
Figure 2.23: Starting the attack
As with the content discovery scan, this credential brute-force will generate a fair amount of HTTP 401 errors. If we're lucky, at least one will be successful, as seen in the figure that follows:
Figure 2.24: Attack results screen
Now, because every request in the Intruder attack is recorded, we can inspect each one or sort all of them by column to better illustrate the results of the attack. In the preceding example, we can clearly see that the successful authentication request returned an HTTP status code of 200, while the majority of the other requests returned an expected 401. The status code is not the only way to determine success at a quick glance, however. A deviation in the content length of the response may be a good indicator that we are on the right track.
Now that we have a payload that has successfully gained access to the Admin Tools authentication realm, we can run it through the Decoder module to see the plaintext credentials.
This figure shows the Decoder module revealing the guessed credentials:
Figure 2.25: Burp Suite Decoder
Credential brute-forcing is just one of the many uses for Intruder. You can get creative with custom payloads and payload processing.
Consider a scenario where the vuln.app.local application generates PDF files with sensitive information and stores them in an unprotected directory called /pdf/. The filenames appear to be the MD5 digest of the date the file was generated, but the application will not generate a PDF file every day. You could try and guess each day manually, but that's not ideal. You can even spend some time whipping up a Python script that can automate this task. The better alternative is to leverage Burp Suite to do this easily with a few clicks. This has the added benefit of recording the attack responses in one window for easy inspection.
Once again, we can send a previously recorded request to the target /pdf/ folder directly to the Intruder module.
This figure shows that the PDF's name, minus the extension, is identified as the payload position using the Add button:
Figure 2.26: Intruder Payload Positions configuration screen
The following figure shows the Dates payload type options available in Intruder:
Figure 2.27: Intruder's Payloads screen
In this attack, you will use the Dates payload type with the proper date format, going back a couple of years. The payload processor will be the MD5 hash generator, which will generate a hash of each date and return the equivalent string. This is similar to our Base64-encode processor from the previous attack.
Once again, the payload options have been configured and we can start the attack.
The following figure shows a few requests with the 200 HTTP status code and a large length indicating a PDF file is available for download:
Figure 2.28: Intruder attack Results screen
Intruder will generate the payload list based on our specified date format and calculate the hash of the string, before sending it to the application, all with a few clicks. In no time, we have discovered at least three improperly protected, potentially sensitive documents that are available anonymously.