Understanding Metasploit modules
Let's dig deeper into the process of writing a module. Metasploit has various modules, such as payloads, encoders, exploits, NOP generators, auxiliaries, and the latest additions, which are the evasion modules. In this section, we will cover the essentials of developing a module; then, we will look at how we can create our custom modules. We will discuss the development of auxiliary and post-exploitation modules. Additionally, we will cover core exploit modules in the next chapter. However, for this chapter, let's examine the essentials of module building in detail.
Metasploit module building in a nutshell
Before diving deep into building modules, let's understand how components are arranged in the Metasploit Framework, and what they do.
The architecture of the Metasploit Framework
Metasploit contains various components, such as necessary libraries, modules, plugins, and tools. A diagrammatic view of the structure of Metasploit is as follows:
Figure 2.1 – Metasploit architecture
Let's see what these components are and how they work. It is best to start with the libraries that act as the heart of Metasploit. We can see the core libraries in the following table:
We have many types of modules in Metasploit, and they differ in functionalities. We have payload modules for creating access channels to exploited systems. We have auxiliary modules to carry out operations such as information gathering, fingerprinting, fuzzing an application, and logging in to various services. Let's examine the basic functionality of these modules, as shown in the following table:
Understanding the file structure
The file structure in Metasploit is laid out in the scheme shown in the following screenshot:
Figure 2.2 – Metasploit file structure
The preceding directory can be referred to through the /opt/metasploit-framework/embedded/framework path. We will cover the most relevant directories, which will aid us in building modules for Metasploit, in the following table:
The libraries layout
Metasploit modules are the buildup of various functions contained in different libraries, and general Ruby programming. Now, to use these functions, we first need to understand what they are. How can we trigger these functions? What number of parameters do we need to pass? Moreover, what will these functions return?
Let's have a look at how these libraries are organized; this is illustrated in the following screenshot:
Figure 2.3 – Contents of the /lib directory
As we can see in the preceding screenshot, we have the critical rex libraries along with all other essential ones in the /lib directory. The /base and /core libraries are also a crucial set of libraries and are located under the /msf directory:
Figure 2.4 – Library content for the /msf directory
Now, under the /msf/core libraries folder, we have libraries for all the modules we used earlier in the first chapter; this is illustrated in the following screenshot:
Figure 2.5 – Libraries in the msf/core directory
These library files provide the core for all modules. However, for different operations and functionalities, we can refer to any library we want. Some of the most widely used library files in most of the Metasploit modules are located in the core/exploits/ directory, as shown in the following screenshot:
Figure 2.6 – Libraries in the core/exploits directory
As we can see, it's easy to find all the relevant libraries for various types of modules in the core/ directory. Currently, we have core libraries for exploits, payload, post-exploitation, encoders, and various other modules.
Important note
Visit the Metasploit Git repository at https://github.com/rapid7/metasploit-framework to access the complete source code.
Working with existing Metasploit modules
The best way to start writing modules is to delve deeper into the existing Metasploit modules and see how they work internally.
The format of a Metasploit module
The skeleton for Metasploit auxiliary modules is reasonably straightforward. We can see the universal header section in the code shown here:
require 'msf/core'
class MetasploitModule < Msf::Auxiliary
def initialize(info = {})
super(update_info(info,
'Name' => 'Module name',
'Description' => %q{
Say something that the user might want to know.
},
'Author' => [ 'Name' ],
'License' => MSF_LICENSE
))
end
def run
# Main function end
end
A module starts by including the necessary libraries using the require keyword, which in the preceding code is followed by the msf/core libraries. Thus, it includes the core libraries from the /msf directory.
The next major thing is to define the class type, that is, to specify the kind of module we are going to create. We can see that we have set MSF::Auxiliary for the same purpose.
In the initialize method, which is the default constructor in Ruby, we define the Name, Description, Author, License, CVE details, and so on. This method covers all the relevant information for a particular module: Name generally contains the software name that is being targeted; Description includes the excerpt on the explanation of the vulnerability; Author is the name of the person who develops the module; and License is MSF_LICENSE, as stated in the code example listed previously. The auxiliary module's primary method is the run method. Hence, all the operations should be performed inside it unless and until you have plenty of other methods. However, the execution will still begin with the run method.
Disassembling the existing HTTP server scanner module
Let's work with a simple module for an HTTP version scanner, and see how it works. The path to this Metasploit module is /modules/auxiliary/scanner/http/http_version.rb.
Let's examine this module systematically:
##
# This module requires Metasploit: https://metasploit.com/download
# Current source: https://github.com/rapid7/metasploit-framework
##
require 'rex/proto/http'
class MetasploitModule < Msf::Auxiliary
Let's discuss how things are arranged here. The copyright lines, starting with the # symbol, are the comments and are included in all Metasploit modules. The require 'rex/proto/http' statement tasks the interpreter to include a path to all the HTTP protocol methods from the rex library. Therefore, the path to all the files from the /lib/rex/proto/http directory is now available to the module, as shown in the following screenshot:
Figure 2.7 – Library files in the /lib/rex/proto/http directory
All these files contain a variety of HTTP methods, which include functions to set up a connection, the GET and POST request, response handling, and so on.
In the next line, Msf::Auxiliary defines the code as an auxiliary type module. Let's continue with the code, as follows:
# Exploit mixins should be called first include Msf::Exploit::Remote::HttpClient include Msf::Auxiliary::WmapScanServer
# Scanner mixin should be near last include Msf::Auxiliary::Scanner
The preceding section includes all the necessary library files that contain methods used in the modules. Let's list the path for these included libraries, as follows:
Let's look at the next piece of code:
def initialize super(
'Name' => 'HTTP Version Detection',
'Description' => 'Display version information about each system', 'Author' => 'hdm',
'License' => MSF_LICENSE
)
register_wmap_options({ 'OrderID' => 0, 'Require' => {},
})
end
This part of the module defines the initialize method, which initializes the basic parameters such as Name, Author, Description, and License for this module and initializes the WMAP parameters as well. Now, let's have a look at the last section of the code:
# Fingerprint a single host def run_host(ip)
begin
connect
res = send_request_raw({ 'uri' => '/', 'method' => 'GET' }) fp = http_fingerprint(:response => res) print_good("#{ip}:#{rport} #{fp}") if fp
report_service(:host => rhost, :port => rport, :sname => (ssl ? 'https' : 'http'), :info => fp)
rescue ::Timeout::Error, ::Errno::EPIPE ensure
disconnect
end
end
end
The function here is the meat of the scanner.
Libraries and functions
Let's see some essential methods from the libraries that are used in this module, as follows:
Let's now understand the module. Here, we have a method named run_host with the IP as the parameter to establish a connection to the required host. The run_host method is referred from the /lib/msf/core/auxiliary/scanner.rb library file. This method will run once for each host, as shown in the following screenshot:
Figure 2.8 – The scanner.rb library having the run_host method
Next, we have the begin keyword, which denotes the beginning of the code block. In the next statement, we have the connect method, which establishes the HTTP connection to the server, as discussed in the table previously.
Next, we define a variable named res, which will store the response. We will use the send_raw_request method from the /core/exploit/http/client.rb file with the parameter URI as /, and the method for the request as GET:
Figure 2.9 – The /core/exploit/http/client.rb library having the send_raw_request method
The preceding method will help you to connect to the server, create a request, send a request, and read the response. We save the response in the res variable.
This method passes all the parameters to the request_raw method from the /rex/proto/http/client.rb file, where all these parameters are checked. We have plenty of parameters that can be set in the list of parameters. Let's see what they are:
Figure 2.10 – The /rex/proto/http/client.rb library having the raw_request method
The res variable is a variable that stores the results. In the next statement, the http_fingerprint method from the /lib/msf/core/exploit/http/client.rb file is used for analyzing the data in the fp variable. This method will record and filter out information such as Set- cookie, Powered-by, and other such headers. This method requires an HTTP response packet to make the calculations. So, we will supply response => res as a parameter, which denotes that fingerprinting should occur on the data received from the request generated previously using res. However, if this parameter is not given, it will redo everything and get the data again from the source. The next statement prints out a good informational message with details such as IP, port, and the service name, but only when the fp variable is set. The report_service method stores the information to the database. It will save the target's IP address, port number, service type (HTTP or HTTPS, based on the service), and the service information. The last line, rescue::Timeout::Error, ::Errno::EPIPE, will handle exceptions if the module times out.
Now, let's run this module and see what the output is:
Figure 2.11 – Using the http_version metasploit module
So far, we have seen how a module works. We can see that on a successful fingerprint of the application, the information is posted on the console and saved in the database. Additionally, on a timeout, the module doesn't crash and is handled well. Let's take this a step further and try writing our custom module.