Probing static memory layout

The tools used for inspecting the static memory layout usually work on the object files. To get some initial insight, we'll start with an example, example 4.1, which is a minimal C program that doesn't have any variable or logic as part of it:

int main(int argc, char** argv) {

return 0;

}

Code Box 4-1 [ExtremeC_examples_chapter4_1.c]: A minimal C program

First, we need to compile the preceding program. We compile it in Linux using gcc:

$ gcc ExtremeC_examples_chapter4_1.c -o ex4_1-linux.out

$

Shell Box 4-1: Compiling example 4.1 using gcc in Linux

After a successful compilation and having the final executable binary linked, we get an executable object file named ex4_1-linux.out. This file contains a predetermined static memory layout that is specific to the Linux operating system, and it will exist in all future processes spawned based on this executable file.

The size command is the first tool that we want to introduce. It can be used to print the static memory layout of an executable object file.

You can see the usage of the size command in order to see the various segments found as part of the static memory layout as follows:

$ size ex4_1-linux.out

text data bss dec hex filename

1099 544 8 1651 673 ex4_1-linux.out

$

Shell Box 4-2: Using the size command to see the static segments of ex4_1-linux.out

As you see, we have Text, Data, and BSS segments as part of the static layout. The shown sizes are in bytes.

Now, let's compile the same code, example 4.1, in a different operating system. We have chosen macOS and we are going to use the clang compiler:

$ clang ExtremeC_examples_chapter4_1.c -o ex4_1-macos.out

$

Shell Box 4-3: Compiling example 4.1 using clang in macOS

Since macOS is a POSIX-compliant operating system just like Linux, and the size command is specified to be part of the POSIX utility programs, macOS should also have the size command. Therefore, we can use the same command to see the static memory segments of ex4_1-macos.out:

$ size ex4_1-macos.out

__TEXT __DATA __OBJC others dec hex

4096 0 0 4294971392 4294975488 100002000

$ size -m ex4_1-macos.out

Segment __PAGEZERO: 4294967296

Segment __TEXT: 4096

Section __text: 22

Section __unwind_info: 72

total 94

Segment __LINKEDIT: 4096

total 4294975488

$

Shell Box 4-4: Using the size command to see the static segments of ex4_1-macos.out

In the preceding shell box, we have run the size command twice; the second run gives us more details about the found memory segments. You might have noticed that we have Text and Data segments in macOS, just like Linux, but there is no BSS segment. Note that the BSS segment also exists in macOS, but it is not shown in the size output. Since the BSS segment contains uninitialized global variables, there is no need to allocate some bytes as part of the object file and it is enough to know how many bytes are required for storing those global variables.

In the preceding shell boxes, there is an interesting point to note. The size of the Text segment is 1,099 bytes in Linux while it is 4 KB in macOS. It can also be seen that the Data segment for a minimal C program has a non-zero size in Linux, but it is empty in macOS. It is apparent that the low-level memory details are different on various platforms.

Despite these little differences between Linux and macOS, we can see that both platforms have the Text, Data, and BSS segments as part of their static layout. From now on, we gradually explain what each of these segments are used for. In the upcoming sections, we'll discuss each segment separately and we give an example slightly different from example 4.1 for each, in order to see how differently each segment responds to the minor changes in the code.

BSS segment

We start with the BSS segment. BSS stands for Block Started by Symbol. Historically, the name was used to denote reserved regions for uninitialized words. Basically, that's the purpose that we use the BSS segment for; either uninitialized global variables or global variables set to zero.

Let's expand example 4.1 by adding a few uninitialized global variables. You see that uninitialized global variables will contribute to the BSS segment. The following code box demonstrates example 4.2:

int global_var1;

int global_var2;

int global_var3 = 0;

int main(int argc, char** argv) {

return 0;

}

Code Box 4-2 [ExtremeC_examples_chapter4_2.c]: A minimal C program with a few global variables either uninitialized or set to zero

The integers global_var1, global_var2, and global_var3 are global variables which are uninitialized. For observing the changes made to the resulting executable object file in Linux, in comparison to example 4.1, we again run the size command:

$ gcc ExtremeC_examples_chapter4_2.c -o ex4_2-linux.out

$ size ex4_2-linux.out

text data bss dec hex filename

1099 544 16 1659 67b ex4_2-linux.out

$

Shell Box 4-5: Using the size command to see the static segments of ex4_2-linux.out

If you compare the preceding output with a similar output from example 4.1, you will notice that the size of the BSS segment has changed. In other words, declaring global variables that are not initialized or set to zero will add up to the BSS segment. These special global variables are part of the static layout and they become preallocated when a process is loading, and they never get deallocated until the process is alive. In other words, they have a static lifetime.

Let's compile example 4.2 in macOS and have a look at the output of the size command:

$ clang ExtremeC_examples_chapter4_2.c -o ex4_2-macos.out

$ size ex4_2-macos.out

__TEXT __DATA __OBJC others dec hex

4096 4096 0 4294971392 4294979584 100003000

$ size -m ex4_2-macos.out

Segment __PAGEZERO: 4294967296

Segment __TEXT: 4096

Section __text: 22

Section __unwind_info: 72

total 94

Segment __DATA: 4096

Section __common: 12

total 12

Segment __LINKEDIT: 4096

total 4294979584

$

Shell Box 4-6: Using the size command to see the static segments of ex4_2-macos.out

And again, it is different from Linux. In Linux, we had preallocated 8 bytes for the BSS segment, when we had no global variables. In example 4.2, we added three new uninitialized global variables whose sizes sum up to 12 bytes, and the Linux C compiler expanded the BSS segment by 8 bytes. But in macOS, we still have no BSS segment as part of the size's output, but the compiler has expanded the data segment from 0 bytes to 4KB, which is the default page size in macOS. This means that clang has allocated a new memory page for the data segment inside the layout. Again, this simply shows how much the details of the memory layout can be different in various platforms.

In Shell Box 4-6, we have a section named __common, inside the _DATA segment, which is 12 bytes, and it is in fact referring to the BSS segment that is not shown as BSS in the size's output. It refers to 3 uninitialized global integer variables or 12 bytes (each integer being 4 bytes). It's worth taking note that uninitialized global variables are set to zero by default. There is no other value that could be imagined for uninitialized variables.

Let's now talk about the next segment in the static memory layout; the Data segment.

Data segment

In order to show what type of variables are stored in the Data segment, we are going to declare more global variables, but this time we initialize them with non-zero values. The following example, example 4.3, expands example 4.2 and adds two new initialized global variables:

int global_var1;

int global_var2;

int global_var3 = 0;

double global_var4 = 4.5;

char global_var5 = 'A';

int main(int argc, char** argv) {

return 0;

}

Code Box 4-3 [ExtremeC_examples_chapter4_3.c]: A minimal C program with both initialized and uninitialized global variables

The following shell box shows the output of the size command, in Linux, and for example 4.3:

$ gcc ExtremeC_examples_chapter4_3.c -o ex4_3-linux.out

$ size ex4_3-linux.out

text data bss dec hex filename

1099 553 20 1672 688 ex4_3-linux.out

$

Shell Box 4-7: Using the size command to see the static segments of ex4_3-linux.out

We know that the Data segment is used to store the initialized global variables set to a non-zero value. If you compare the output of the size command for examples 4.2 and 4.3, you can easily see that the Data segment is increased by 9 bytes, which is the sum of the sizes of the two newly added global variables (one 8-byte double and one 1-byte char).

Let's look at the changes in macOS:

$ clang ExtremeC_examples_chapter4_3.c -o ex4_3-macos.out

$ size ex4_3-macos.out

__TEXT __DATA __OBJC others dec hex

4096 4096 0 4294971392 4294979584 100003000

$ size -m ex4_3-macos.out

Segment __PAGEZERO: 4294967296

Segment __TEXT: 4096

Section __text: 22

Section __unwind_info: 72

total 94

Segment __DATA: 4096

Section __data: 9

Section __common: 12

total 21

Segment __LINKEDIT: 4096

total 4294979584

$

Shell Box 4-8: Using the size command to see the static segments of ex4_3-macos.out

In the first run, we see no changes since the size of all global variables summed together is still way below 4KB. But in the second run, we see a new section as part of the _DATA segment; the __data section. The memory allocated for this section is 9 bytes, and it is in accordance with the size of the newly introduced initialized global variables. And still, we have 12 bytes for uninitialized global variables as we had in example 4.2, and in macOS.

On a further note, the size command only shows the size of the segments, but not their contents. There are other commands, specific to each operating system, that can be used to inspect the content of segments found in an object file. For instance, in Linux, you have readelf and objdump commands in order to see the content of ELF files. These tools can also be used to probe the static memory layout inside the object files. As part of two previous chapters we explored some of these commands.

Other than global variables, we can have some static variables declared inside a function. These variables retain their values while calling the same function multiple times. These variables can be stored either in the Data segment or the BSS segment depending on the platform and whether they are initialized or not. The following code box demonstrates how to declare some static variables within a function:

void func() {

static int i;

static int j = 1;

...

}

Code Box 4-4: Declaration of two static variables, one initialized and the other one uninitialized

As you see in Code Box 4-4, the i and j variables are static. The i variable is uninitialized and the j variable is initialized with value 1. It doesn't matter how many times you enter and leave the func function, these variables keep their most recent values.

To elaborate more on how this is done, at runtime, the func function has access to these variables located in either the Data segment or the BSS segment, which has a static lifetime. That's basically why these variables are called static. We know that the j variable is located in the Data segment simply because it has an initial value, and the i variable is supposed to be inside the BSS segment since it is not initialized.

Now, we want to introduce the second command to examine the content of the BSS segment. In Linux, the objdump command can be used to print out the content of memory segments found in an object file. This corresponding command in macOS is gobjdump which should be installed first.

As part of example 4.4, we try to examine the resulting executable object file to find the data written to the Data segment as some global variables. The following code box shows the code for example 4.4:

int x = 33; // 0x00000021

int y = 0x12153467;

char z[6] = "ABCDE";

int main(int argc, char**argv) {

return 0;

}

Code Box 4-5 [ExtremeC_examples_chapter4_4.c]: Some initialized global variables which should be written to the Data segment

The preceding code is easy to follow. It just declares three global variables with some initial values. After compilation, we need to dump the content of the Data segment in order to find the written values.

The following commands will demonstrate how to compile and use objdump to see the content of the Data segment:

$ gcc ExtremeC_examples_chapter4_4.c -o ex4_4.out

$ objdump -s -j .data ex4_4.out

a.out: file format elf64-x86-64

Contents of section .data:

601020 00000000 00000000 00000000 00000000 ...............

601030 21000000 67341512 41424344 4500 !....4..ABCDE.

$

Shell Box 4-9: Using the objdump command to see the content of the Data segment

Let's explain how the preceding output, and especially the contents of the section .data, should be read. The first column on the left is the address column. The next four columns are the contents, and each of them is showing 4 bytes of data. So, in each row, we have the contents of 16 bytes. The last column on the right shows the ASCII representation of the same bytes shown in the middle columns. A dot character means that the character cannot be shown using alphanumerical characters. Note that the option -s tells objdump to show the full content of the chosen section and the option -j .data tells it to show the content of the section .data.

The first line is 16 bytes filled by zeros. There is no variable stored here, so nothing special for us. The second line shows the contents of the Data segment starting with the address 0x601030. The first 4 bytes is the value stored in the x variable found in example 4.4. The next 4 bytes also contain the value for the y variable. The final 6 bytes are the characters inside the z array. The contents of z can be clearly seen in the last column.

If you pay enough attention to the content shown in Shell Box 4-9, you see that despite the fact that we write 33, in decimal base, as 0x00000021, in hexadecimal base it is stored differently in the segment. It is stored as 0x21000000. This is also true for the content of the y variable. We have written it as 0x12153467, but it is stored differently as 0x67341512. It seems that the order of bytes is reversed.

The effect explained is because of the endianness concept. Generally, we have two different types of endianness, big-endian and little-endian. The value 0x12153467 is the big-endian representation for the number 0x12153467, as the biggest byte, 0x12, comes first. But the value 0x67341512 is the little-endian representation for the number 0x12153467, as the smallest byte, 0x67, comes first.

No matter what the endianness is, we always read the correct value in C. Endianness is a property of the CPU and with a different CPU you may get a different byte order in your final object files. This is one of the reasons why you cannot run an executable object file on hardware with different endianness.

It would be interesting to see the same output on a macOS machine. The following shell box demonstrates how to use the gobjdump command in order to see the content of the Data segment:

$ gcc ExtremeC_examples_chapter4_4.c -o ex4_4.out

$ gobjdump -s -j .data ex4_4.out

a.out: file format mach-o-x86-64

Contents of section .data:

100001000 21000000 67341512 41424344 4500 !...g4..ABCDE.

$

Shell Box 4-10: Using the gobjdump command in macOS to see the content of the Data segment

It should be read exactly like the Linux output found as part of Shell Code 4-9. As you see, in macOS, there are no 16-byte zero headers in the data segment. Endianness of the contents also shows that the binary has been compiled for a little-endian processor.

As a final note in this section, other tools like readelf in Linux and dwarfdump in macOS can be used in order to inspect the content of object files. The binary content of the object files can also be read using tools such as hexdump.

In the following section, we will discuss the Text segment and how it can be inspected using objdump.

Text segment

As we know from Chapter 2, Compilation and Linking, the linker writes the resulting machine-level instructions into the final executable object file. Since the Text segment, or the Code segment, contains all the machine-level instructions of a program, it should be located in the executable object file, as part of its static memory layout. These instructions are fetched by the processor and get executed at runtime when the process is running.

To dive deeper, let's have a look at the Text segment of a real executable object file. For this purpose, we propose a new example. The following code box shows example 4.5, and as you see, it is just an empty main function:

int main(int argc, char** argv) {

return 0;

}

Code Box 4-6 [ExtremeC_examples_chapter4_5.c]: A minimal C program

We can use the objdump command to dump the various parts of the resulting executable object file. Note that the objdump command is only available in Linux, while other operating systems have their own set of commands to do the same.

The following shell box demonstrates using the objdump command to extract the content of various sections present in the executable object file resulting from example 4.5. Note that the output is shortened in order to only show the main function's corresponding section and its assembly instructions:

$ gcc ExtremeC_examples_chapter4_5.c -o ex4_5.out

$ objdump -S ex4_5.out

ex4_5.out: file format elf64-x86-64

Disassembly of section .init:

0000000000400390 <_init>:

... truncated.

.

.

Disassembly of section .plt:

00000000004003b0 <__libc_start_main@plt-0x10>:

... truncated

00000000004004d6 <main>:

4004d6: 55 push %rbp

4004d7: 48 89 e5 mov %rsp,%rbp

4004da: b8 00 00 00 00 mov $0x0,%eax

4004df: 5d pop %rbp

4004e0: c3 retq

4004e1: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)

4004e8: 00 00 00

4004eb: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)

00000000004004f0 <__libc_csu_init>:

... truncated

.

.

.

0000000000400564 <_fini>:

... truncated

$

Shell Box 4-11: Using objdump to show the content of the section corresponding to the main function

As you see in the preceding shell box, there are various sections containing machine-level instructions: the .text, .init, and .plt sections and some others, which all together allow a program to become loaded and running. All of these sections are part of the same Text segment found in the static memory layout, inside the executable object file.

Our C program, written for example 4.5, had only one function, the main function, but as you see, the final executable object file has a dozen other functions.

The preceding output, seen as part of Shell Box 4-11, shows that the main function is not the first function to be called in a C program and there are logics before and after main that should be executed. As explained in Chapter 2, Compilation and Linking, in Linux, these functions are usually borrowed from the glibc library, and they are put together by the linker to form the final executable object file.

In the following section, we start to probe the dynamic memory layout of a process.