This exercise is compiled on Debian 2.6.32 with NX and ASLR enabled. However, those
protections do not effect the difficulty of the exercise. Exploiting this challenge could have been performed with or without those protections.
ABO #3 source code:
It may seem that a straight-forward buffer overflow is possible; however, this is not the case due to exit(). The exercise takes two user arguments. The binary will display the second user argument using puts(). Exploiting this challenge requires the need to overwrite a function pointer to avoid the exit() “restriction.” If the function pointer fn is overwritten with the address to system(), the second argument will be used as system()’s argument.
The functions puts() and system() are compiled into the binary. Since the function pointers (puts()/system()) are declared (extern), the pointers to system() and puts() can be found in the disassemble during assignment or from within gdb.
Modified output of objdump -M intel -d abo3. The full dump can be found here
In the snippet above, the instructions are assigning system and puts to fn. The addresses 0x8048364 and 0x8048394 are system() and puts(), respectively. When you look at the source code, the function pointer fn was assigned two values; first assigned to system() followed by puts().
The next issue is to determine the offset between buf and fn.
This is a modified output of gdb (using mammon_’s .gdbinit) while stepping through the binary.
In the above gdb output, 0x41414141 can be seen and indicates the location of the controlled string. Before the call to eax, eax is loaded with the address of puts at $esp+0x11c. Simple mathematics will show that only 256 bytes is needed to the overwrite the function pointer fn.