Tuesday, August 30, 2016

Learning with DVRF - Step 11 - Part 2 - Let's get that GDB running

Step 11 - “Hello” GDB demonstrated in 5 parts
In the previous post, we were able to get the appropriate “pre-checks” done for GDB/pwndbg. We trudged through the steps necessary to get GDB/pwndbg correctly aligned with steps to get a better debugging experience. This post will briefly examine the “hello” binary in GDB/pwndbg and then go through each line step-by-step in GDB/pwndbg. This is blog post 11, part 2 of 5, of post x in the learning with DVRF project series.

Part 2: What do all of these symbols mean?

  1. In pwndbg, type in:


  2. Here we can see no warnings, we’re getting stack information from objects outside of our binary (e.g. uClibc), accurate backtrace information with stack information from external libraries (e.g. uClibc), and things look to be better since we now have more accurate representations of information.
  3. Now we can debug all the things! Actually, at this point, we are going to get the same results as when we did this manually in a previous blog post. The main (no, not the function, hah!) object of this post was to get more familiarity with GDB/pwndbg (such as start/stopping GDB, hooking up binaries, setting/removing/checking breakpoints), making sure we broke on the actual entry point of main, and getting our symbols from external libraries loaded appropriately.
  4. There are some subtle differences though from when we disassembled this by “hand” within objdump and doing this interactively within GDB/pwngdb. We will step through “hello” with GDB/pwndbg to observe how the stack and registers fluctuate with each step.
  5. Let’s review the four panes in GDB/pwndbg:

  6. In the first “pane” labeled “Registers” we can see the MIPS registers and how they’re used within our program. This is pretty much a way better view of the registers than relying upon “i r / info registers” with the default view that GDB provides.

    pwndbg registers:

    GDB’s default view of registers:

  7. The second pane of “Code” is the MIPS assembly of the code being executed. This looks fairly similar to the disassembled output from objdump. However, we see that instead of seeing the values in decimal (e.g. in the first line) we get a more consistent view of numbers and see hex representation of the number being manipulated.

  8. The third pane of “Stack” represents a view of the stack as it changes through the execution of the program.

  9. The fourth pane of “Backtrace” represents the caller functions which shows how the program got to where it is now.

Thursday, August 25, 2016

Learning with DVRF - Step 11 - Part 1 - Let's get that GDB running

Step 11 - “Hello” GDB demonstrated in 5 parts In the previous post, we were able to get the appropriate “pre-checks” done for GDB/pwndbg. We trudged through the steps necessary to get GDB/pwndbg correctly aligned with steps to get a better debugging experience. This three part post will briefly examine the “hello” binary in GDB/pwndbg and then go through each line step-by-step in GDB/pwndbg. This is blog post 11, part 1 of 5, of post x in the learning with DVRF project series.

Part 1: Set debugger to debug!

  1. Log into the VM
  2. Open up a Terminal
  3. In the Terminal window with Qemu, press the up arrow to re-run the Qemu launcher:

  4. In another Terminal window, browse to the buildroot directory and launch objdump:


    cd buildroot/buildroot-2016.05/output/host/usr/bin

    ./mipsel-linux-objdump -d /home/andy/Downloads/DVRF-master/Firmware/_DVRF_v03.bin.extracted/squashfs-root/pwnable/Intro/hello

  5. We can use the output from objdump above to make sure we have the appropriate hex address to set a breakpoint. As reference, we can see that 004007b0 is the hex address for <main>:

  6. In the Terminal window that we used for objdump, let’s launch gdb-multiedit:


  7. At this point, we should have Qemu emulating the “hello” binary in a MIPS environment in one Terminal window with a GDB stub in a waiting/blocking mode. In another Terminal window, we should have GDB/pwndbg running. To think about it more, we have our host system (e.g. this Mac) running VMware which is running a x64 Linux OS which is then running Qemu to emulate a MIPS environment. Interesting.
  8. In pwndbg, we need to set our target. In pwndbg, type in:

    target remote

  9. Let’s set the path to the “hello” binary with the “file” command. In pwndbg, type in:

    file /home/andy/Downloads/DVRF-master/Firmware/_DVRF_v03.bin.extracted/squashfs-root/pwnable/Intro/hello

  10. In GDB, we need to should set the path to our shared library. In pwndbg, type in:

    set solib-search-path /home/andy/Downloads/DVRF-master/Firmware/_DVRF_v03.bin.extracted/squashfs-root/lib/

  11. Let’s verify that with the corresponding show command. In pwndbg, type in:

    show solib-search-path

  12. Let’s verify that the libraries are loaded with the symbols loaded. In pwndbg, type in:

    info sharedlibrary
  13. We can see that we do have the symbols of the functions within uClibc loaded:

  14. We did not compile the binary with debugging information, so this warning is to be expected
  15. Let’s set the breakpoint to the main entry point with the appropriate address. In pwndbg, type in:

    break *4007b0

  16. At this point, we should really be ready to actually do step-by-step debugging of “hello” with the appropriate corresponding C libraries.

Tuesday, August 23, 2016

Four Common Places Where Vulnerable MIPS Systems Are Hiding

Security vulnerabilities are lurking in common places with the flood of highly advanced computers in devices all around us. These vulnerabilities can have a big impact on a lot of things such as people, processes, and “things”. Not only do these devices lack adequate security attention as compared to popular office software suites, advanced hacking groups such as nation states are also focusing on efforts to circumvent these embedded devices. A common problem found with embedded devices include a lack of security such as not having reliable updates to firmware. This spells trouble as computers become cheaper with more capability and are used in more places to automate and support business initiatives.
In a recent news story, we saw that the Equation Group recently had some of their exploits leaked by another group. In an analysis by Moyix, we see the Equation Group focused not only on exploiting software on common computer architectures like Intel x86 and x86-64, but also ARM and MIPS as below:

Many security professionals focus on x86 and x86-64 system exploitation but focus less on the ARM and MIPS architectures. Based on anecdotal evidence, if analysis is done on an embedded device CPU architecture, ARM is usually the focus of research efforts. As we can see above, even advanced groups such as Equation Group focus on learning and exploiting both ARM and MIPS systems.
Embedded devices with a MIPS chip is a common occurrence that can be found in many places. Here are four common scenarios where MIPS is used and where vulnerabilities can be hiding:
  1. Home Network Routers

    Typically, home network routers that connect wired and wireless devices to the internet are powered with a MIPS chip. With these potentially vulnerable devices, there is a good chance that unless the router is set to automatically download and install firmware, the stock firmware is still in place once the router is installed. While some vendors may release updated firmware, the hardware devices may not make the update process intuitive or seen as a critical step needed to be taken. To add upon that, hardware vendors are constantly refreshing the product lines and may stop supporting devices with security updates as even as one year old. Consumers are left with options such as installing custom firmware from places like OpenWRT to remediate software vulnerabilities or buying new hardware.

    As consumer electronics become cheaper and have more widespread usage, similar devices are also going into ICS/SCADA facilities to help automate their workloads. When once manual operations and on-site OT personnel were needed, these operations are being replaced with far more cost effective embedded devices. These embedded devices can relay and sometimes perform, the same actions as their previous OT technicians. As a result of this new need from ICS/SCADA industries, a flood of devices have entered each market. Unfortunately, these devices are not always held up to the same scrutiny as the devices available from major vendors such as Rockwell Automation, Siemens, and more. The devices could be running a MIPS processor, an ARM processor, or even an AVR processor. The ICS/SCADA industry spreads across many verticals such as amusement parks, water reclamation facilities, and energy distribution so the need of secure and safe devices is critical.
  3. IoT / Mobile Devices

    Mobile devices are common and highly accessible across the world that use MIPS, ARM, and other processors. Devices such as Android phones and tablets are highly customizable and can run on low-end hardware which enables more device acquisition across the world. As these devices thrive and proliferate across the world, the goal of device availability is more important than ensuring they are secure. However, these devices enable security analysts with lots of opportunity to research and report vulnerabilities with a low cost to start research in the embedded field. Low-cost Android tablets available on Amazon include tablets (while not specifically MIPS) from Alldaymall and Dragon Touch (both roughly $40 each) which could potentially hold vulnerabilities with a low cost barrier of entry.
  4. Automotive

    A more intelligent car has more features to appeal to potential buyers. As more features are implemented in vehicles, more computers and/or processing power is necessary to enable these features. Processors like MIPS, ARM, and others power these features in vehicles to provide a better experience to drivers. Automobile manufacturers like Tesla use MIPS processors to power various features such as the Automated Driver Assistance Systems (ADAS) and detection of objects on the road. While the manufacturing processes have been focused and refined for decades, the introduction of advanced computers and custom software has not had the same amount of time to mature and develop. The barrier to entry for investigation into this segment of the embedded market makes research more difficult than research into a home router. However, this does not mean that the security in vehicles are impenetrable. As we have seen with recent research into automobile security, we still have more work as a community to help us stay secure.

Making sense of a new computer architecture such as MIPS and then understanding security issues can be daunting. Learn MIPS and software security at:


Thursday, August 18, 2016

Learning with DVRF - Step 10 - gdb, you know me!

Step 10 - gdb, you know me!

Now that we have a good understanding of MIPS assembly, let’s run the “hello” program through GDB that has extra flavor with pwndbg and make sure it’s in a good place for debugging. This is blog post 10 of x in the learning with DVRF project series.

Part 1: Let's get "hello" attached

  1. Log into the VM
  2. Open up a Terminal
  3. Browse to the squashfs-root directory. In Terminal, type in:


    cd Downloads/DVRF-master/Firmware/_DVRF_v03.bin.extracted/squashfs-root/

  4. Now we need to set up the Qemu environment to run our binary in a simulated MIPS environment. In Terminal, type in:

    sudo chroot . ./qemu-mipsel-static -g 1144 ./pwnable/Intro/hello

  5. The Terminal window in the previous step is actively running and waiting for a debugger to attach to the running process. The “-g” switch we used against the Qemu binary asked Qemu to launch the process in the simulated MIPS environment and open up a “stub” connection to a debuggable process (in our case, the “hello” binary). This switch will allow us to connect from GDB (or another applicable debugger) to the debuggable stub on port 1144 on the IP address of the host. Here is the description from the Qemu binary on the “-g” option:

  6. Open up another Terminal window
  7. In this second Terminal window, type in:


    cd buildroot/buildroot-2016.05/output/host/usr/bin

  8. To help us with future steps, let’s open up objdump with the disassembly of the “hello” binary. In the second Terminal, type in:

    ./mipsel-linux-objdump -d /home/andy/Downloads/DVRF-master/Firmware/_DVRF_v03.bin.extracted/squashfs-root/pwnable/Intro/hello

  9. In the objdump disassembly, browse to the “main” section

  10. Now that we have the stub actively listening for the debugger to attach to the process and the disassembly in another window, let’s open a third Terminal window. In the third Terminal window we will use GDB with the pwndbg addition to actively step through the actively running “hello” binary. All of this is done…. actively.
  11. Open up a third Terminal window. In the third Terminal window, type in:


  12. We should have a successfully loaded a GDB/pwndbg session which is ready to attach to the stub in the first Terminal window:

Part 2: What do we "C"?
  1. In pwndbg, type in:
  2. target remote
  3. The target command should connect to the debugger stub session on the local IP address ( on port 1144. Note: I’m randomly picking ports when I do this. Feel free to use whatever and won’t conflict with whatever else is running on the system.
  4. The process hasn’t been started yet in the first window. However, we need to match up the almost-running process in the first Terminal window to the symbols of the “hello” binary. The symbols are what helps us “translate” the machine code to more human readable output. We need some way to match up what we see in the disassembly in GDB/pwndbg to a more friendly way of what we see in the objdump Terminal window. Since we did not strip the symbols in the compilation of the “hello” binary, we can attach our GDB/pwndbg session with the binary on disk. In the GDB/pwndbg Terminal window, type in:
  5. file /home/andy/Downloads/DVRF-master/Firmware/_DVRF_v03.bin.extracted/squashfs-root/pwnable/Intro/hello
  6. In many first run cases, we want to set a generic breakpoint to the “main” function of a program. We will let the program go through other preliminary phases and land automatically on the “main” function. In the previous step, we let our mapping to the actual “hello” binary help us map to the memory location of the beginning of the “main” function. With that, let’s set create a breakpoint to the “main” function so we can start stepping through execution from that memory address. In GDB, type in: break main
  7. With this proceeding step, I am getting odd behavior sometimes as gdb/pwndbg thinks the entry point for “main” is way off. The breakpoint is many times set on a memory address 8 or so instructions into the “main” function. This is what I see now:
  8. I’m not really too worried about it right now. The better way and especially so for this example is to set it to the memory address of "main" from objdump. If we take a look at the Terminal window with objdump, we can see the memory address associated with “main”. Since we don’t have ASLR or anything else on right now, this should be the same address we see in the pwndbg/gdb window. From objdump:

  9. In pwndbg, type in: break *4007b0
  10. In the preceding step, we are setting a breakpoint at a specific memory address with the “*” and the hexadecimal memory address. If we wanted to see our currently set breakpoints, we can type in: info breakpoints Or i b
  11. Let’s not have two breakpoints as that would be annoying since they’re in the same function (main). In the above screenshot we can see that at this point, the second breakpoint knows it's at the top level of the main function. So helpful! Let’s remove the first breakpoint so we just hit the breakpoint upon settling into main. In pwndbg, type in: del break 1

  12. Let’s verify we just have one. In pwndbg, type in: i b
  13. Let’s actually run this program now! In pwndbg, type in: c
  14. We get a lot of text right away that flies up. If we scroll to right below the “c” from above, we see pwndbg/GDB have some issue with referencing a function.

  15. The C libraries that we are using in pwndbg/GDB are not the same C libraries that Qemu is using with the emulated environment hosting the “hello” binary. Enter in “n” to proceed with the next step of execution: n

  16. pwndbg/gdb seems to be working without the reference to the uClibc library. However, we should point pwndbg/gdb to the same library as a better reference point. We can use “set solib-search-path” to the “lib” file path within the squashfs folder structure. We can see right now what our search path is set to with “show solib-search-path”:
  17. We can set the path with set solib-search-path and continue on in our pwndbg session. Or we can start fresh with the appropriate configurations done in the beginning before we start stepping through the program. I’m going to go ahead and start fresh again. In pwndbg, type in: q
  18. The preceding step should stop pwndbg/gdb as well as terminate the Qemu gdbstub session.