How to debug multi-threaded programs

Debugging applications that use POSIX threads can be a challenge. Before using a debugger tool, you may consider using ASSERT or printf, because they are good tracing tools and can be easily implemented.

GDB basically can accomplish four different kinds of goals, start your program: Specifying anything that might affect its behavior. Make your program stop on specified conditions. Examine what has happened, when your program has stopped. Finally, change things in your program, so you can experiment with correcting the effects of one bug and go on to learn about another.

In order to debug a program effectively, you need to generate debugging information when you compile it. This debugging information is stored in the object file; it describes the data type of each variable or function and the correspondence between source line numbers and addresses in the executable code.

To request debugging information, specify the `-g' option when you run the compiler. For example:

g++ -g hello.c

To start gdb, simply type “gdb a.out” at UNIX/ LINUX command shell: (you can replace a.out here with your own executable program)

Basic Commands

help

Gdb provides online documentation. Just typing help will give you a list of topics. Then you can type help topic to get information about that topic (or it will give you more specific terms that you can ask for help about). Or you can just type help command and get information about any other command.

 run

Starts the program. If you do not set up any breakpoints the program will run until it terminates or core dumps :)

print -var-

This command prints a variable located in the current scope. For example

print i

or if a is an array

print a[3]

Also you can print variables of variables, again if a is an array and i is an integer,

print a[i]

Finally you can shorten this up, by saying

p a[3]

'p' is just shorthand of print.

list

list linenumber will print out some lines from the source code around linenumber. If you give it the argument function it will print out lines from the beginning of that function. Just list without any arguments will print out the lines just after the lines that you printed out with the previous list command.

next

This executes the current command, and moves to the next command in the program, this too can be made in shorthand by 'n'

step

This steps through the next command. There are differences between step and next. If you are at a function call, and you hit next, then the function will exectute and return. But if you hit step, then you will go to the first line of that function. 's' is just the shorthand of step. To step out a function, use command finish

break -number or function-

This sets a break point. Its basic functionality is to type break and a filename and line number. For example lets say we want to stop in word.cc line fourty-three, we could do the following in gdb:

(gdb) break word.cc:43
Breakpoint 2 at 0x11044: file word.cc, line 43.

Break, like all other gdb function can be shorteed to its first letter 'b'. Also we can stop on a function, lets say we want to stop on the function 'main'.

(gdb) b main 
Breakpoint 3 at 0x110bc: file hello.cc, line 40.

Finally conditional break points can be set up. Lets say you have a for loop and you want to see what the value of x is when the index reaches 8001, there is no way you will step through this, so what you want to do is set a conditional breakpoint. Conditionals work just like what we talked about previously, but add some extra at the end.

(gdb) b word.cc:64 if isset==1
Breakpoint 4 at 0x1100c: file word.cc, line 64

continue

Once a breakpoint is hit, and you want to continue to the next breakpoint or simply go to the exiting state of the program, you can use this command. The shorthand of this, in case you didn't catch the trend, is 'c'

where

This command is analogous to the backtrace command, and it shows you were in the stack you currnetly are. For example

 (gdb) run
Starting program: /home/bhumphre/classwork/ta/tutorial/hello
 


Program received signal SIGSEGV, Segmentation fault.
0xff2b6dec in strlen () from /usr/lib/libc.so.1
(gdb) where
#0  0xff2b6dec in strlen () from /usr/lib/libc.so.1  
#1  0xff2ffe18 in _doprnt () from /usr/lib/libc.so.1
#2  0xff3019d0 in printf () from /usr/lib/libc.so.1  
#3  0x11064 in word::printword (this=0xffbef8b0) at word.cc:21 #4  0x11110 in main () at hello.cc:14  

Tips:

To run gdb inside Emacs, and open one window for debug, one window for source code, you can use the following commands. Just type them one by one on LINUX/UNIX system:

emacs

Meta –X          (equals Alt key + X key)

gdb

a.out                (or other executable program you want to debug)

break main      (add break point before main function)

run                   (ask GDB to execute until break point)

step                  (or next)

…….                (try other gdb commands)

 

Debugging program with multiple threads:

GDB provides these facilities for debugging multi-thread programs:

1) Thread Information

The GDB thread debugging facility allows you to observe all threads while your program runs--but whenever GDB takes control, one thread in particular is always the focus of debugging. This thread is called the current thread. Debugging commands show program information from the perspective of the current thread.

 

For debugging purposes, GDB associates its own thread number--always a single integer with each thread in your program.

info threads

Display a summary of all threads currently in your program. GDB displays for each thread (in this order):

1.      the thread number assigned by GDB

2.      the target system's thread identifier (systag)

3.      the current stack frame summary for that thread

An asterisk `*' to the left of the GDB thread number indicates the current thread

 

Example:

 (gdb) info threads


  3 process 35 thread 27  0x34e5 in sigpause ()


  2 process 35 thread 23  0x34e5 in sigpause ()


* 1 process 35 thread 13  main (argc=1, argv=0x7ffffff8)


    at threadtest.c:68


 

thread threadno

Make thread number threadno the current thread. The command argument threadno is the internal GDB thread number, as shown in the first field of the `info threads' display. GDB responds by displaying the system identifier of the thread you selected, and its current stack frame summary:

(gdb) thread 2
[Switching to process 35 thread 23]
0x34e5 in sigpause ()

2) Breakpoint

When your program has multiple threads, you can choose whether to set breakpoints on all threads, or on a particular thread.

break linespec thread threadno

break linespec thread threadno if ...

linespec specifies source lines; there are several ways of writing them, but the effect is always to specify some source line. Use the qualifier `thread threadno' with a breakpoint command to specify that you only want GDB to stop the program when a particular thread reaches this breakpoint. threadno is one of the numeric thread identifiers assigned by GDB, shown in the first column of the `info threads' display. If you do not specify `thread threadno' when you set a breakpoint, the breakpoint applies to all threads of your program. You can use the thread qualifier on conditional breakpoints as well; in this case, place `thread threadno' before the breakpoint condition, like this:

(gdb) break frik.c:13 thread 28 if bartab > lim

 

Whenever your program stops under GDB for any reason, all threads of execution stop, not just the current thread. This allows you to examine the overall state of the program, including switching between threads, without worrying that things may change underfoot.

Conversely, whenever you restart the program, all threads start executing. This is true even when single-stepping with commands like step or next.

 

In particular, GDB cannot single-step all threads in lockstep. Since thread scheduling is up to your debugging target's operating system (not controlled by GDB), other threads may execute more than one statement while the current thread completes a single step. Moreover, in general other threads stop in the middle of a statement, rather than at a clean statement boundary, when the program stops.

 

You might even find your program stopped in another thread after continuing or even single-stepping. This happens whenever some other thread runs into a breakpoint, a signal, or an exception before the first thread completes whatever you requested.

3) Watchpoint

You can use a watchpoint to stop execution whenever the value of an expression changes, without having to predict a particular place where this may happen.

watch expr

Set a watchpoint for an expression. GDB will break when expr is written into by the program and its value changes.

rwatch expr

Set a watchpoint that will break when watch expr is read by the program. If you use both watchpoints, both must be set with the rwatch command.

awatch expr

Set a watchpoint that will break when args is read and written into by the program. If you use both watchpoints, both must be set with the awatch command.

info watchpoints

This command prints a list of watchpoints, breakpoints, and catchpoints; it is the same as info break.

GDB tutorial (detailed information)

http://www.delorie.com/gnu/docs/gdb/gdb_toc.html