How To Trace System calls and Signals With Strace Command With Examples?
Linux Operating systems provides ability to track system calls with
strace utility. As simple explanation strace intercepts and prints system calls made by related process. We know that Linux is actually an operating system kernel.
Operating system kernel is responsible from low level operations like device and hardware management, memory management, processes management, providing interface for user level processes and applications.
Strace is especially used for diagnostic, instructional and debugging operations.
Syntax of strace is like below.
strace [-CdffhikqrtttTvVxxy] [-In] [-bexecve] [-eexpr]... [-acolumn] [-ofile] [-sstrsize] [-Ppath]... -ppid... / [-D]
[-Evar[=val]]... [-uusername] command [args]
strace -c[df] [-In] [-bexecve] [-eexpr]... [-Ooverhead] [-Ssortby] -ppid... / [-D] [-Evar[=val]]... [-uusername] command
Simple and fast help about tool can be get with the following command.
$ strace -h
Linux processes are generally started by calling them from bash or running background commands. In this example we will start a process by calling the command. But by calling command we will send the command to the strace as a parameter like below. Keep in mind that this is not a emulation mode the command will be run and complete its job. While doing its job the system calls and related information will be printed to the console. While tracing information like system call name, address, library, file stat will be provided.
$ strace ls
Trace Specific System Calls
In the previous example we have traced a command completely. This have created a lot of information and related output. This may be too much for us because we are looking only some of them. To make things more clear to track we can provide the system call we want to trace. In the following example we have only want to list
open system call which is used to open files.
$ strace -e open ls
Save Trace Result To File
Up to now we have print the trace result to the terminal. Our example process was simple to run and created smaller output. But what will happen if we want to run a long and complex process? Or we want to analyze the trace output later. In this situations we can save the trace output to a file. In this example we have saved the trace output to the file named
ls-trace.txt by using
-o option. Then we can read the file with
$ strace -o ls-trace.txt ls
Trace Already Running Process
We have created new processes to trace in previous examples. But there are some situations we want to trace all ready fired and running process. For example we have web servers running Apache and we want to trace Apache. How can we trace all ready running process? First we will find our process PID.
$ ps -C snapd
Trace With Process ID
We have learned that our snap daemon named
snapd have PID 1193. Than we will provide this PID to the strace to trace with
-p option like below.
$ strace -p 1193
Write Output To A File
Another useful usage is saving the trace to a file with
$ strace -o snapd-trace.txt -p 12793
tail command reading the trace output in real time.
$ tail -f snapd-trace.txt
Print Time Stamp
Time stamp information is very important especially metering the performance. In normal usage of strace time information is not provided. Time stamp information can be printed with
-t option. In the example we simply print time stamp which is consist of current hour, minute and second information.
$ strace -t ls
Print Relative Timestamp
Previous example we have printed time stamp information in normal day format. But we may want to use relative time stamp. Relative time stamp is calculated according to the process start time and process start time is set as 0 and all other system calls time is expressed according to the process start time.
$ strace -r ls
Print Time Stamp More Precise
In the previous example we have printed the time stamp. This time stamp have normal precision where hour, minute and second information is provided. In some situations we may need to get more precise metric from the trace. The
-tt option will create more precise metrics in nano seconds about the trace.
$ strace -tt ls
Generate Statistics Report of System Calls
While tracing a process there is a lot of system calls used. Metrics and statistics about these calls can be printed in a table. This table will include metrics like time, seconds, call count, errors and related system call.
$ strace -c ls
Follow Forked Threads
Operating systems provides threads to multiple a processes for more performance. Some daemon or server processes like Apache, Nginx etc. uses mainly threads. Normally strace do not trace these threads. The
-ff options made the strace to find threads with the related process and trace them too.
$ strace -f ls