1 .\" Copyright 1994-6 Dr. Greg Wettstein, Enjellic Systems Development.
2 .\" Copyright 1997-2007 Martin Schulze <joey@infodrom.org>
3 .\" May be distributed under the GNU General Public License
5 .TH KLOGD 8 "27 May 2007" "Version 1.5" "Linux System Administration"
7 klogd \- Kernel Log Daemon
30 is a system daemon which intercepts and logs Linux kernel
35 Sets the default log level of console messages to \fIn\fR.
38 Enable debugging mode. This will generate \fBLOTS\fR of output to
42 Log messages to the specified filename rather than to the syslog facility.
45 Signal the currently executing klogd daemon. Both of these switches control
46 the loading/reloading of symbol information. The \-i switch signals the
47 daemon to reload the kernel module symbols. The \-I switch signals for a
48 reload of both the static kernel symbols and the kernel module symbols.
51 Avoid auto-backgrounding. This is needed especially if the
53 is started and controlled by
57 Execute in 'one\-shot' mode. This causes \fBklogd\fP to read and log
58 all the messages that are found in the kernel message buffers. After
59 a single read and log cycle the daemon exits.
62 Enable paranoia. This option controls when klogd loads kernel module symbol
63 information. Setting this switch causes klogd to load the kernel module
64 symbol information whenever an Oops string is detected in the kernel message
68 Force \fBklogd\fP to use the system call interface to the kernel message
72 Use the specified file as the source of kernel symbol information.
75 Print version and exit.
78 Omits EIP translation and therefore doesn't read the System.map file.
81 When symbols are expanded, print the line twice. Once with addresses
82 converted to symbols, once with the raw text. This allows external
83 programs such as ksymoops do their own processing on the original
86 The functionality of klogd has been typically incorporated into other
87 versions of syslogd but this seems to be a poor place for it. In the
88 modern Linux kernel a number of kernel messaging issues such as
89 sourcing, prioritization and resolution of kernel addresses must be
90 addressed. Incorporating kernel logging into a separate process
91 offers a cleaner separation of services.
93 In Linux there are two potential sources of kernel log information: the
95 file system and the syscall (sys_syslog) interface, although
96 ultimately they are one and the same. Klogd is designed to choose
97 whichever source of information is the most appropriate. It does this
98 by first checking for the presence of a mounted
100 file system. If this is found the
102 file is used as the source of kernel log
103 information. If the proc file system is not mounted
106 system call to obtain kernel messages. The command line switch
108 can be used to force klogd to use the system call interface as its
111 If kernel messages are directed through the
112 .BR syslogd " daemon the " klogd
113 daemon, as of version 1.1, has the ability to properly prioritize
114 kernel messages. Prioritization of the kernel messages was added to it
115 at approximately version 0.99pl13 of the kernel. The raw kernel messages
118 \<[0\-7]\>Something said by the kernel.
120 The priority of the kernel message is encoded as a single numeric
121 digit enclosed inside the <> pair. The definitions of these values is
122 given in the kernel include file kernel.h. When a message is received
123 from the kernel the klogd daemon reads this priority level and assigns
124 the appropriate priority level to the syslog message. If file output
125 (\fB-f\fR) is used the prioritization sequence is left pre\-pended to the
128 The klogd daemon can also be used in a 'one\-shot' mode for reading the
129 kernel message buffers. One shot mode is selected by specifying the
130 \fB\-o\fR switch on the command line. Output will be directed to either the
131 syslogd daemon or to an alternate file specified by the \fB-f\fR switch.
133 For example, to read all the kernel messages after a system
134 boot and record them in a file called krnl.msg the following
135 command would be given.
138 klogd -o -f ./krnl.msg
140 .SH KERNEL ADDRESS RESOLUTION
141 If the kernel detects an internal error condition a general protection
142 fault will be triggered. As part of the GPF handling procedure the
143 kernel prints out a status report indicating the state of the
144 processor at the time of the fault. Included in this display are the
145 contents of the microprocessor's registers, the contents of the kernel
146 stack and a tracing of what functions were being executed at the time
150 .B EXTREMELY IMPORTANT
151 in determining what caused the internal error condition. The
152 difficulty comes when a kernel developer attempts to analyze this
153 information. The raw numeric information present in the protection
154 fault printout is of very little use to the developers. This is due
155 to the fact that kernels are not identical and the addresses of
156 variable locations or functions will not be the same in all kernels.
157 In order to correctly diagnose the cause of failure a kernel developer
158 needs to know what specific kernel functions or variable locations
159 were involved in the error.
161 As part of the kernel compilation process a listing is created which
162 specified the address locations of important variables and function in
163 the kernel being compiled. This listing is saved in a file called
164 System.map in the top of the kernel directory source tree. Using this
165 listing a kernel developer can determine exactly what the kernel was
166 doing when the error condition occurred.
168 The process of resolving the numeric addresses from the protection
169 fault printout can be done manually or by using the
171 program which is included in the kernel sources.
175 will attempt to resolve kernel numeric addresses to their symbolic
176 forms if a kernel symbol table is available at execution time. If you
177 require the original address of the symbol, use the
179 switch to preserve the numeric address. A
180 symbol table may be specified by using the \fB\-k\fR switch on the
181 command line. If a symbol file is not explicitly specified the
182 following filenames will be tried:
187 .I /usr/src/linux/System.map
190 Version information is supplied in the system maps as of kernel
191 1.3.43. This version information is used to direct an intelligent
192 search of the list of symbol tables. This feature is useful since it
193 provides support for both production and experimental kernels.
195 For example a production kernel may have its map file stored in
196 /boot/System.map. If an experimental or test kernel is compiled with
197 the sources in the 'standard' location of /usr/src/linux the system
198 map will be found in /usr/src/linux/System.map. When klogd starts
199 under the experimental kernel the map in /boot/System.map will be
200 bypassed in favor of the map in /usr/src/linux/System.map.
202 Modern kernels as of 1.3.43 properly format important kernel addresses
203 so that they will be recognized and translated by klogd. Earlier
204 kernels require a source code patch be applied to the kernel sources.
205 This patch is supplied with the sysklogd sources.
207 The process of analyzing kernel protections faults works very well
208 with a static kernel. Additional difficulties are encountered when
209 attempting to diagnose errors which occur in loadable kernel modules.
210 Loadable kernel modules are used to implement kernel functionality in
211 a form which can be loaded or unloaded at will. The use of loadable
212 modules is useful from a debugging standpoint and can also be useful
213 in decreasing the amount of memory required by a kernel.
215 The difficulty with diagnosing errors in loadable modules is due to
216 the dynamic nature of the kernel modules. When a module is loaded the
217 kernel will allocate memory to hold the module, when the module is
218 unloaded this memory will be returned back to the kernel. This
219 dynamic memory allocation makes it impossible to produce a map file
220 which details the addresses of the variable and functions in a kernel
221 loadable module. Without this location map it is not possible for a
222 kernel developer to determine what went wrong if a protection fault
223 involves a kernel module.
226 has support for dealing with the problem of diagnosing protection
227 faults in kernel loadable modules. At program start time or in
228 response to a signal the daemon will interrogate the kernel for a
229 listing of all modules loaded and the addresses in memory they are
230 loaded at. Individual modules can also register the locations of
231 important functions when the module is loaded. The addresses of these
232 exported symbols are also determined during this interrogation
235 When a protection fault occurs an attempt will be made to resolve
236 kernel addresses from the static symbol table. If this fails the
237 symbols from the currently loaded modules are examined in an attempt
238 to resolve the addresses. At the very minimum this allows klogd to
239 indicate which loadable module was responsible for generating the
240 protection fault. Additional information may be available if the
241 module developer chose to export symbol information from the module.
243 Proper and accurate resolution of addresses in kernel modules requires
246 be informed whenever the kernel module status changes. The
250 switches can be used to signal the currently executing daemon that
251 symbol information be reloaded. Of most importance to proper
252 resolution of module symbols is the
254 switch. Each time a kernel module is loaded or removed from the
255 kernel the following command should be executed:
263 switch can also be used to insure that module symbol information is up
264 to date. This switch instructs
266 to reload the module symbol information whenever a protection fault
267 is detected. Caution should be used before invoking the program in
268 \&'paranoid\&' mode. The stability of the kernel and the operating
269 environment is always under question when a protection fault occurs.
270 Since the klogd daemon must execute system calls in order to read the
271 module symbol information there is the possibility that the system may
272 be too unstable to capture useful information. A much better policy
273 is to insure that klogd is updated whenever a module is loaded or
274 unloaded. Having uptodate symbol information loaded increases the
275 probability of properly resolving a protection fault if it should occur.
277 Included in the sysklogd source distribution is a patch to the
278 modules-2.0.0 package which allows the
283 utilities to automatically signal
285 whenever a module is inserted or removed from the kernel. Using this
286 patch will insure that the symbol information maintained in klogd is
287 always consistent with the current kernel state.
288 .SH CONSOLE LOG LEVEL
291 daemon allows the ability to alter the presentation of
292 kernel messages to the system console. Consequent with the
293 prioritization of kernel messages was the inclusion of default
294 messaging levels for the kernel. In a stock kernel the the default
295 console log level is set to 7. Any messages with a priority level
296 numerically lower than 7 (higher priority) appear on the console.
298 Messages of priority level 7 are considered to be 'debug' messages and
299 will thus not appear on the console. Many administrators,
300 particularly in a multi\-user environment, prefer that all kernel
301 messages be handled by klogd and either directed to a file or to
302 the syslogd daemon. This prevents 'nuisance' messages such as line
303 printer out of paper or disk change detected from cluttering the
308 is given on the commandline the
310 daemon will execute a system call to inhibit all kernel messages from
311 being displayed on the console. Former versions always issued this
312 system call and defaulted to all kernel messages except for panics.
313 This is handled differently currently so
315 doesn't need to set this value anymore. The
316 argument given to the \fB\-c\fR switch specifies the priority level of
317 messages which will be directed to the console. Note that messages of
318 a priority value LOWER than the indicated number will be directed to
321 For example, to have the kernel display all messages with a
324 or more severe the following
325 command would be executed:
331 The definitions of the numeric values for kernel messages are given in
333 .IR kernel.h " which can be found in the " /usr/include/linux
334 directory if the kernel sources are installed. These values parallel
335 the syslog priority values which are defined in the file
336 .IR syslog.h " found in the " /usr/include/sys " sub\-directory."
338 The console log level is usually configured with the
340 program, directly or via its configuration file
341 .IR /etc/sysctl.conf .
342 In this file the following line
345 kernel.printk = 4 4 1 7
348 corresponds to the sampe setting above.
352 will respond to eight signals:
353 .BR SIGHUP ", " SIGINT ", " SIGKILL ", " SIGTERM ", " SIGTSTP ", "
354 .BR SIGUSR1 ", "SIGUSR2 " and " SIGCONT ". The"
355 .BR SIGINT ", " SIGKILL ", " SIGTERM " and " SIGHUP
356 signals will cause the daemon to close its kernel log sources and
357 terminate gracefully.
360 .BR SIGTSTP " and " SIGCONT
361 signals are used to start and stop kernel logging. Upon receipt of a
363 signal the daemon will close its
364 log sources and spin in an idle loop. Subsequent receipt of a
366 signal will cause the daemon to go through its initialization sequence
367 and re-choose an input source. Using
368 .BR SIGSTOP " and " SIGCONT
369 in combination the kernel log input can be re-chosen without stopping and
370 restarting the daemon. For example if the \fI/proc\fR file system is to be
371 un-mounted the following command sequence should be used:
382 Notations will be made in the system logs with
385 documenting the start/stop of logging.
388 .BR SIGUSR1 " and " SIGUSR2
389 signals are used to initiate loading/reloading of kernel symbol information.
392 signal will cause the kernel module symbols to be reloaded. Signaling the
395 will cause both the static kernel symbols and the kernel module symbols to
398 Provided that the System.map file is placed in an appropriate location the
399 signal of generally greatest usefulness is the
401 signal. This signal is designed to be used to signal the daemon when kernel
402 modules are loaded/unloaded. Sending this signal to the daemon after a
403 kernel module state change will insure that proper resolution of symbols will
404 occur if a protection fault occurs in the address space occupied by a kernel
410 One Source for kernel messages
413 .I /var/run/klogd.pid
414 The file containing the process id of
417 .I /boot/System.map, /System.map, /usr/src/linux/System.map
418 Default locations for kernel system maps.
421 Probably numerous. Well formed context diffs appreciated.
423 The kernel log daemon
425 was originally written by Steve Lord <lord@cray.com>, Greg Wettstein
426 made major improvements. Martin Schulze <joey@infodrom.org> fixed
427 some bugs and took over maintenance.