Saturday, March 28, 2009

How to prepare a system for kernel crash dump analysis?

In simple steps:

1 - Collect the below details about the system from where the crash dump was generated.

- Architecture of the system. i686, x86_64 and etc.
- Version of the kernel running while the system crashed.

2 - Prepare a system which runs the same OS as of the crashed system and the same arch. (Not mandatory, but below steps depends on this). Using the same system which crashed is also fine.

3 - Download the kernel-debug package matching the version of the kernel which was running while the system crashed and install it using "rpm -ivh packagename" command.

Debug info packages for RHEL can be downloaded from here

4 - Run crash as below.

# crash /usr/lib/debug/lib/modules/kernel-version/vmlinux path-to-the-vmcore

Eg, I had a customer recently reported a kernel crash with rhel5 kernel "2.6.18-128.1.1.el5xen". Below is the steps that I did.

- Downloaded debug kernel from here

- Installed it using "rpm -ivh kernel-xen-debuginfo-2.6.18-128.1.1.el5.x86_64.rpm"

- Executed crash as below.

# crash /usr/lib/debug/lib/modules/2.6.18-128.1.1.el5xen/vmlinux /root/vmcore

No comments: