Crash files in android

How to make sense of Android crash logs

Android Crash Log Errors: Explained

Crashes on Android can be immensely frustrating for users, so much so that after experiencing as little as two crashes, the typical user will uninstall your app. Fortunately, the Android Framework provides some great tools for debugging crashes, and provides several useful crash logs that developers can read to determine what caused that critical issue. In this blog post we’ll cover the three most important crash logs used by the system: exception stack traces, ANR traces, and NDK tombstones.

Exception Stack trace

JVM stack traces are the most common type of crash that typical Android applications will encounter, as the majority of apps are written in either Kotlin or Java. In JVM languages, an Exception is thrown in exceptional circumstances, and contains debug information about the error condition that went wrong, such as a stack trace with file/line number information, and an error message.

If an app hasn’t got a crash reporting SDK installed, then the next best method for retrieving crash logs is to use adb to view logcat. This is a convenient method if physical access to the device is an option, because the default UncaughtExceptionHandler in Android apps prints out the entire stack trace to Logcat before terminating the process, meaning the crash is effectively logged out to an accessible location for developers.

ANR Trace

ANRs (Application Not Responding) occur when an application does not respond to user input for a noticeable period of time. The visible effect of this is that an app has ‘frozen’ from a user’s perspective, which can be immensely frustrating. Common causes include performing disk reads/writes on the main thread, and other long-running tasks, which prevents the User Interface from updating in response to user input.

If the app is in the foreground, after approximately 5 seconds a dialog will be shown which allows the user to kill the app. At this point a trace including details of the ANR will be written to disk, from which valuable information for debugging can be retrieved. Again, this requires physical access to the device unless you have a crash reporting SDK installed that supports ANR detection.

Tombstone

Tombstone crash logs are written when a native crash in C/C++ code occurs in an Android application. The Android platform writes a trace of all the running threads at the time of the crash to /data/tombstones, along with additional information for debugging, such as information about memory and open files. Tombstones are the closest to the metal in terms of information, as they will record details such as raw memory addresses, and as such can be a bit trickier to understand unless you’re familiar with debugging native code. Again, tombstones require physical access to a rooted device in order to be read.

How to get Crash Logs from an Android device

As a prerequisite to all these steps, you should have installed Android Studio and added the command line tools to your path. These local methods make use of the adb tool. You should also have a device or emulator connected which has had developer options enabled.

Note: if you are comfortable using the Device File Explorer in Android Studio directly, you can open device files directly from there rather than using adb pull.

Exception Stack trace

By default, exception stack traces are printed out to the Logcat tool on Android devices. It is possible to retrieve crash logs via the following steps:

1. Run the following command

— CODE language-shell —
adb logcat AndroidRuntime:E *:S

1. Trigger a crash on the device. The stack trace will show up as new text in the terminal.
2. Save the terminal output to a file of your choice for inspection later

If a crash has occurred recently on the device, you can skip step 2. This is because Logcat retains a buffer of recent logs which should include the exception. This is time sensitive however — so if you’re looking for a crash from a day ago, that information may be gone forever unless you use a crash reporting tool such as Bugsnag.

ANR Trace

1. Trigger an ANR on the device.
2. Run the following command, replacing the destination with a file of your choice

1. Inspect the information in the ANR crash log by opening the saved file

Alternatively, you can inspect summary ANR information by running the following command

— CODE language-shell —
adb logcat ActivityManager:E *:S

Tombstone

1. Root your device or emulator so that you can access the tombstone directory. (Beware when rooting devices, as this step can potentially brick your phone)
2. Trigger a native crash on the device.
3. Run the following command to determine what tombstone crash logs are present on the device

1. Run the following command, replacing the destination with a file of your choice. tombstone_01 is shown as an example filename here, that would be obtained in the previous step

— CODE language-shell —
adb pull /data/tombstones/tombstone_01

1. Inspect the information in the Tombstone crash log by opening the saved file

Making sense of Android Crash Log Data

Exception Stack trace

Reading a JVM stack trace can be intimidating at first, but by breaking it down into its constituent parts the task becomes fairly easy. Let’s walk through it step by step, with the following RuntimeException that has been thrown in an example application:

— CODE language-shell —
2019-08-27 16:10:28.303 10773-10773/com.bugsnag.android.example E/AndroidRuntime: FATAL EXCEPTION: main
В В Process: com.bugsnag.android.example, PID: 10773
В В java.lang.RuntimeException: Fatal Crash
В В В В at com.example.foo.CrashyClass.sendMessage(CrashyClass.java:10)
В В В В at com.example.foo.CrashyClass.crash(CrashyClass.java:6)
В В В В at com.bugsnag.android.example.ExampleActivity.crashUnhandled(ExampleActivity.kt:55)
В В В В at com.bugsnag.android.example.ExampleActivity$onCreate$1.invoke(ExampleActivity.kt:33)
В В В В at com.bugsnag.android.example.ExampleActivity$onCreate$1.invoke(ExampleActivity.kt:14)
В В В В at com.bugsnag.android.example.ExampleActivity$sam$android_view_View_OnClickListener$0.onClick(ExampleActivity.kt)
В В В В at android.view.View.performClick(View.java:5637)
В В В В at android.view.View$PerformClick.run(View.java:22429)
В В В В at android.os.Handler.handleCallback(Handler.java:751)
В В В В at android.os.Handler.dispatchMessage(Handler.java:95)
В В В В at android.os.Looper.loop(Looper.java:154)
В В В В at android.app.ActivityThread.main(ActivityThread.java:6119)
В В В В at java.lang.reflect.Method.invoke(Native Method)
В В В В at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886)
В В В В at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776)

The first place to start is towards the top of our crash log. Here we can see the process ID that the system assigned the executing app, along with the package name, which can be useful when correlating against other information obtained via logcat. In our example app, the package name is “com.bugsnag.android.example”:

— CODE language-shell —
Process: com.bugsnag.android.example, PID: 10773

The next useful piece of information is the exception class. In Java/Kotlin, all exceptions and errors are classes which extend Throwable or one of Throwable’s subclasses, and each exception class can have a different semantic meaning. For example, a developer may wish to throw an IllegalStateException if the program entered an unexpected state, or an IllegalArgumentException if a user attempted to save null as their name. In our case, we’ve thrown a RuntimeException, whose fully qualified class name is displayed below:

— CODE language-shell —
java.lang.RuntimeException: Fatal Crash

The error message is also printed to the crash log, which can be very useful for providing additional debug information. In our case, we have just supplied the text “Fatal Crash”, but we could equally pass the values of our variables at the time of the crash if we wanted further information for debugging.

The next thing in our crash log is the juicy part of the information — a stack trace of the thread where the exception occurred. Looking at the top stackframe will usually allow us to find exactly where the error was thrown from, and the frames below it will allow us to observe what the program state was at the time of the crash. The top stackframe is the following:

— CODE language-shell —
com.example.foo.CrashyClass.sendMessage(CrashyClass.java:10)

We can immediately see that this contains some very useful information. We are given the class, “com.example.foo.CrashyClass”, so we can open the source file and hunt for a bug there. We’re also given the method name, “sendMessage”, and the line number of the crash, 10, so we can pinpoint exactly in our source where the exception was thrown from.

Understanding the crash log from this point onwards is a case of reading the stack frames below, which are in the order in which the methods were originally called. Consider the example below:

— CODE language-shell —
at com.example.foo.CrashyClass.sendMessage(CrashyClass.java:10)
В В В В at com.example.foo.CrashyClass.crash(CrashyClass.java:6)
В В В В at com.bugsnag.android.example.ExampleActivity.crashUnhandled(ExampleActivity.kt:55)

Starting at the top, we can tell that “sendMessage” was invoked by “crash”, which was in turn invoked by a method named “crashUnhandled”, which appears to be the origin of our crash. Of course, the full exception stack trace was somewhat more complex, and also involved method calls in the Android framework, but the basic principle remains the same.

An important note is that in production, apps are often obfuscated by tools such as Proguard which can mean the original symbols are not present, and the stack trace becomes unintelligible. Fortunately, most crash reporting services provide plugins that automatically upload a mapping file that contains the information necessary to symbolicate crash reports from your production apps.

ANR Trace

ANR traces contain a very large amount of information. As an ANR can potentially be caused by multiple apps contending for a limited number of resources, the full crash log contains stack traces for multiple different processes. This full information can be very useful for debugging when all else fails, but most of the time the summary ANR information printed into Logcat is sufficient to debug the error. Consider the following:

— CODE language-shell —
2019-08-27 16:12:57.301 1717-1733/system_process E/ActivityManager: ANR in com.bugsnag.android.example (com.bugsnag.android.example/.ExampleActivity)
В В PID: 10859
В В Reason: Input dispatching timed out (Waiting to send non-key event because the touched window has not finished processing certain input events that were delivered to it over 500.0ms ago. В Wait queue length: 33. В Wait queue head age: 6178.1ms.)
В В Load: 0.32 / 0.62 / 0.37
В В CPU usage from 323086ms to -1ms ago (2019-08-27 13:16:43.467 to 2019-08-27 16:12:54.131):
В В В 5.7% 1717/system_server: 2.6% user + 3% kernel / faults: 21251 minor
В В В 4.7% 10392/com.bugsnag.android.example: 1.3% user + 3.3% kernel / faults: 587 minor
В В В 3.9% 2375/com.google.android.gms: 2.9% user + 0.9% kernel / faults: 71377 minor 51 major
В В В 3.1% 16/ksoftirqd/2: 0% user + 3.1% kernel
В В В 2.5% 2254/com.google.android.googlequicksearchbox:search: 1.1% user + 1.4% kernel / faults: 10193 minor
В В В 1.2% 10427/kworker/u8:0: 0% user + 1.2% kernel
В В В 0.9% 8990/kworker/u8:2: 0% user + 0.9% kernel
В В В 0.8% 1342/surfaceflinger: 0.1% user + 0.7% kernel / faults: 35 minor
В В В 0.5% 1344/adbd: 0% user + 0.4% kernel / faults: 8471 minor
В В В 0.4% 1896/com.google.android.gms.persistent: 0.3% user + 0% kernel / faults: 1106 minor
В В В 0.4% 1288/logd: 0.1% user + 0.3% kernel / faults: 43 minor
В В В 0.3% 1806/com.android.systemui: 0.2% user + 0% kernel / faults: 404 minor
В В В 0.2% 1916/com.android.phone: 0.1% user + 0% kernel / faults: 203 minor
В В В 0.2% 1410/audioserver: 0% user + 0.1% kernel / faults: 119 minor
В В В 0.1% 10429/kworker/u8:3: 0% user + 0.1% kernel
В В В 0.1% 10378/com.google.android.apps.photos: 0.1% user + 0% kernel / faults: 426 minor
В В В 0.1% 8/rcu_preempt: 0% user + 0.1% kernel
В В В 0% 1396/jbd2/dm-0-8: 0% user + 0% kernel
В В В 0% 2179/com.google.android.apps.nexuslauncher: 0% user + 0% kernel / faults: 802 minor 1 major
В В В 0% 1409/zygote: 0% user + 0% kernel / faults: 857 minor
В В В 0% 3951/com.android.defcontainer: 0% user + 0% kernel / faults: 265 minor
В В В 0% 10137/kworker/u9:0: 0% user + 0% kernel
В В В 0% 1987/wpa_supplicant: 0% user + 0% kernel
В В В 0% 10205/com.google.android.apps.docs: 0% user + 0% kernel / faults: 50 minor
В В В 0% 1378/dmcrypt_write: 0% user + 0% kernel
В В В 0% 2111/com.google.process.gapps: 0% user + 0% kernel / faults: 356 minor
В В В 0% 3882/com.android.printspooler: 0% user + 0% kernel / faults: 241 minor
В В В 0% 8829/kworker/u9:2: 0% user + 0% kernel
В В В 0% 9808/kworker/u9:4: 0% user + 0% kernel
В В В 0% 19/migration/3: 0% user + 0% kernel
В В В 0% 1420/rild: 0% user + 0% kernel
В В В 0% 10138/kworker/u9:1: 0% user + 0% kernel
В В В 0% 1339/lmkd: 0% user + 0% kernel
В В В 0% 1419/netd: 0% user + 0% kernel / faults: 59 minor
В В В 0% 1793/com.android.inputmethod.latin: 0% user + 0% kernel / faults: 12 minor
В В В 0% 10146/com.android.gallery3d: 0% user + 0% kernel / faults: 95 minor
В В В 0% 10181/android.process.acore: 0% user + 0% kernel / faults: 52 minor
В В В 0% 1281/kworker/0:1H: 0% user + 0% kernel
В В В 0% 10162/kworker/2:1: 0% user + 0% kernel
В В В 0% 10348/com.google.android.partnersetup: 0% user + 0% kernel / faults: 92 minor
В В В 0% 20/ksoftirqd/3: 0% user + 0% kernel
В В В 0% 10308/android.process.media: 0% user + 0% kernel / faults: 16 minor
В В В 0% 1336/healthd: 0% user + 0% kernel
В В В 0% 1354/logcat: 0% user + 0% kernel
В В В 0% 1709/hostapd: 0% user + 0% kernel
В В В 0% 3/ksoftirqd/0: 0% user + 0% kernel
В В В 0% 1341/servicemanager: 0% user + 0% kernel
В В В 0% 2091/com.google.android.ext.services: 0% user + 0% kernel / faults: 10 minor
В В В 0% 10475/com.google.android.apps.photos:CameraShortcut: 0% user + 0% kernel / faults: 29 minor
В В В 0% 4/kworker/0:0: 0% user + 0% kernel
В В В 0% 12/ksoftirqd/1: 0% user + 0% kernel
В В В 0% 1422/fingerprintd: 0% user + 0% kernel
В В В 0% 1591/dhcpclient: 0% user + 0% kernel
В В В 0% 1706/ipv6proxy: 0% user + 0% kernel
В В В 0% 1913/sdcard: 0% user + 0% kernel
В В В 0% 2137/com.google.android.googlequicksearchbox:interactor: 0% user + 0% kernel / faults: 3 minor
В В В 0% 687/kworker/1:1: 0% user + 0% kernel
В В В 0% 1297/vold: 0% user + 0% kernel / faults: 10 minor
В В В 0% 1413/installd: 0% user + 0% kernel / faults: 35 minor
В В В 0% 1//init: 0% user + 0% kernel
В В В 0% 11/migration/1: 0% user + 0% kernel
В В В 0% 466

First off, the crash log contains information about which process on the system suffered an ANR, and gives the process ID and package name, which comes in useful when finding the appropriate stack traces in the more detailed ANR trace:

— CODE language-shell —
E/ActivityManager: ANR in com.bugsnag.android.example (com.bugsnag.android.example/.ExampleActivity)
В В PID: 10859

The Android framework gives us a reason for the ANR. In this case, the user touched the screen several times, and the dispatch queue waited for over 6 seconds without showing a visible response to these touch events. This represents a very bad user experience that would be noticeable as the whole app would appear to freeze from a user’s perspective:

— CODE language-shell —
Reason: Input dispatching timed out (Waiting to send non-key event because the touched window has not finished processing certain input events that were delivered to it over 500.0ms ago. Wait queue length: 33. Wait queue head age: 6178.1ms.)

Finally, we’re given some CPU load information. While some ANRs have simple causes such as performing IO on the main thread, this is not always the case. Sometimes an ANR can occur on a low-end device due to a lot of resource-hungry apps competing for CPU, so determining whether there are other apps using lots of resources at the same time as our application can be very helpful:

— CODE language-shell —
CPU usage from 323086ms to -1ms ago (2019-08-27 13:16:43.467 to 2019-08-27 16:12:54.131):
В В В 5.7% 1717/system_server: 2.6% user + 3% kernel / faults: 21251 minor
В В В 4.7% 10392/com.bugsnag.android.example: 1.3% user + 3.3% kernel / faults: 587 minor
В В В 3.9% 2375/com.google.android.gms: 2.9% user + 0.9% kernel / faults: 71377 minor 51 major
В В В 3.1% 16/ksoftirqd/2: 0% user + 3.1% kernel
В В В 2.5% 2254/com.google.android.googlequicksearchbox:search: 1.1% user + 1.4% kernel / faults: 10193 minor

Tombstone

Like ANR traces, tombstones also contain a very large amount of information that wouldn’t be possible to walk through entirely. We’ll consider a truncated example, which shows the most important information near the top:

Источник