Category: Software testing

You might have encountered the java.lang.VerifyError DEX verification error when developing an Android app. There are several reasons for this. Most commonly being the IDE messing up with the build process and cleaning and rebuilding might solve the problem. In some cases it could also be tools that we use (for example, security tools for obfuscation). There are several resources for this case on the net and is relatively easy to fix.

However, what I wanted to write about in this post is not from the developer point of view, but rather from automated software testing point of view (that involves instrumentation or code transformation), where you have hundreds or even thousands of Android apps to test and you don’t have their source code. Here is my experience.

For a given security testing experiment of Android apps, I had to mutate apps that satisfy some mutation criteria. After the mutation is applied, I had to automatically verify whether the mutation didn’t break the app. To achieve this, I had to apply the mutation (and all the steps necessary to make app ready for install), then install the app on the emulator and test the mutated component if it crashes. In order to understand if the crash is caused by the applied mutation, I wrap the introduced statements within try-catch block and log the exception.

However, running the mutated apps on the emulator failed with the java.lang.VerifyError. The strict Runtime refused to load the “inconsistent” bytecode into the VM because it found a “Bad method” or some other reasons. This might depend on the level of instrumentation that you are applying and hence if you’re just introducing, say, a log statement only, maybe you will not encounter this problem and the instrumented app might run without a problem.

Since mutation is applied automatically in different real world apps, addressing the problem for each app is a bit difficult. For example, it is known that the Runtime will report error if we try to wrap a  synchronized block in a  try-catch block. Therefore, while doing the mutation, it will be a bit difficult (but not impossible) to first know if a call that I wrapped in a  try-catch block will eventually have a synchronized block in it. Even if I know this in advance (say, for example, during static analysis to check mutation criteria), it has no help as I cannot skip the  try-catch block since I need it to see if failure is caused by mutation and I cannot also remove the synchronized block since I will interfere with the design of the app.

Cause

This is just one case as the Android Runtime checks for several inconsistencies that were ignored during the Dalvik VM time. To mention some of inconsitences that are caught by the new Runtime:

• extending a class that was declared final
• overriding a package-private methods
• invalid control-flow
• unbalanced moniterenter/moniterexit (this might be the reason for the synchronized block but I haven’t checked the final bytecode for the said inconsistency)
• passing wrong argument to a method

Solution

A more general solution would have been understanding what modifications are making the verification fail and improving the instrumentation. However, that is out of my scope for the moment.

So, the solution that I found is specific to my problem. Considering that the ART is introduced in Android 5 (API 21), the easiest workaround I found was using an emulator running, say, API 20. Since I know what kind of mutations I am applying and I also monitor executions, resorting to a less restrictive Runtime would’t affect the general behavior of the app under test.

Therefore, if your instrumented Android app isn’t running on the emulator for ART  java.lang.VerifyError  error, just use emulators running API level below 21 and it should be an easy workaround.

Cheers!