Jon Douglas The rantings of a developer

Multidex in depth and overriding the main dex list in Xamarin.Android

This post is a continuation of Multidex in Xamarin.Android

Today we are going to talk about the basic mechanics of the classes.dex file and how multidex handles more than 65k methods.

The classes.dex file is used as a main dex list in your application. It houses all of the classes needed for your application. An APK requires at least one classes.dex(DEX) file which has all the executable code of our application stored inside. The size of a DEX file’s method index is 16-bit which means (2^16) or 65,536 total references a single dex list can have.

However we run into scenarios where we surpass the 65,536 reference count. This is when we need to enable multidex which gives us overfill classes.dex files in the form of classes{n}.dex where n >= 1.

Now we can have as many classes/references as we desire since our application can grow over time. Let’s talk about some of the disadvantages to this however:

1) Multidex tries it’s best to know what dependencies are needed at startup. Since it only loads the main dex list classes.dex at first, you will need to ensure any startup classes/references are inside the MainDexList(classes.dex) or you will crash at startup.

2) Secondary DEX files will be added to the classloader after Multidex is initialized via install(Context) or the other two ways I described in the previous blog article - Multidex in Xamarin.Android.

3) Multidex doesn’t efficiently store classes/references to the max count in each list it creates. It’ll try it’s best however.

Okay cool, we have a rundown of what’s going on, but let’s dig a little deeper to the tooling that generates a DEX list. Let’s introduce our friend dx which is a command line tool to generate respective .dex files. There’s a couple parameters we want to keep in mind.

1) --multi-dex: This will enable multidex and create 1 or more classes{n}.dex files.

2) --main-dex-list=<file>: This will parse through a list of class file names in which classes defined will be put in the classes.dex file.

3) --minimal-main-dex: Certain classes selected by --main-dex-list above will be put in the main DEX list(classes.dex)

Typical Issues

1) Custom Application class is not found on dexPathList:

This is very straight forward now that we know what’s going on with multidex. Simply put, our custom application class is not being put on the main classes.dex list. Therefore it cannot even open the entrypoint of the application nor initialize the secondary DEX lists.

2) Other classes needed at startup are not found on dexPathList:

This is also quite straight forward. You might have a framework dependency such as MVVMCROSS or other items that register at startup which need to be on the main classes.dex list.

How to investigate Multidex issues

There is really one tool that is needed now-a-days to investigate the behavior. That tool is classyshark:

https://github.com/google/android-classyshark

This tool can read either .dex or .apk files. Since we are primarily dealing with .dex files, we can directly drag and drop them into classyshark to see all of the classes listed, and also the total method count.

You could technically go further into reading about dx tooling, but it’s not really worth going that far into unless there’s a critical bug. Google recommends: As a general rule, you should rely on the build tools to call them as needed.

Overriding the main dex list(classes.dex)

Xamarin.Android now offers a simple way to override this list. You can do the following:

1) Create a new Text file in your main application root. (Name it multidex.keep)

2) Set the Build Action to MultiDexMainDexList

3) Include any classes you want on the main dex list inside

Note: It’s always a good idea to see a previous multidex.keep file in your obj\Debug folder for a reference.

I hope this helps!

Multidex in Xamarin.Android

If you’re an Android developer, you’re bound to run into a scenario where multidex will be needed. Simply put, multidex is needed when our DEX limit is met. That limit is ~65k references. For API levels < 21, we need to ensure we can handle this scenario correctly. Note: API 21 or greater handles multidex automatically.

Remember back to the days when Google Play Services was this behemoth package? This made multidex a first class citizen and thus a problem in the long run. However since that, Google has separated the Google Play Services libraries into separate packages to follow a more sane pattern.

Here are a few things that we can do as Android developers to prevent using multidex in our applications:

  1. Only use the libraries based on their functionality. This means you can use specific google play services package for different purposes(Maps, Location, etc). You no longer have to reference a giant package that contains everything even if we only use a subset of it.

  2. Enable Proguard so it can strip our unused code.

However what happens when we run into the limitation where these both do not suffice? This is where multidex comes into play.

Tip: Use various tooling to keep an eye on your DEX method count:

https://github.com/mihaip/dex-method-counts (Old)

https://github.com/google/android-classyshark (New)

Let’s first take a look at how we can implement multidex. Given the following class:

https://developer.android.com/reference/android/support/multidex/MultiDexApplication.html

Off the bat we see three different methods of how we can include this in our applications:

To use the legacy multidex library there is 3 possibility:

  1. Declare this class as the application in your AndroidManifest.xml.

  2. Have your Application extends this class.

  3. Have your Application override attachBaseContext starting with

protected void attachBaseContext(Context base) { super.attachBaseContext(base); MultiDex.install(this); }

Great! We have a starting point. Let’s talk about how this is handled in Xamarin.Android.

Given the two current builds at the time of this post (STABLE - 6.0.X and BETA/ALPHA - 6.1.X). Keep these in mind as I’ll be referencing these later.

Xamarin.Android handles multidex in two different ways. Both of these ways are mentioned above in the possibilities.

  • STABLE - Follows #1 and #2, in which the <application> element will be injected with either mono.android.app.Application, or a Custom Application class that extends mono.android.app.Application.

  • BETA/ALPHA - Follows #1 and #2, in which the <application> element will be injected with either android.support.multidex.MultiDexApplication, or a Custom Application class that extends android.support.multidex.MultiDexApplication.

Now that we know how it’s handled, we can follow a few diagnostic steps to figure out any potential issues.

  • Application.class or CustomApplication.class is not found on the dexPathList:

Typically indicates that our Application or CustomApplication class is being put on a different dex list other than the primary dex list. This can be extremely problematic as it’s typically the entry to our application. Typically the easiest check here is to search for your application string inside of your classes.dex file (obj\Release\android\bin or obj\Release\android depending on the version of Xamarin.Android - STABLE vs. BETA/ALPHA respectfully)

  • Know the version of Android SDK build-tools that you are running.

You can enable diagnostic build output within your IDE to see statements displaying the version of build-tools. Otherwise you can always set a custom build-tools version via the $(AndroidSdkBuildToolsVersion) MSBuild property. (Hint you can add a <AndroidSdkBuildToolsVersion>23.0.3</AndroidSdkBuildToolsVersion> in your .csproj as well)

https://developer.xamarin.com/guides/android/under_the_hood/build_process/#Packaging_Properties

  • My multidex.keep file is empty! What do I do?

This is typically a bug within the Android SDK build-tools. It’s typically best to ensure you’re on the latest version of build-tools, and that you search google for any outstanding issues. I’ve found that on Windows, the multidex.keep file will always generate empty on build-tools <= 23.0.3. However we can fix this by manually changing the invoked .bat file.

We can go into our android-sdk\build-tools\23.0.3 folder here and find a mainClassesDex.bat file. This does just what it’s called; creates the main classes.dex file. We now know that there’s an issue between the time Xamarin invokes this command, and the command itself.

We can take a look at the main bulk of the command here:

if DEFINED output goto redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder "%disableKeepAnnotated%" "%tmpJar%" "%params%"
goto afterClassReferenceListBuilder
:redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder "%disableKeepAnnotated%" "%tmpJar%" "%params%" 1>"%output%"
:afterClassReferenceListBuilder

Sadly this doesn’t work and we need to change the params to actually work:

SET params=%params:'=%  
if DEFINED output goto redirect  
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder %disableKeepAnnotated% "%tmpJar%" %params%  
goto afterClassReferenceListBuilder  
:redirect
call "%java_exe%" -Djava.ext.dirs="%frameworkdir%" com.android.multidex.MainDexListBuilder %disableKeepAnnotated% "%tmpJar%" %params% 1>"%output%"  
:afterClassReferenceListBuilder

Ideally we also want to see the final command to ensure any syntax issues/etc.

  • Is it a bug in Xamarin tooling or the Android SDK?

This can be tricky because with certain scenarios it can totally be Xamarin’s fault. However in most cases, it seems to come down to a bug in the Android SDK, more specifically the build-tools we use to invoke this functionality.

However with the new BETA/ALPHA(Xamarin.Android 6.1.X) builds, we can now create our own custom multidex.keep file via the new Build Action of MultiDexMainDexList. You can now simply create a multidex.keep file in your project, set the build action to MultiDexMainDexList, add the respective classes that you want to keep, and boom! You’re off to the races.

I hope this helps!

WakefulIntentService Xamarin.Android

WakefulIntentService

I’ve seen this problem quite a lot recently…How do I receive notifications when my device is not awake? Is there a way to give an Android device coffee or have it listen to that Avicii song?(https://www.youtube.com/watch?v=IcrbM1l_BoI).

So you may ask: “What does WakefulIntentService do?”

WakefulIntentService keeps a device awake by using the mechanism of WakeLock. A WakeLock is better known as a way to keep the device awake indefinitely until the WakeLock is released somehow.

The good part about this code, is that it has been tested much further than a normal implementation of a WakeLock mechanism.

https://github.com/commonsguy/cwac-wakeful (Original Implementation in Java)

https://github.com/JonDouglas/xamarin-android-tutorials/tree/master/Wakeful

Other Alternatives

You could additionally use the Android Support package’s WakefulBroadcastReceiver, which is a way to trigger work to be done by a broadcast. This however depends completely on the situation.

https://developer.android.com/reference/android/support/v4/content/WakefulBroadcastReceiver.html

Pros

  • Greater flexibility as it’s not strictly tied to an IntentService.
  • Uses one WakeLock per request and therefore is more resilient for potential problems with a static WakeLock.

Cons

  • Can be prone to a leak as the developer must call CompleteWakefulIntent().
  • It is a time-limited WakeLock and therefore will release after one minute. Which limits the flexibility with no overrides.
  • If the process is terminated by anything which causes the service to restart, the restarted service will not be under a WakeLock.

New Blog

I’ve decided to rewrite my blog completely in Jekyll. The reasons being is that my old blog was on Wordpress and took too long to create posts and get new content properly formatted. I’m a huge fan of Markdown, so it seems like a perfect fit. I’ve also moved it to a free host via Github Pages.

Because of the conversion, I’ll only be converting the popular articles on my blog and not any of my personal stuff.