Xamarin.Android - Where Do These Permissions Come From?

Preface

I get this question quite often:

Why does my application have extra permissions added to my AndroidManifest.xml?

Many developers think it’s a problem with Xamarin.Android, but they aren’t aware of what really happens with third party libraries and their applications.

To clear up that muck, let’s first talk about the general process.

Typically when you have an external library, those libraries have their own manifests. Those manifests can have various <uses-permission> elements inside, or they can even specify them inside the AssemblyInfo.cs like the following:

[assembly: UsesPermission(Android.Manifest.Permission.AccessCoarseLocation)]

Either way, these permissions will get inserted into an AndroidManifest.xml at the end of the day for that library and then eventually into your application’s AndroidManifest.xml.

Manifest Merging

What do we know about Android development thus far? Well we know that each .apk file can only have one AndroidManifest.xml defined. Thus that means at the end of the day, everything needs to merge into a master AndroidManifest.xml.

Every application must have an AndroidManifest.xml file (with precisely that name) in its root directory. The manifest file provides essential information about your app to the Android system, which the system must have before it can run any of the app’s code.

https://developer.android.com/guide/topics/manifest/manifest-intro.html

Great! Now we are one step ahead of the game and can understand a little bit about the Android build process. However that brings us to a little rut…Does Xamarin.Android follow the same build process as say native Android?

The answer is no. They both have their own unique build processes and thus not everything we learn about native Android can be converted directly to Xamarin.Android. In this specific case, if we learned about how native Android merges multiple manifest files we would find out that this is done via Gradle and not MSBuild which Xamarin.Android uses under the hood.

Example

Let’s take a File -> New Blank Android Project. Let’s rebuild it from the start and see a final AndroidManifest.xml. We would find this file in the obj\Debug\android\AndroidManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="IDontWantYourPermission.IDontWantYourPermission" android:versionCode="1" android:versionName="1.0">
  <!--suppress UsesMinSdkAttributes-->
  <uses-sdk android:minSdkVersion="16" />
  <uses-permission android:name="android.permission.INTERNET" />
  <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
  <application android:label="IDontWantYourPermission" android:name="android.app.Application" android:allowBackup="true" android:icon="@drawable/icon" android:debuggable="true">
<activity android:icon="@drawable/icon" android:label="IDontWantYourPermission" android:name="md5bdb5a298d3d71fd07b60a60955b14ddd.MainActivity">
  <intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
  </intent-filter>
</activity>
<provider android:name="mono.MonoRuntimeProvider" android:exported="false" android:initOrder="2147483647" android:authorities="IDontWantYourPermission.IDontWantYourPermission.mono.MonoRuntimeProvider.__mono_init__" />
<!--suppress ExportedReceiver-->
<receiver android:name="mono.android.Seppuku">
  <intent-filter>
<action android:name="mono.android.intent.action.SEPPUKU" />
<category android:name="mono.android.intent.category.SEPPUKU.IDontWantYourPermission.IDontWantYourPermission" />
  </intent-filter>
</receiver>
  </application>
</manifest>

We only see two permissions being added by default in a Debug configuration:

  <uses-permission android:name="android.permission.INTERNET" />
  <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

However, now let’s add a NuGet package that we believe might be adding permissions we don’t want in our project. For this example, I’m going to use James Montemagno's http://www.nuget.org/packages/Xam.Plugin.Geolocator

After adding it, rebuilding the solution, and checking the new obj\Debug\android\AndroidManifest.xml:

  <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
  <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

So surely enough we would never have known these items were added to our project unless we checked the final generated AndroidManifest.xml.

Okay…But that doesn’t help me understand where a permission is coming from if my project is huge!

Fair enough. Let’s figure out what we can provide to make this easier for us.

Here’s a few ways we can figure out this information:

1) Search through all third-party dependencies via a decompiler like dotPeek and view the Assembly’s information to see if a permission is being added.

EX: (Using dotPeek)

2) Use a grep tool to search for a uses-permission and UsesPermission string.

EX: (Using grepWin - https://sourceforge.net/projects/grepwin/files/)

Proguard

Why ProGuard?

ProGuard detects and removes unused classes, fields, methods, and attributes from your packaged application. It can even do the same for referenced libraries which can help you avoid the 64k reference limit.

The ProGuard tool from the Android SDK will also optimize the bytecode, remove unused code instructions, and obfuscates the remaining classes, fields, and methods with short names.

There are four optional steps that ProGuard uses:

• In the shrinking step, ProGuard starts from these seeds and recursively determines which classes and class members are used. All other classes and class members are discarded.

• In the optimization step, ProGuard further optimizes the code. Among other optimizations, classes and methods that are not entry points can be made private, static, or final, unused parameters can be removed, and some methods may be inlined.

• In the obfuscation step, ProGuard renames classes and class members that are not entry points. In this entire process, keeping the entry points ensures that they can still be accessed by their original names.

• The preverification step is the only step that doesn’t have to know the entry points.

ProGuard reads input jars in which it will shrink, optimize, obfuscate, and preverify them. ProGuard will then write these results to one or more output jars.

Reference Documentation

Please note only the following steps are run in Xamarin.Android:

The Xamarin.Android ProGuard configuration does not obfuscate the .apk and it is not possible to enable obfuscation through ProGuard even through the use of custom configuration files. Thus Xamarin.Android’s ProGuard will only run the shrinking step.

How does ProGuard work with Xamarin.Android

One important item to know ahead of time before using ProGuard is how it works within the Xamarin.Android build process. This can be thought of in two separate steps:

1. Xamarin Android Linker
2. Proguard

Xamarin.Android Linker

First, the linker employs static analysis of your application to determine which assemblies are actually used, which types are actually used, and which members are actually used. It will always run before the ProGuard step. Because of this, the linker can strip an assembly/type/member that you might expect ProGuard to run on.

You can find more information on this topic here: https://developer.xamarin.com/guides/android/advanced_topics/linking/

Proguard

Secondly, ProGuard will then run and remove unused Java bytecode to optimize the .apk. (shrinking step)

Enabling ProGuard

ProGuard can be enabled by checking the Enable Proguard option inside of your Packaging Properties. You must also ensure your project is set to the Release configuration as the Linker must run in order for ProGuard to run.

Optional: You can add a custom ProGuard Configuration file for more control with the ProGuard tooling. To do this, you can create a new .cfg file and apply the build action of ProguardConfiguration.

Customize which code to keep

For majority of situations, the default ProGuard configuration file that Xamarin.Android provides will be sufficient to remove all-and only-the unused code.

You can find the default ProGuard Configuration File at obj\Release\proguard\proguard_xamarin.cfg if you wanted to see what by default is added to the configuration.

EX:

# This is Xamarin-specific (and enhanced) configuration.

-dontobfuscate

-keep class mono.MonoRuntimeProvider { *; <init>(...); }
-keep class mono.MonoPackageManager { *; <init>(...); }
-keep class mono.MonoPackageManager_Resources { *; <init>(...); }
-keep class mono.android.** { *; <init>(...); }
-keep class mono.java.** { *; <init>(...); }
-keep class mono.javax.** { *; <init>(...); }
-keep class opentk.platform.android.AndroidGameView { *; <init>(...); }
-keep class opentk.GameViewBase { *; <init>(...); }
-keep class opentk_1_0.platform.android.AndroidGameView { *; <init>(...); }
-keep class opentk_1_0.GameViewBase { *; <init>(...); }

-keep class android.runtime.** { <init>(***); }
-keep class assembly_mono_android.android.runtime.** { <init>(***); }
# hash for android.runtime and assembly_mono_android.android.runtime.
-keep class md52ce486a14f4bcd95899665e9d932190b.** { *; <init>(...); }
-keepclassmembers class md52ce486a14f4bcd95899665e9d932190b.** { *; <init>(...); }

# Android's template misses fluent setters...
-keepclassmembers class * extends android.view.View {
   *** set*(***);
}

# also misses those inflated custom layout stuff from xml...
-keepclassmembers class * extends android.view.View {
   <init>(android.content.Context,android.util.AttributeSet);
   <init>(android.content.Context,android.util.AttributeSet,int);
}

However there might be cases where ProGuard might not be able to properly analyze and can potentially remove code your application actually needs.

If this happens, you can add a -keep line to your custom ProGuard configuration file.

EX:

-keep public class MyClass

What command is ProGuard running?

ProGuard is simply a .jar provided with the Android SDK. Thus it invokes a command like the following:

java -jar proguard.jar options ...

The ProGuard Task

The Proguard task is found inside the Xamarin.Android.Build.Tasks.dll assembly. It is apart of the _CompileToDalvikWithDx Target which is apart of the _CompileDex Target.

EX: Default Parameters in a File->New Project

ProguardJarPath = C:\Android\android-sdk\tools\proguard\lib\proguard.jar
AndroidSdkDirectory = C:\Android\android-sdk\
JavaToolPath = C:\Program Files (x86)\Java\jdk1.8.0_92\\bin
ProguardToolPath = C:\Android\android-sdk\tools\proguard\
JavaPlatformJarPath = C:\Android\android-sdk\platforms\android-25\android.jar
ClassesOutputDirectory = obj\Release\android\bin\classes
AcwMapFile = obj\Release\acw-map.txt
ProguardCommonXamarinConfiguration = obj\Release\proguard\proguard_xamarin.cfg
ProguardGeneratedReferenceConfiguration = obj\Release\proguard\proguard_project_references.cfg
ProguardGeneratedApplicationConfiguration = obj\Release\proguard\proguard_project_primary.cfg
ProguardConfigurationFiles

		{sdk.dir}tools\proguard\proguard-android.txt;
		{intermediate.common.xamarin};
		{intermediate.references};
		{intermediate.application};
		;
	
JavaLibrariesToEmbed = C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\MonoAndroid\v7.0\mono.android.jar
ProguardJarInput = obj\Release\proguard\__proguard_input__.jar
ProguardJarOutput = obj\Release\proguard\__proguard_output__.jar
DumpOutput = obj\Release\proguard\dump.txt
PrintSeedsOutput = obj\Release\proguard\seeds.txt
PrintUsageOutput = obj\Release\proguard\usage.txt
PrintMappingOutput = obj\Release\proguard\mapping.txt

EX: Command generated from File -> New Project

C:\Program Files (x86)\Java\jdk1.8.0_92\\bin\java.exe -jar C:\Android\android-sdk\tools\proguard\lib\proguard.jar -include obj\Release\proguard\proguard_xamarin.cfg -include obj\Release\proguard\proguard_project_references.cfg -include obj\Release\proguard\proguard_project_primary.cfg "-injars 'obj\Release\proguard\__proguard_input__.jar';'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\MonoAndroid\v7.0\mono.android.jar'" "-libraryjars 'C:\Android\android-sdk\platforms\android-25\android.jar'" -outjars "obj\Release\proguard\__proguard_output__.jar" -optimizations !code/allocation/variable 

Optional Parameter:

Custom proguard.cfg file with Build Action of ProguardConfiguration. This will add the proguard.cfg file to the ProguardConfigurationFiles parameter.

ProGuard Options

• Input/Output Options
• Keep Options
• Shrinking Options
• Optimization Options (This step is skipped in Xamarin.Android)
• Obfuscation Options (This step is skipped in Xamarin.Android)
• Preverification Options (This step is skipped in Xamarin.Android)
• General Options
• Class Paths
• File Names
• File Filters
• Filters
• Overview of Keep Options
• Keep Option Modifiers
• Class Specifications

Examples of ProGuard Configurations

A simple Android activity

-injars  bin/classes
-outjars bin/classes-processed.jar
-libraryjars /usr/local/java/android-sdk/platforms/android-9/android.jar

-dontpreverify
-repackageclasses ''
-allowaccessmodification
-optimizations !code/simplification/arithmetic

-keep public class mypackage.MyActivity

A complete Android application

-injars  bin/classes
-injars  libs
-outjars bin/classes-processed.jar
-libraryjars /usr/local/java/android-sdk/platforms/android-9/android.jar

-dontpreverify
-repackageclasses ''
-allowaccessmodification
-optimizations !code/simplification/arithmetic
-keepattributes *Annotation*

-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider

-keep public class * extends android.view.View {
public <init>(android.content.Context);
public <init>(android.content.Context, android.util.AttributeSet);
public <init>(android.content.Context, android.util.AttributeSet, int);
public void set*(...);
}

-keepclasseswithmembers class * {
public <init>(android.content.Context, android.util.AttributeSet);
}

-keepclasseswithmembers class * {
public <init>(android.content.Context, android.util.AttributeSet, int);
}

-keepclassmembers class * implements android.os.Parcelable {
static android.os.Parcelable$Creator CREATOR;
}

-keepclassmembers class **.R$* {
public static <fields>;
}

Please note that in these cases above, the Xamarin.Android build process will supply the input, output, and library jars. Thus you can focus on other options like -keep.

Troubleshooting

File Issues

Unknown option '-keep' in line 1 of file 'proguard.cfg'

This issue happens mainly on Windows because the .cfg file has the wrong encoding. You need to ensure it has encoding set to UTF-8 in your favorite text editor.

Proguard Processing Issues

Majority of the issues that can happen when ProGuard is processing can be found here.

ProGuard Versions

You can find all of the ProGuard versions at the SourceForge page.

Note: If you are trying to use ProGuard against Android 7.0, you will need to download a newer version of ProGuard as the Android SDK does not ship a new version that is compatible with JDK 1.8

http://stackoverflow.com/questions/39514518/xamarin-android-proguard-unsupported-class-version-number-52-0/39514706#39514706

You will be able to provide a custom ProGuard path in the future via the following Pull Request:

https://github.com/xamarin/xamarin-android/pull/267

Otherwise you can use this NuGet package to support a new version of proguard.jar.

P/Invoking Native Calls

There was a great question the other day. I figured I’d blog about this for others trying to figure out the answer.

How does one invoke a SIGABRT on a device?

For those unfamiliar with what a SIGABRT is, here’s the definition from wikipedia:

SIGABRT is sent by the process to itself when it calls the abort libc function, defined in stdlib.h. The SIGABRT signal can be caught, but it cannot be blocked; if the signal handler returns then all open streams are closed and flushed and the program terminates (dumping core if appropriate). This means that the abort call never returns. Because of this characteristic, it is often used to signal fatal conditions in support libraries, situations where the current operation cannot be completed but the main program can perform cleanup before exiting. It is used when an assertion fails.

Here’s what the community has to say about it:

http://stackoverflow.com/questions/3413166/when-does-a-process-get-sigabrt-signal-6

http://stackoverflow.com/questions/11161126/what-causes-a-sigabrt-fault

Well now that question becomes a bit morphed into:

How does one invoke a SIGABRT signal in Xamarin on both iOS and Android?

Seeing that we already found out the answer via these stack overflow links, we might want to dig in a little further to the actual potential library call we want to use. In our case, we want to look at abort():

http://www.cplusplus.com/reference/cstdlib/abort/

Aborts the current process, producing an abnormal program termination.

The function raises the SIGABRT signal (as if raise(SIGABRT) was called). This, if uncaught, causes the program to terminate returning a platform-dependent unsuccessful termination error code to the host environment.

Perfect! This is exactly what we need to call into. The next question is…How do we do that?

Based on the Xamarin iOS documentation, we are introduced into the world of P/Invoke(We can assume the same for Xamarin Android as well):

https://developer.xamarin.com/guides/ios/advanced_topics/native_interop/#Accessing_C_Methods_from_C

There’s four steps to doing just this:

• Determine which C function you want to invoke
• Determine its signature
• Determine which library it lives in
• Write the appropriate P/Invoke declaration

Let’s take a look at this first step now.

Determine which C function you want to invoke

We already determined that we need to invoke abort().

Determine its signature

The signature for this call is fairly straight forward. It’s void and it has 0 arguments. We can define the structure as the following:

static extern void abort ();

Determine which library it lives in

This is the tough one. Especially for those of us who have not dwelled into the native library side of things for awhile. Let’s use Github as a tool here to see if there’s previous examples. My first search is using DllImport to see examples of this in the Xamarin.iOS / Mac repository.

https://github.com/xamarin/xamarin-macios/search?p=2&q=DllImport&type=Code&utf8=%E2%9C%93

Unfortunately that’s not what I want. I want to invoke this method from the standard C library. Let’s refine again, this time with the following query: DllImport libc

https://github.com/xamarin/xamarin-macios/search?utf8=%E2%9C%93&q=DllImport+libc&type=Code

Oh perfect! I’m starting to see some libc paths. There’s two that seem to be useful here:

[DllImport ("/usr/lib/libc.dylib")]

and

[DllImport("libc")]

However what about Xamarin.Android? Let’s do the same thing using our refined query of DllImport libc:

https://github.com/xamarin/xamarin-android/search?utf8=%E2%9C%93&q=DllImport+libc

Perfect! It looks like both Xamarin.iOS and Xamarin.Android can use the path of:

[DllImport("libc")]

Note: You can make use of the following command to see the symbol table

nm -a <path to .dylib>

If you need to find dependencies of a .dylib you can use:

otool -L <path to .dylib>

Write the appropriate P/Invoke declaration

Now to figure out “How many licks does it take to get to the Tootsie Roll center of a Tootsie Pop?”

A one…A two-Hoo!…A three…

[DllImport ("libc")]
static extern void abort ();

Android:

Calling this function in the OnCreate method:

11-08 12:48:36.172 D/Mono( 6570): DllImport attempting to load: '/system/lib/libc.so'.
11-08 12:48:36.172 D/Mono( 6570): DllImport loaded library '/system/lib/libc.so'.
11-08 12:48:36.172 D/Mono( 6570): DllImport searching in: '/system/lib/libc.so' ('/system/lib/libc.so').
11-08 12:48:36.172 D/Mono( 6570): Searching for 'abort'.
11-08 12:48:36.172 D/Mono( 6570): Probing 'abort'.
11-08 12:48:36.172 D/Mono( 6570): Found as 'abort'.
11-08 12:48:36.180 I/monodroid-gref( 6570): +g+ grefc 9 gwrefc 0 obj-handle 0x100019/L -> new-handle 0x1004ae/G from thread '(null)'(1)
11-08 12:48:36.212 F/libc( 6570): Fatal signal 6 (SIGABRT), code -6 in tid 6570 (App23.App23)

iOS:

Calling this function in the FinishedLaunching method:

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Triggered by Thread:  6

Big thanks to Brendan Zagaeski for helping make my initial theory turn into a reality and blog post!

Porting Android Libraries to Xamarin.Android

When it comes to Android, there is a vast open source community with thousands of libraries to use inside our Android application.

https://github.com/wasabeef/awesome-android-libraries

https://github.com/codepath/android_guides/wiki/Must-Have-libraries

However not all of these awesome libraries have an equivalent library in the Xamarin community. This of course makes it awkward for us as a developer for multiple reasons:

• Not enough time to write port the respective Java -> C# code
• Not familiar enough with Java/C# to know the language equivalents
• Just want to use the library and not care about the internals

Because of this, this gives us only a few options to use one of these libraries:

1. Port over the Java code to C#
2. Create a Xamarin.Android Binding Project
3. Wait for somebody else to do it (Xamarin Community)

However there are pros/cons of each of these items:

Porting over the Java code to C#:

This is personally my favorite way to create a Xamarin.Android library. You will get to learn how the library was made simply by porting it over.

Pros

• Everything will be in C#/Xamarin.Android and easier to maintain in the future
• Easier to debug
• It is fairly easy to port Java to C# (Guide)

Cons

• It takes awhile to write in terms of keyboard strokes
• Certain language features can be hard to port over (Reflection, Anonymous Inner Classes, Enums, etc)

Create a Xamarin.Android Binding Project:

This scenario basically allows the Xamarin.Android Binding Generator do all the work for you. In most simple cases it works quite well, but in more complicated cases, you might end up pulling out your hair.

Pros

• Easy to generate a quality binding without writing much code
• Ability to transform the binding how you see fit to match C# conventions
• Does not take much time

Cons

• Hard to debug generated code
• Have to learn how to troubleshoot bindings / bindings guide

Wait for somebody else to do it (Xamarin Community)

Waiting on other people is typically a last resort when it comes to deadlines.

Pros

• All the work has been done for you and tested
• Plug and Play
• Officially maintained by somebody

Cons

• Have to wait and rely on other people
• Libraries can be abandoned by maintainers

Overall with these three scenarios laid out, it’s really up to you to make the decision on what might be best for your project’s needs.

Here’s a quick guide to help you find if a library already exists:

1. Search NuGet and Xamarin Components for the library you’re after
2. Search Google for the <Library Name> + Xamarin (Note: Some libraries might be hosted on other providers like CodePlex, GitLab, etc)
3. Search Github for snippets of the Java library to see if there’s any Xamarin counterparts

Multidex in depth and overriding the main dex list in Xamarin.Android

This post is a continuation of Multidex in Xamarin.Android

Today we are going to talk about the basic mechanics of the classes.dex file and how multidex handles more than 65k methods.

The classes.dex file is used as a main dex list in your application. It houses all of the classes needed for your application. An APK requires at least one classes.dex(DEX) file which has all the executable code of our application stored inside. The size of a DEX file’s method index is 16-bit which means (2^16) or 65,536 total references a single dex list can have.

However we run into scenarios where we surpass the 65,536 reference count. This is when we need to enable multidex which gives us overfill classes.dex files in the form of classes{n}.dex where n >= 1.

Now we can have as many classes/references as we desire since our application can grow over time. Let’s talk about some of the disadvantages to this however:

1) Multidex tries it’s best to know what dependencies are needed at startup. Since it only loads the main dex list classes.dex at first, you will need to ensure any startup classes/references are inside the MainDexList(classes.dex) or you will crash at startup.

2) Secondary DEX files will be added to the classloader after Multidex is initialized via install(Context) or the other two ways I described in the previous blog article - Multidex in Xamarin.Android.

3) Multidex doesn’t efficiently store classes/references to the max count in each list it creates. It’ll try it’s best however.

Okay cool, we have a rundown of what’s going on, but let’s dig a little deeper to the tooling that generates a DEX list. Let’s introduce our friend dx which is a command line tool to generate respective .dex files. There’s a couple parameters we want to keep in mind.

1) --multi-dex: This will enable multidex and create 1 or more classes{n}.dex files.

2) --main-dex-list=<file>: This will parse through a list of class file names in which classes defined will be put in the classes.dex file.

3) --minimal-main-dex: Certain classes selected by --main-dex-list above will be put in the main DEX list(classes.dex)

Typical Issues

1) Custom Application class is not found on dexPathList:

This is very straight forward now that we know what’s going on with multidex. Simply put, our custom application class is not being put on the main classes.dex list. Therefore it cannot even open the entrypoint of the application nor initialize the secondary DEX lists.

2) Other classes needed at startup are not found on dexPathList:

This is also quite straight forward. You might have a framework dependency such as MVVMCROSS or other items that register at startup which need to be on the main classes.dex list.

How to investigate Multidex issues

There is really one tool that is needed now-a-days to investigate the behavior. That tool is classyshark:

https://github.com/google/android-classyshark

This tool can read either .dex or .apk files. Since we are primarily dealing with .dex files, we can directly drag and drop them into classyshark to see all of the classes listed, and also the total method count.

You could technically go further into reading about dx tooling, but it’s not really worth going that far into unless there’s a critical bug. Google recommends: As a general rule, you should rely on the build tools to call them as needed.

Overriding the main dex list(classes.dex)

Xamarin.Android now offers a simple way to override this list. You can do the following:

1) Create a new Text file in your main application root. (Name it multidex.keep)

2) Set the Build Action to MultiDexMainDexList

3) Include any classes you want on the main dex list inside

Note: It’s always a good idea to see a previous multidex.keep file in your obj\Debug folder for a reference.

I hope this helps!