Jon Douglas The rantings of a developer

Xamarin.Android - Things

All the Android Things - Weather Station

When Android Things was first announced, my inner inventor came out. I wanted to start prototyping all of these ideas that have been spinning around my head. I wanted to be the next Red Green.

(If you aren’t familiar. He was the main character of The Red Green Show, which was a popular Canadian parody of a typical DIY/home improvement show. In the show, they would have some zany premise of automating things that are often painful like mowing your lawn, rolling down manual car windows, and serving beverages).

Android Things

Android Things provides the same Android development tools that we know and love to work with embedded devices. You can see a quick overview of this here:

There are two main components of the Things Support Library that we need to know before starting a prototype.

Peripheral I/O API

The Peripheral I/O APIs let your apps communicate with sensors and actuators using industry standard protocols and interfaces. The following interfaces are supported: GPIO, PWM, I2C, SPI, UART.

User Driver API

User drivers extend existing Android framework services and allow apps to inject hardware events into the framework that other apps can access using the standard Android APIs.

What you’ll need

Optional: You can also pick up a Pimoroni Rainbow HAT for a plug-and-play buffet of inputs, sensors, etc to explore Android Things.

Getting Started

Once you’ve gathered your materials, you’ll need to flash your device. After you’ve flashed your device, you will need to setup a network connection. You can follow the various instructions for doing this here:

Set Up Your Development Environment

After you’ve flashed your device and have it connected to your network, we want to ensure it’s available via adb. You can use the following command to ensure this:

adb devices

When you bootup the Android Things device, you can either have it connected to an ethernet cord, or to your wireless network. Either way, you will want to use adb connect <ip address> to connect to the device.

Android Weather Station Sample

This sample is an integration of multiple peripheral sensors to analyze and display current weather information.

You can find the source code of this sample here:

https://github.com/JonDouglas/XamarinComponents/tree/master/Android/AndroidThingsContribDrivers/samples/WeatherStation

Sample

Monitor

Displays the current weather situation. Sunny, Cloudy, or Rainy.

Raspberry Pi 3

Displays the current realtime temperature in Celsius. Here’s a video of it in action:

https://www.screencast.com/t/i3vaGchmH7

Summary

Android Things is really fun to tinker around with. Let your inner Red Green or Tim “The Tool Man” Taylor shine!

Xamarin.Android - Book

Xamarin.Android Book

There’s been one thing that Xamarin developers have repeatedly told me in the past:

You should write a book with all of the knowledge you have.

I’ve finally taken this into consideration and decided that I will be writing a complete book on Xamarin.Android.

I hope to help you with the challenges of Android app development using the Xamarin.Android framework whether you are new to app development or already nose deep into a project.

This book will be much different than most Xamarin books in the sense that it will focus purely on the Xamarin.Android platform.

Cover

Subscribe to the email list

One thing I would like to ask each one of you is to subscribe to my email list to get updates on the book and early access to various chapters. I’ve added a field for your feedback as to what you want to see covered in more depth.

You can signup at the following link: Programming Xamarin.Android Email List

Xamarin.Android - UI Performance

Android UI Performance

NOTE: This blog was featured on blog.xamarin.com in which you can view a condensed version here: https://blog.xamarin.com/tips-for-creating-a-smooth-and-fluid-android-ui/

As developers, we want our users to have buttery smooth experiences when using our application. When it comes to UI performance, a buttery smooth experience can be defined as a consistent 60 frames per second(fps). That means that we have to render a frame every 16 ms to achieve this experience.

Note:

You can exceed the 16 ms time to render a frame occasionally, as there are often one or two buffered frames ready to go.

Why 60 fps?

Watch this wonderful video by Google on this topic for a brief introduction:

Why 60 FPS

This gives our whole frame process a little less than 16 ms to fully draw a frame on the screen. If a frame is dropped or delayed, it is known as Jank.

Identifying the problem

Have you ever had an experience like this with your application? Your scrolls are very choppy and your UI doesn’t seem to be that responsive.

Credit: Doug Sillars https://github.com/dougsillars for a great example.

During the time that the application is unresponsive and choppy, if the user attempts to interact with the application and it takes longer than 5000 ms (5 seconds) for the application to respond, the Android OS will give us a lovely message:

This isn’t the best experience we can offer our users.

Digging Deeper

Have you ever seen the following in your adb logcat?

I/Choreographer(1200): Skipped 60 frames!  The application may be doing too much work on its main thread.

This is the Android Choreographer warning you in advance that your application is not performing to the buttery smooth experience we are aiming for. In fact it’s telling you that you are skipping frames and not providing your users a 60 FPS experience. We will need a few tips & tricks to resolve these issues.

Overall Tips & Tricks for UI Performance

  • Measure overall UI performance with Systrace to get a baseline.
  • Use Hierarchy Viewer to identify and flatten view hierarchies.
  • Reduce overdraw by flattening layouts and drawing less pixels on screen.
  • Enable StrictMode to identify and make fewer potentially blocking calls on the main UI thread.

Getting Started

Counting Janky Frames

First you’ll want to get an overview of the total janky frames. After your application has been running and interacted with to reproduce the jank, run the following command:

adb shell dumpsys gfxinfo <PACKAGE_NAME>

Sample Output:

Stats since: 524615985046231ns
Total frames rendered: 8325
Janky frames: 729 (8.76%)
90th percentile: 13ms
95th percentile: 20ms
99th percentile: 73ms
Number Missed Vsync: 294
Number High input latency: 47
Number Slow UI thread: 502
Number Slow bitmap uploads: 44
Number Slow issue draw commands: 135

Numbers never lie, so it’s apparent we have a bit of Janky frames(~9%). Let’s dig in further with Systrace.

Systrace

Systrace gives you an overview of the whole android system and tells you what’s going on at specific intervals of time.

Getting Started

Let’s start a systrace on our device. First open up Android Device Monitor to get started.

Once inside, we see the option to start a Systrace:

We then are prompted with what we would like to trace:

This will generate a trace.html file that will give us information about our system for the trace duration.

EX: Systrace over 30 seconds:

Okay great! But what does this all mean? Let’s take it a step at a time.

Alerts & Frames

Alerts will give you a description of what the current situation is with a respective frame(s). It might let you know that there was a long View.OnDraw() call and it might give you suggestions on how you can fix the relevant frame(s).

You can then dig straight into the frame.

And see how much time spent during each step

Finally you can mark that frame using the m hotkey and see what work is being done on various threads such as various CPU Threads, the UI Thread, and the RenderThread.

This example is showing a Yellow Frame, but we can get a general idea of what an idea performant Frame might look like.

From our Alert, we can see that we might want to avoid significant work in View.OnDraw() or Drawable.Draw(), especially allocations or drawing to Bitmaps. In other words, Google gives us a tip to watch this video:

Avoiding Allocations in onDraw()

For our Frame, we can see that our Adapter.GetView() should recycle the incoming View instead of creating a new one.

Systrace Terms

Frame Color

Green - Great performance

Yellow - Less than ideal performance

Red - Bad performance

Scheduling Delay

Scheduling delays happen when the thread that is processing a specific slice was not scheduled on the CPU for a long amount of time. Thus it takes longer for this thread to fire up and complete.

Wall Duration

The amount of time that passed from the moment a slice is started until it’s finished.

CPU Duration

The amount of time the CPU spent processing that slice.

Overdraw

Debugging overdraw is fairly easy. You can enable this in your Android device’s settings:

Settings -> Developer Options -> Debug GPU overdraw -> Show overdraw areas.

Once enabled, you will see many different colors on your layouts.

  • White - No overdraw
  • Blue - Pixels that are 1x overdrawn
  • Green - Pixels that are 2x overdrawn
  • Pink - Pixels that are 3x overdrawn
  • Red - Pixels that are 4x overdrawn

Bad Layout Performance:

Good Layout Performance:

You can then identify why this layout might be overdrawing so much via a tool like Hierarchy Viewer.

Hierarchy Viewer

The first thing we want to know regarding our View Hierarchy is how deep or nested our layouts are.

Let’s take the previous example of bad layout performance:

We can see that we are 4 layers deep which is less than ideal for our ListView. We really need to flatten this out.

Okay that’s a little better! We are only 3 layers deep now. However it’s still not great. Let’s try to remove one more layer.

Much better! We just optimized our whole view hierarchy and we will reap the performance benefits. Let’s take a look at the overall Layout and Draw timings for proof.

  • Bad Layout - 31 views / Layout: 0.754 ms / Draw: 7.273 ms
  • Better Layout - 26 views / Layout: 0.474 ms / Draw: 6.191 ms
  • Good Layout - 17 views / Layout: 0.474 ms / Draw: 1.888 ms

StrictMode

StrictMode is a very useful tool for battle testing your application. Enabling StrictMode as a means to ensure you are not putting extra work in certain places of your application like disk reads, disk writes, and network calls is ideal for a great user experience. In a nutshell StrictMode does the following:

  1. Logs a message to LogCat under the StrictMode tag
  2. Display a dialog (If PenaltyLog() is enabled)
  3. Crash your application (If PenaltyDeath() is enabled)

This can help you determine what type of policies you’d like to battle test your application with.

Enabling StrictMode in your Android Application

protected override void OnCreate(Bundle bundle)
{
    StrictMode.SetThreadPolicy(new StrictMode.ThreadPolicy.Builder().DetectAll().PenaltyLog().Build());

    StrictMode.SetVmPolicy(new StrictMode.VmPolicy.Builder().DetectLeakedSqlLiteObjects().DetectLeakedClosableObjects().PenaltyLog().PenaltyDeath().Build());

    base.OnCreate(bundle);
}

Summary

There are a plethora of tools available to use on your Xamarin.Android application. Use them to track the important performance-related items about your application such as rendering performance to achieve a buttery smooth 60 fps experience for your customers. Use tools like Systrace, GPU overdraw, Hierarchy Viewer, and StrictMode to pinpoint performance related issues in your application and fix them.

Xamarin.Android - Entity Framework

Preface

Entity Framework has been one of my favorite projects for quite some time. If you didn’t know, Entity Framework split off from the 6.X version to a new re-written 7.X (Core) version, in which the goal is to keep the ORM lightweight and very extensible.

Now the challenge for me has always been:

I want to use Entity Framework in my Xamarin projects.

This hasn’t really been possible for the last 3 years. In fact, I’ve attempted it a few years back and gave up because the tooling was simply just not there. However this is my personal redemption at getting this to work.

Tools

Let’s talk about the tooling needed to make this possible. Consider this more of a high level approach:

  • SQLite Provider- We will need a place to store our data. Seeing that SQLite is a perfect fit for mobile, we will use Entity Framework’s SQLite package:

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Sqlite/

  • Entity Framework Core - We will need an ORM to manage our CRUD operations and database migrations:

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/

Great! Now we’re off to the races.

Getting Started

Creating the NetStandard library

The first thing we’ll do is create a netstandard library. To do this, create a PCL project:

Since this is not a netstandard library quite yet, let’s go ahead and convert that by going to the project’s Properties

Once we have that set, we want to ensure we have the minimum requirements for the two NuGet packages we linked above:

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Sqlite/

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/

By the looks of things, the dependency is netstandard 1.3. So let’s make sure our netstandard project is targeting that version:

Creating the Xamarin.Android project

Nothing too special here, we’re just going to create a File -> New Single-View App (Android) Project:

Adding the NuGet packages

This step is typically easiest to do on the project level rather than the solution level as NuGet is sometimes not that friendly with different project structures(.csproj vs project.json)

Adding the NuGet packages to the netstandard library

Adding the NuGet packages to the Xamarin.Android project

Defining the DbContext

If you’ve used Entity Framework before, you will be very familiar with how we define a DbContext and our underlying Models that define our database schema.

Let’s start with a simple data model that we will call Cat.cs:

    public class Cat
    {
        [Key]
        public int CatId { get; set; }
        public string Name { get; set; }
        public int MeowsPerSecond { get; set; }
    }

Let’s now make sure that this is apart of our DbContext by defining a new context we’ll call CatContext.cs. You may notice that we have a string DatabasePath. We will use this later when we need to tell Entity Framework where to store our Database on disk:

    public class CatContext : DbContext
    {
        public DbSet<Cat> Cats { get; set; }

        private string DatabasePath { get; set; }

        public CatContext()
        {

        }

        public CatContext(string databasePath)
        {
            DatabasePath = databasePath;
        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlite($"Filename={DatabasePath}");
        }
    }

Implementing the Context

First, we need to make sure our Xamarin.Android project is referencing our netstandard library.

Now that we have that, let’s implement our MainActivity.cs with some Entity Framework code!

[Activity(Label = "EntityFrameworkWithXamarin.Droid", MainLauncher = true, Icon = "@drawable/icon")]
    public class MainActivity : Activity
    {
        int count = 1;

        protected async override void OnCreate(Bundle bundle)
        {
            base.OnCreate(bundle);

            // Set our view from the "main" layout resource
            SetContentView(Resource.Layout.Main);

            // Get our button from the layout resource,
            // and attach an event to it
            Button button = FindViewById<Button>(Resource.Id.MyButton);

            button.Click += delegate { button.Text = string.Format("{0} clicks!", count++); };

            TextView textView = FindViewById<TextView>(Resource.Id.TextView1);

            var dbFolder = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
            var fileName = "Cats.db";
            var dbFullPath = Path.Combine(dbFolder, fileName);
            try
            {
                using (var db = new CatContext(dbFullPath))
                {
                    await db.Database.MigrateAsync(); //We need to ensure the latest Migration was added. This is different than EnsureDatabaseCreated.

                    Cat catGary = new Cat() { CatId = 1, Name = "Gary", MeowsPerSecond = 5 };
                    Cat catJack = new Cat() { CatId = 2, Name = "Jack", MeowsPerSecond = 11 };
                    Cat catLuna = new Cat() { CatId = 3, Name = "Luna", MeowsPerSecond = 3 };

                    List<Cat> catsInTheHat = new List<Cat>() { catGary, catJack, catLuna };

                    if(await db.Cats.CountAsync() < 3)
                    {
                        await db.Cats.AddRangeAsync(catsInTheHat);
                        await db.SaveChangesAsync();
                    }

                    var catsInTheBag = await db.Cats.ToListAsync();

                    foreach(var cat in catsInTheBag)
                    {
                        textView.Text += $"{cat.CatId} - {cat.Name} - {cat.MeowsPerSecond}" + System.Environment.NewLine;
                    }
                }

            }
            catch (Exception ex)
            {
                System.Diagnostics.Debug.WriteLine(ex.ToString());
            }
        }
    }

Let’s try to run this code.

Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'no such table: Cats'.

It looks like we’re missing a core Entity Framework feature, and that’s Migrations to create our database schema.

Well shucks…this is awkward. We don’t have a great way to generate Entity Framework Migrations from within a Xamarin.Android project or the netstandard library. Let’s work with a quick workaround by creating a new netcore Console Application so we can generate Migrations.

We need to add the Entity Framework Tools, Entity Framework Core Design, and Entity Framework Core to this project so we can use the command line to generate our Migrations.

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Tools.DotNet

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/

https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Design/

We now need to move over our Cat.cs and CatContext.cs to ensure there’s a DbContext it can generate Migrations for. Your Console App should now look like this:

Now we can generate a schema for our context. Let’s use the new dotnet tooling to do this. Open up a new console in our current Console App directory:

Now we need to generate an initial Migration.

dotnet ef migrations add Initial

This should generate a migration:

Now we need to take the initial migrations generated in the Migrations folder of our project and simply move them over to our netstandard library.

Note: You can simply change the namespaces of these two generated files to the name of your netstandard namespace.

Let’s try running the Xamarin.Android project again and see if we run into any other exceptions

It looks like it worked! Our simple attempt at adding Cat models and retrieving them works!

If we wanted to take a closer look at the SQLite file that gets generated, use an emulator and open up ADM(Android Device Monitor):

Taking a closer look into the data/data/files folder, we will see our Cats.db that we created.

You can now take that file and open it in any SQLite explorer.

Note: I personally use DB Browser for SQLite (http://sqlitebrowser.org/)

Conclusion

Source Code: https://github.com/JonDouglas/EntityFrameworkWithXamarin

It’s been a three year battle with you Entity Framework. However I have to give my thanks to everyone involved in getting Entity Framework to the state it currently is. It’s been so much fun seeing how these projects turn out after a long period of development.

Xamarin.Android - Where Do These Permissions Come From?

Preface

I get this question quite often:

Why does my application have extra permissions added to my AndroidManifest.xml?

Many developers think it’s a problem with Xamarin.Android, but they aren’t aware of what really happens with third party libraries and their applications.

To clear up that muck, let’s first talk about the general process.

Typically when you have an external library, those libraries have their own manifests. Those manifests can have various <uses-permission> elements inside, or they can even specify them inside the AssemblyInfo.cs like the following:

[assembly: UsesPermission(Android.Manifest.Permission.AccessCoarseLocation)]

Either way, these permissions will get inserted into an AndroidManifest.xml at the end of the day for that library and then eventually into your application’s AndroidManifest.xml.

Manifest Merging

What do we know about Android development thus far? Well we know that each .apk file can only have one AndroidManifest.xml defined. Thus that means at the end of the day, everything needs to merge into a master AndroidManifest.xml.

Every application must have an AndroidManifest.xml file (with precisely that name) in its root directory. The manifest file provides essential information about your app to the Android system, which the system must have before it can run any of the app’s code.

https://developer.android.com/guide/topics/manifest/manifest-intro.html

Great! Now we are one step ahead of the game and can understand a little bit about the Android build process. However that brings us to a little rut…Does Xamarin.Android follow the same build process as say native Android?

The answer is no. They both have their own unique build processes and thus not everything we learn about native Android can be converted directly to Xamarin.Android. In this specific case, if we learned about how native Android merges multiple manifest files we would find out that this is done via Gradle and not MSBuild which Xamarin.Android uses under the hood.

Example

Let’s take a File -> New Blank Android Project. Let’s rebuild it from the start and see a final AndroidManifest.xml. We would find this file in the obj\Debug\android\AndroidManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="IDontWantYourPermission.IDontWantYourPermission" android:versionCode="1" android:versionName="1.0">
  <!--suppress UsesMinSdkAttributes-->
  <uses-sdk android:minSdkVersion="16" />
  <uses-permission android:name="android.permission.INTERNET" />
  <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
  <application android:label="IDontWantYourPermission" android:name="android.app.Application" android:allowBackup="true" android:icon="@drawable/icon" android:debuggable="true">
<activity android:icon="@drawable/icon" android:label="IDontWantYourPermission" android:name="md5bdb5a298d3d71fd07b60a60955b14ddd.MainActivity">
  <intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
  </intent-filter>
</activity>
<provider android:name="mono.MonoRuntimeProvider" android:exported="false" android:initOrder="2147483647" android:authorities="IDontWantYourPermission.IDontWantYourPermission.mono.MonoRuntimeProvider.__mono_init__" />
<!--suppress ExportedReceiver-->
<receiver android:name="mono.android.Seppuku">
  <intent-filter>
<action android:name="mono.android.intent.action.SEPPUKU" />
<category android:name="mono.android.intent.category.SEPPUKU.IDontWantYourPermission.IDontWantYourPermission" />
  </intent-filter>
</receiver>
  </application>
</manifest>

We only see two permissions being added by default in a Debug configuration:

  <uses-permission android:name="android.permission.INTERNET" />
  <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

However, now let’s add a NuGet package that we believe might be adding permissions we don’t want in our project. For this example, I’m going to use James Montemagno's http://www.nuget.org/packages/Xam.Plugin.Geolocator

After adding it, rebuilding the solution, and checking the new obj\Debug\android\AndroidManifest.xml:

  <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
  <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

So surely enough we would never have known these items were added to our project unless we checked the final generated AndroidManifest.xml.

Okay…But that doesn’t help me understand where a permission is coming from if my project is huge!

Fair enough. Let’s figure out what we can provide to make this easier for us.

Here’s a few ways we can figure out this information:

1) Search through all third-party dependencies via a decompiler like dotPeek and view the Assembly’s information to see if a permission is being added.

EX: (Using dotPeek)

2) Use a grep tool to search for a uses-permission and UsesPermission string.

EX: (Using grepWin - https://sourceforge.net/projects/grepwin/files/)