Build all the things! with FAKE
Since Xamarin Evolve this year, I really have been wanting to improve our CI setup at Hitcents after learning about F# build scripts with FAKE. If you have not seen Greg Shackles talk about CI at Xamarin Evolve, I recommend you check it out.
At Hitcents currently, we had been using Jenkins, and setup some complicated build steps in their web UI to build our various projects. Some builds were going to Dropbox, some to TestFlight, and some were even not setup at all for CI--our devs making manual builds. Clicking buttons manually is awful!
A couple of things happened to free up the time to change our build/CI setup in our apps across the board: TestFlight was shutting down and we needed to switch to Hockey, we hit a somewhat slow point in our current projects, and our office flooded… So let’s build all the things!
So our somewhat long list of different things we need to build at Hitcents:
- Xamarin.iOS apps
- Xamarin.Android apps
- Universal Windows apps (Win8 & WinPhone)
- ASP.Net MVC apps running on Windows Azure
- Unity games built for iOS
- Unity games built for Android
- Unity games built as Universal Windows apps (Win8 & WinPhone)
Our deployment requirements are a combination of:
- Upload builds to Hockey
- Archive builds to upload later to an App Store
- Deploy to Windows Azure
- For iOS apps, upload dSYM files to Raygun
And the various SCMs we use are:
- Subversion - the old stuff
- Git - the newer stuff
- PlasticSCM - anything in Unity
And you might say, “that’s a lot of stuff to build…” Which is due to the wide range of types of apps we work on at Hitcents.
So the option I went with is to use TeamCity with a build agent running on a physical Windows machine and another on a physical Mac. Jenkins is OK, but TeamCity just seems like an overall more polished CI software. As for using physical machines, I originally tried a subscription at MacInCloud, but the performance was pretty bad and the limited hard-drive space was really an issue for us. Using their service might work fine for you if you aren’t building games with large amounts of assets. We also have plenty of spare PCs at Hitcents, so a physical Windows machine was just a convenient option.
So why use FAKE? Or why write build scripts in general?
- Running a build script should be really easy, so any developer could do it if an ad-hoc build is needed.
- Having your build in a script, allows you to switch your CI software at will. If Jenkins isn’t working for you, switching to TeamCity is straightforward.
- Your build script is kept in source control. In the case of what we were doing before, our build steps were kept manually in Jenkins, which was a nightmare to maintain.
- Your build scripts in FAKE can take advantage of anything in the .Net BCL. Need to do something crazy with Regex or web requests? No problem.
I had only briefly used F# in the past, but luckily enough, using FAKE is pretty straightforward. There are tons of helper methods in FAKE to make things easier, make sure to check out the full list here.
Setting up TeamCity
Your TeamCity setup is generally not that important, since it will be only running 1 command per build configuration. Use your best judgement on what works for you and your team. In general, some great tips are:
- If your repo is small, set it up to do a completely fresh checkout every time
- If your repo is large, at least setup TeamCity to reset the repository to a “clean” state
- Archive any files you deem important during your build process such as test results files and, of course, your builds
Getting Started with FAKE
To get started with a FAKE script, the general idea is that you would do something like this:
To explain this a little bit, you create a list of functions passed as “targets”. Then at the bottom, you can make them depend on one another. In this case, the server depends on the server-tests target. I’m using a built-in method MSBuild, which works great for building things on Windows.
Do all of this inside a script named build.fsx. To go along with this, I generally include a *.bat file for Windows and a *.sh file for Mac
This also requires NuGet to be installed in the .nuget folder in the root of your repository. It will use NuGet to download FAKE, then run fake.
Usage would be from our example above:
Greg shackles has an excellent starter project for this stuff here. I’d recommend starting there to simplify setting up the details. Once you get one project working, you should be able to just copy-paste a few files to get things going in your next project.
So let’s look at some of the details here, I’ll go through some of our more advanced build scripts with FAKE. Of course, on any of these, I won’t say that this is the absolute *best way* to do all of these things with FAKE, but they work for us. Hopefully this will help other get started with some of these advanced scenarios.
Building Xamarin.iOS apps
FAKE has a pretty good set of helpers for building Xamarin apps. In our case, we are always building for iOS on a Mac, so no Visual Studio pairing is required. In the case of a simple iOS app, generally you do something like this:
And this will run mdtool on a solution file for a particular configuration and then archive the IPA on TeamCity. As for the app store submission process, we generally set both Ad-Hoc and AppStore builds to make ipa packages under Project Options | Build | iOS IPA Options. We archive both in TeamCity, and upload the Ad-Hoc build to Hockey (I’ll show how to do that later). When we are ready for an app store submission, I generally download an IPA directly from TeamCity and then submit it with Application Loader.
Building Xamarin.Android apps
Generally we do Android development at Hitcents on Windows, so likewise we also build our Android apps on Windows. In the case of Android, we just need a single APK that we can distribute via Hockey or Google Play. Here is an example of building a simple Android app with MSBuild on Windows:
So the MSBuild function of FAKE takes in an array of F# tuples, that get passed to MSBuild as properties. You can find these variables documented by Xamarin for Android here.
Building Windows Universal Apps
Very similar to Android, the key was figuring out the magic MSBuild properties:
I then setup TeamCity to archive everything in the build directory at the root of the project. This was simpler than calling TeamCityHelper, since alot of different files are built. Keep an eye on the AppVersion property, we’ll use this later.
Deploying to Windows Azure
We also use MSBuild properties for this, pretty straightforward:
I setup “myprofile” in Visual Studio, and tested it once manually. It will save this file in Properties/PublishProfiles, and one small edit to the XML will store the password for CI to be able to use:
We only use this for deploying to our dev environment, so beware if you need to “more carefully” deploy to production.
Setting version numbers for iOS
So one of the key points of using CI for builds is to tag the build with a build number, for future reference. Let’s walk through how I set that up with FAKE.
TeamCity has an environment variable named BUILD_NUMBER, which is probably the simplest way we could set this up. (If I remember correctly, Jenkins is also similar)
At the top of my FAKE script, I have something like this:
To increase the version number for each app store release, we would increment version to “1.0.1” and “1.0.2” as we go. We would keep the 4th number as the build number from TeamCity.
Generally, I always use a helper method named “Exec” that Greg used in his samples:
We’ll use this throughout the article.
Then to update the Info.plist file on iOS, I use this helper method:
Here we take advantage of the built-in tool, PlistBuddy, which is installed along with Xcode on any OS X Machine.
And invoke the method like this:
UpdatePlist version versionNumber project
Setting version numbers for Android
For Android, we can do something very similar, but instead use a built-in FAKE helper called XmlPoke:
In our case, we invoke it like so:
UpdateManifest version build project
We use the build number as the Android version code, and then the full version number as the version name. Use your own discretion if this works for you.
Setting version numbers for Windows
As a quick workaround, I used this NuGet package to setup the AppVersion MSBuild property we used in the FAKE script earlier. It works by adding this line to your csproj file:
If I have time, I should figure out a simple way to use XmlPoke on Package.appmanifest instead. This would be alot cleaner and not require any NuGet dependencies.
Uploading builds to Hockey
Hockey currently has desktop apps for uploading builds. They come bundled with command-line apps that a CI service could use fairly easily.
On Mac: /usr/local/bin/puck
On Windows: C:\Program Files (x86)\HockeyApp\Hoch.exe
The command line arguments are fairly different, so I use the following FAKE methods to get this working:
To upload a build:
Hockey pathToIPAorAPK hockeyAppId hockeyKey versionNumber
Of course, after I got all this working, Greg in parallel setup FAKE helpers for Hockey. This option is probably cleaner, since it doesn’t require you to install any Hockey helper apps on your build server. Pick the option that works best for you, both seem pretty straightforward.
As for Windows builds, Windows Phone is supported on Hockey *only* if you purchase an enterprise signing certificate. For Windows 8, we just chose “Custom” as the platform on Hockey and zipped up everything (the powershell script, etc.)
Uploading dDYM files to Raygun
This is a somewhat undocumented feature of Raygun, so hopefully this will be pretty cool to those out there. Check out this article on Raygun’s blog, which shows an example of using curl to do this. It emulates a web request that the Raygun web app does, that allows you to upload via the dSYM command line. I got a FAKE version working as follows:
This could be cleaned up a bit, and use HttpClient instead. There are also some weird parameters to be passed in:
Raygun raygunAppId raygunToken (project + "/bin/iPhone/AppStore/YourApp.app.dSYM")
The raygunAppId is a short string value in the URL on the Raygun dashboard as mentioned in the article above. The token, is a standard basic HTTP credential of my username and password on the Raygun website that I generated with postman. This *is* somewhat of a hack, you can see why.
Running NUnit tests
We generally write all shared, cross-platform code for our apps in Visual Studio on Windows. So likewise, all our unit tests are written there as well. We use NUnit and take advantage of async/await where possible, since this can be used in NUnit tests now. The issue with this, is that the version of NUnit that is shipped with Xamarin Studio is not quite up to date and cannot run async tests. If you really need tests to run on a Mac, I would recommend avoiding async in your tests at the moment. You can run async code by calling Wait() on any Tasks and checking the Result value. Another issue, is that you generally need an NUnit test running installed on your CI server, I used NuGet to solve this problem.
My helper method:
The NUnit helper method from FAKE will automatically detect the installed NUnit runner, run the tests, then export the results so that TeamCity will see them automatically. Pretty cool right? To call it, just pass in the path to a DLL file. Sadly, this will not work on a Mac, since the version of NUnit.Runners does not seem to work running on Mono on OS X.
I feel really automated right now. I mean *really* automated. No more manual uploading builds to Hockey, dSYMs to Raygun, or building apps on my machine for the app store. I feel alot better, alot cleaner, and my cholesterol just dropped a few points.
Hopefully this helps some people out there getting started with CI for Xamarin. In my next post, I’ll cover how to do this with Unity apps, which is a bit more complicated and specific to Unity.