November 13th, 2019 at 11:02:47 AM
App development is hard. Simultaneously developing an app for Android and iOS is harder, but it can be parallelized at the expense of code reuse – a caveat so obvious and seemingly inconsequential that it hardly bears mentioning when considering the differences between the JVM languages used for Android and Apple's amalgam of Objective-C and Swift.
Forgoing code reuse comes naturally in this case. Two experienced software developers can efficiently work on the same feature at the same time within two different codebases. Both developers can look at the same wireframe while building the UI for their respective app. Any application logic that interacts directly with the frameworks unique to Android and iOS is also work that should be done independently. Given the right project, the benefits of this approach can outweigh the potential complications that are absent from a "lead platform" approach.
What complications arise when it is absolutely paramount that both mobile apps share a common codebase for complex application logic? How do you adapt either approach to meet this need?
You might think to yourself, If there's some complex functionality that's already difficult to implement in one app, don't even bother trying to implement it in either of them. Find a middleware solution. Toss it in the cloud. While we're at it, how about a progressive web app that lives in a gussied-up WebView?
What if the overarching purpose of a project requires certain logic to be executed natively on a user's mobile device? What if just one of these shared components is a Java library containing over 41,000 lines of code?
Our team at Software Verde is currently closing out a project that faced these challenges, and J2ObjC was instrumental to our success.
Why We Went with J2ObjC
Originally, our project necessitated the sharing of just a few components between the mobile apps and two Java-based backend applications. There was little concern about accomplishing this in Android; maintaining shared code changes between the backend applications and the app are comfortably managed with scripts and build tools, and libraries like Bouncy Castle can be pulled into the app with Gradle.
How does a small team go about maintaining this consistency on Apple's mobile platform? More importantly, if cryptography constitutes the core of our app, how do we ensure the soundness of the encryption protocols provided by Bouncy Castle on iOS?
Rewriting the necessary Java code in Swift was clearly not an option. Even if we somehow avoided any issues when doing the initial copy by hand, it would have been a nightmare to maintain the Swift codebase when the Java codebase was in constant flux. There also exists a Bouncy Castle library for Dart, which put Google's Flutter on the table as both a solution to this problem and a means of reducing the development time needed for both apps. However, that still left us with thousands of lines of custom Java code that needed to be accurately transpiled or rewritten in Dart.
J2ObjC quickly became our best option. Instead of attempting an "everything and the kitchen sink" approach, J2ObjC focuses on accurately transpiling "application logic and data models" with native Objective-C performance. In other words, it cannot convert your entire Android app into a shiny new iOS app. However, this focus was exactly what we needed to port over computationally intensive code that should not be modified (Bouncy Castle), along with any platform-independent logic that existed within the Android app or Java backend.
Finally, despite the rising prevalence of Dart and Flutter, J2ObjC is still actively maintained by Google Developers. The GitHub project still sees several releases with updates and bug fixes, including a fix for an issue we encountered that was quickly addressed by the project's developers.
With J2ObjC, we had a path to success: implement the core application logic and models in Java, then transpile anything relevant to the iOS app. Everything else could be parallelized, and gridlock between developers would be nonexistent.
Creating a Reusable Transpilation Workflow
Integrating J2ObjC into our workflow presented its own set of challenges. Even though J2ObjC can transpile an entire directory hierarchy of Java source files at the command line, it quickly became apparent that this would not be an infrequent process. Continual changes to the Java codebase would require a re-transpilation of all relevant source code for the iOS app, and it was often advantageous to transpile uncommitted Java code on a local machine.
There was also the matter of amassing all of the necessary Java source files into a single directory path that could be digested by J2ObjC as a command line argument. JAR files needed to be retrieved from four different projects, libraries needed to be downloaded from Maven and GitHub, and specific source files had to be included or excluded before running J2ObjC. This process, along with some twists that will be discussed later, had to be repeatable and consistent for each developer working on the project.
The solution is a fairly long bash script that can be run within the iOS project directory. It corrals the necessary Java source files before transpiling them, removes any previously-transpiled files, then moves the newly created Objective-C files into the correct Xcode project directory. Ultimately, the script allows multiple developers to work from the same codebase while still offering enough flexibility for one developer to transpile any local changes to their Java code.
The ability to quickly and repeatedly transpile multiple Java libraries in a matter of minutes cannot be overstated. Java code could be transformed into working Objective-C at our command, and whoever was tasked with working on the iOS app could focus on writing UI and iOS-specific code. With this workflow in place, J2ObjC had earned our respect as a remarkably practical answer to an incredibly complicated problem. All that remained was adding the transpiled files to the Xcode project.
Build Rules, Bridging Headers, and Wrappers
Xcode must be properly configured before attempting to build and run a J2ObjC project. Adding the build rules recommended by Google is relatively painless one-time process. However, it can be especially painful to maintain the .xcodeproj project file when changes are made by multiple developers in a Git repository.
Xcode does not immediately recognize any files that have been added to the project directory. Consequently, whenever a new Java source file is included in the transpilation process, its Objective-C equivalent must be manually added to the project in Xcode, a step that's easy to forget. Even when you do remember, doing so will generate a unique hash that is added to the .xcodeproj file along with the new file's filepath. Two developers on different branches adding new files to Xcode would result in headache-inducing merge conflicts within the .xcodeproj file, and a busted project file would make Xcode inoperable until the conflicts were resolved.
With smaller projects, it would be trivial to track a handful of changes to the project. Our Xcode project grew to contain over 7,600 .h and .m Objective-C class files that would be replaced every time we ran our J2ObjC script. We quickly adapted to avoid merge conflicts, but build errors were a frequent occurrence whenever new Java source files were transpiled. Given Xcode's inability to detect newly added or removed files, removing and re-adding the parent directory containing all of the Objective-C files emerged as the safest (but most time-consuming) method of silencing Xcode's build errors.
Including the Objective-C files in the project is not the final step. A bridging header file is needed to expose the imported Objective-C code in Swift. This file contains import statements for each Objective-C class that will be referenced in Swift, as well as any classes that are referenced in the method calls of the aforementioned classes. Our team experienced some growing pains in maintaining the bridging header; it was a common source of build errors, and mysteriously unavailable class properties and methods in Swift often led to a frustrating realization: something's missing from the bridging header.
Once any bridging header issues have been corrected, the project will build without issues, and the transpiled Objective-C code will become fully usable in Swift – assuming you don't mind typing out class names like ComSoftwareverdeBitcoinSecp256k1SignatureSignature with Xcode's infuriatingly finicky code autocompletion. To be fair, J2ObjC's naming convention for transpiled classes includes the full package name, and the transpiled files are not intended to be modified. As such, yet another task was appended to our transpilation workflow: write Swift wrapper classes for any transpiled Objective-C components that would be directly referenced in Swift code. The extra effort brought us sanity in the form of shorter class names while also allowing us to prune unnecessary methods and implement Objective-C interfaces as classes in Swift.
At last, our Java code was usable in Swift! Initial unit tests were promising, and maintaining feature-parity with the Android app no longer seemed like an impossibility. As long as everything worked in Java, it should "just work" in iOS, right?
It Just Works™
Understandably, we ran into a few roadblocks throughout the project when porting features into the iOS app. Objective-C purposefully does not lend itself to exception handling in the same way it is used in Java. Instantiating NSSetUncaughtExceptionHandler within the AppDelegate class provided insight into exceptions that are safely handled in Java but fatal in iOS.
NSSetUncaughtExceptionHandler won't catch everything, however. For example, a method in one of our Java API classes would not return if it threw an exception. In Java, this is a non-issue if the exception is appropriately handled elsewhere. In Objective-C with ARC disabled, a EXC_BAD_ACCESS error would occur on a background thread whenever an API call failed to retrieve a response, immediately crashing the app. Needless to say, a dangling pointer was the source of our torment, and it occurred whenever the transpiled Objective-C code threw an exception during the API call. More information about exception handling and its potential for memory issues can be found in Apple's Documentation Archive.
We addressed these issues with a combination of minor changes to our modifiable Java source code and the addition of Java wrapper classes that would only be used in the transpiled code. Whenever these dangling pointers and data race conditions popped up, one or two of our developers could handle it within a few hours at most. Little did we know that this was only the beginning of our iOS memory-related woes.
Out of Time and Memory
Our project required RSA key pairs and PGP keyrings to be generated locally on a user's mobile device. Using Bouncy Castle in Android, we were able to generate a keyring within 3-5 seconds on a Pixel 2 device. Not bad, but a clear sign that this was a computationally intensive process. How would it fare on iOS?
The transpiled Bouncy Castle code took anywhere from 25-45 seconds to generate the necessary RSA key pairs on an iPad Pro. Meanwhile, the app's memory usage would skyrocket from ~50MB to over 300MB during this process; still, this was far less concerning than the app's workflow careening to a screeching halt whenever a feature required a new PGP keyring. Several features relied on PGP keyrings, and repeatedly staring at an activity indicator for 30 seconds was unacceptable. We implemented a caching system for generating PGP keyrings in the background as a temporary solution. Bear in mind, we wanted to avoid modifying the Bouncy Castle library at all costs, and we were determined that we could look into potential optimizations further down the road.
It wasn't long until we hit another wall. Remember that Java library with over 41,000 lines of code? It was Bitcoin Verde, along with a massive companion wrapper class that acted as the intermediary between Bitcoin Verde and an Android app. Integrating this functionality into the current project's Android app required little effort. Transpiling Bitcoin Verde for iOS required several modifications to our J2ObjC script, but soon enough, we were able to wrap its functionality in Swift code.
For the purposes of this project, Bitcoin Verde was configured to read in a 46 MB file as part of a one-time bootstrapping process for its database. This process would take a minute or two in Android, but it was neither memory nor computationally intensive enough to be a burden. Starting the iOS app with Bitcoin Verde enabled triggered an eruption of memory leaks; the app's memory consumption jumped from ~50 MB to 2 GB in seconds, almost immediately crashing the app at startup.
Was this still going to work?
What Went Wrong
You may recall that "ARC" was previously mentioned when discussing the memory issues with exception handling. ARC stands for Automatic Reference Counting, and it is Apple's solution for removing the burden of memory management from developers. By default, it is enabled for Objective-C and Swift projects in Xcode, but J2ObjC strongly recommends disabling it. This is because J2ObjC explicitly handles how memory is retained and released in transpiled Objective-C code. Per the J2ObjC documentation, enabling ARC introduces a performance cost and the added detriment of leaking memory when exceptions are thrown.
J2ObjC does its best to adhere to Apple's best practices for memory management. Generally, it does an amazing job determining when to retain and release objects from memory. However, it does not always succeed – code that does not leak any memory in Java can leak when transpiled into Objective-C, especially when it includes loops that repeatedly create complex objects or certain quality of life features that are common in Java code.
Loops within Bouncy Castle and the Bitcoin Verde bootstrapping process were the culprits. Hours of debugging determined that local variables were being retained or rapidly duplicated in memory after multiple iterations and would not be released until the loops terminated. In the case of Bouncy Castle, BigInteger values were being retained within a multi-level nested for-loop, and the bootstrapping process was likely replicating large chunks of that 46 MB file in memory over thousands of iterations.
Meanwhile, the Bitcoin Verde wrapper class frequently made use of lambdas when defining anonymous classes in a heavily multi-threaded environment. This led to a myriad of circular references that made it impossible to fully release the Bitcoin Verde wrapper object.
Now that we had identified the problems, how would we mitigate these issues without making major modifications to the Java source code or the ephemeral Objective-C classes?
Annotations to the Rescue
Thankfully, the J2ObjC project also includes a Java annotations library that is useful in squashing memory-related bugs. For our modifiable source code, we added @AutoReleasePool annotations to any troublesome loops, and doing so made quick work of the bootstrapping issue. @WeakOuter was useful for defining callbacks and anonymous classes in the Bitcoin Verde wrapper class, ending the circular references that prevented us from releasing the Bitcoin Verde object.
As for our non-modifiable code in the form of custom or external Java libraries, a recently added feature to J2ObjC allows an external annotations file to be supplied at the command line when transpiling Java code. By adhering to the correct file format, annotations can be added to the target Java source code just before it is transpiled. Once this functionality was added to our J2ObjC build script, we had a consistent and reliable process for addressing any remaining memory issues.
It should be noted that these annotations should not be used lightly – tacking on @AutoReleasePool and @Weak with reckless abandon is a surefire way to run into more EXC_BAD_ACCESS errors, as it will have the undesired effect of releasing object from memory before they are needed in certain situations.
We knew going into this project that taking this approach would have its risks. We anticipated a learning curve with establishing the initial workflow of transpiling and wrapping Java code into usable Swift. The harrowing memory issues caught us off guard, but at the end of the day, it came down to learning more about the strengths and weaknesses of J2ObjC until we mastered the processes that brought us consistent success.
J2ObjC gave us a shared codebase between pure Java, Android, and iOS. As a result, just two developers on our team were able to simultaneously deliver a completed, polished Android and iOS app – on time and under-budget.
With the process down, would we do it all again? It depends on the project. It is hard to imagine building another multi-platform app with nearly as much complexity. Something simpler is better suited to Flutter, or perhaps it is better to stick with one platform for a client's proof of concept or MVP. As long as Objective-C and Java are relevant, however, J2ObjC will remain an excellent option for sharing Java application logic across multiple platforms.