Machine Learning on iOS and Android

With the growing popularity of machine learning, artificial neural networks and other forms of artificial intelligence, the urge for ML and AI processing in mobile devices is quite fair.

  In WWDC 2017 Apple announced CoreML framework which excited many developers for using it to build ML applications.But before that also many choices were available such as –

  • Cloud Service( Google Cloud Vision, IBM Watson, Microsoft Azure Cognitive Services, Amazon Rekognition)
  • TensorFlow Mobile
  • MPS graph API
  • Caffe

Except from using a cloud service, all other choices can only be used for predicting certain things based on a model and not training a model on device. We still cannot train a model on a mobile device.Even though the documentation of these frameworks mention that these are ready to use frameworks but one needs to dive a little bit in machine learning before using them especially when one needs to use their own model or use own set of data on pre existing model.  

  • A) CoreML :- Apple provides CoreML framework for iOS application developers which has a large selection of artificial neural network types,enabling developers to experiment with different designs when developing intelligent apps. If one needs to run machine learning algorithms on user’s device than CoreML is the easiest solution.Apple provides a number of models that developers can use to suit their needs. These models are very particular about the inputs they accept, so you’ll need to massage your data into a format that the model understands. Likewise, you have to convert the model’s output into something that your app can use. One can also convert tensorflow models to core ML using Core ML Tools API.

    db81e861-1e06-4d14-8915-90707d9b114c

    (Image Source: developer.apple.com)

    Thus to summarize CoreML properties– it’s easy to use , Comes with a handy converter tool that supports several different training packages,the Core ML API is very basic, it only lets you load a model and run it. There is no way to add custom code to your models, also CoreML is only available for devices running iOS11 or later.

  • B) TensorFlow Mobile :- TensorFlow Mobile an open-source google project, is the most popular option for running on device machine learning algorithms.TensorFlow basically uses computational graphs for machine learning.
    Pros:

    • It is Fairly easy to export your model and load it into your app. With a little boilerplate code , it is similar to CoreML.
    • The number of pre-trained TensorFlow models available is quite large and also you can train your model on server quite easily using TensorFlow and then integrate it on your device.

    Cons:

    • TensorFlow on iOS does not use the GPU, only the CPU. For fairly basic models this is fine but deep learning will be slow.
    • The iOS version of TensorFlow does not support all operations, so your graph may not actually work on iOS.
    • No Swift or pure Objective C API’s available. TensorFlow API’s are written in C++, So one needs to write objective C++ code.
    • The shared binary is quite big, so it will add 10 to 40 MB to your app bundle.


    tflite-architecture

    (Image Source:TensorFlow Documentation)

    Google has also released a developer preview for lighter version of TensorFlow mobile known as TensorFlow lite, which it claims solves the problems of slow performance and binary size of TF mobile. With TensorFlow lite, Google has also worked with chip manufacturers to design processors that are optimized for machine learning, thereby increasing speed and efficiency. On select Android devices, the Interpreter will use the Android Neural Networks API for hardware acceleration, or default to CPU execution if not available.

With the increasing popularity of powerful smartphones and tablets and the advent of in device AI frameworks does mean that the devices are going to be more smarter than they are today. Imagine, you have seen something and don’t know what it’s called , just click a picture of it and your device will tell what it is.

Mayur is a developer @cuelogic. For any questions or comments please reach out to us at info@cuelogic.com