December 10, 2019
An interview assignment: ML image classifier (WhatsThat app)
Project repository.
For how to install app see below "How to install app" section.
Diagram. Data streams overview.
How to install app
Update 10 Dec:
- Implemented unit and instrumented tests
- Added unit test to CI pipeline
- Started adding dockerized android for instrumented tests (problem: server does not provide /dev/kvm, so emulator needs to be tunes to use ARM system image)
- Added user to Jira, Jenkins, Artifactory. Will share credentials tomorrow in a task report.
- Todo: Add Jenkins-slave with kvm to run instrumented tests.
Update 9 Dec:
- Added MVVM with Jetpack to easily bind image classification with UI output
- Added dagger2 to more simple load of components
- TensorFlow model now works fine
- Project repo: https://gitlab.gotalkmobile.com/shamilg1/image-classifier
- Added Jira/Gitlab/Jenkins/Artifactory integrations and set up CI pipeline with CD to Artifactory.
- Added planned tasks to Jira (Kanban-based).
- Basic kotlin CameraX implementation is done.
- Tomorrow todos: set up basic quantized image analysis integration and further experiment with models.
5 Dec 2019. I've got assignment for the interview with a nice startup on ML image classification using existing pre-trained model. The aim of this project is to share my skills and experience in dealing with project, specifically an Android app.
There're some good ML learning materials not tied to specific framework:
WhatsThat app privacy policy:
No user data is collected by app.P.S. Actually it's completely offline app: contains no networking code at all.
Project summary
Diagram. Data streams overview.
Summary of what's done is below, also see (more in repository readme):
- Android app in kotlin with MVVM and JetPack, TensorFlow image classifier for CameraX preview.
- Tests: unit (Robolectric JVM-based) and instrumented (uses Android emulator to run).
- Repository: Gitlab with safe workflow: development is done in feature branch, merge requests to master with Build/Test CI pipeline checks before able to merge to the master branch.
- CI: Gitlab integrated with Jenkins. Build is executed for any push to feature branch.
- Project tracking: Jira with Kanban. All commit messages are automapped to Jira tickets using "smart commit messages". Merge of MR leads to automatic Jira ticket state change (move to Done)
- CD: publishing is done to Artifactory (artefacts repository)
- Set up separate Jenkins slave to run project tests which require Android emulator for instrumented testing.
How to install app
- How to test: send me google account email to be added to internal testing group. Once confirmed a link will be sent and app can be installed. (Sorry for inconvenience, it's Google Play rules).
- How to install (for hackers: developer settings must be enabled and unsafe unknown sources installs option enabled there): download file, add .apk extension and upload to phone with adb install release.apk. Link to the latest master build: in Artifactory.
- Implemented unit and instrumented tests
- Added unit test to CI pipeline
- Started adding dockerized android for instrumented tests (problem: server does not provide /dev/kvm, so emulator needs to be tunes to use ARM system image)
- Added user to Jira, Jenkins, Artifactory. Will share credentials tomorrow in a task report.
- Todo: Add Jenkins-slave with kvm to run instrumented tests.
- Todo: Add some general diagrams to show work flow, compose full task report.
- Added MVVM with Jetpack to easily bind image classification with UI output
- Added dagger2 to more simple load of components
- TensorFlow model now works fine
- Project repo: https://gitlab.gotalkmobile.com/shamilg1/image-classifier
- Todo: Setup a functional test to be run in Jenkins as pre-build step. I plan to add a dockerized android emulator espresso test
Update 7 Dec:
- Todo: Add some general diagrams to show work flow.
- Added Jira/Gitlab/Jenkins/Artifactory integrations and set up CI pipeline with CD to Artifactory.
- Added planned tasks to Jira (Kanban-based).
- Basic kotlin CameraX implementation is done.
- Tomorrow todos: set up basic quantized image analysis integration and further experiment with models.
5 Dec 2019. I've got assignment for the interview with a nice startup on ML image classification using existing pre-trained model. The aim of this project is to share my skills and experience in dealing with project, specifically an Android app.
I know almost nothing on the practical side of ML, so I've started reading.
Starting point is link from the assignment: firebase.google.com/docs/ml-kit/. The aim is to create Android app to classify an image like in the HBO's Silicon Valley's "Hotdog or not" app. Nice one, and a good chance for me to try ML.
From what I see like there're several frameworks available for Android from Google (I was surprised on wide range of them!):
- TensorFlow (for IoT and mobile devices, has pre-trained models)
- ML Kit
- CloudVision (google cloud backed only, has first-1000-images-free period)
- AutoML (this is on training a custom model)
There're some good ML learning materials not tied to specific framework:
- ML Practicum: Image Classification: https://developers.google.com/machine-learning/practica/image-classification
- ML crash course (this is a dependency for first one): https://developers.google.com/machine-learning/crash-course/
TensorFlow
TensorFlow has some pre-trained models listed at: https://www.tensorflow.org/lite/guide/hosted_models. Seems like there's a balance between accuracy and latency (not sure what NNAPI time is).
There's an Android example image classifier example project: https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android. I believe this is a good start to not to waste time on boilerplate code. Still I will need to build CI pipelines and I'd prefer at least some end to end tests, at least one or two.
I've decided to create a public repo project at my own gitlab's instance as I've all the required CI integrations set up already.
As a first step I'm going to try the TF Android example app with some pre-trained tensorflow-lite model.
From what I can compare now TensorFlow lite fits the task, although if it's not I'll switch to next (CloudVision seems to be a good next step to try). In the next posts I'll share some technical decisions made on this project (like CI, repo, tests, etc.).
Labels: android, artifactory, dagger2, docker, espresso, interview assignment, jenkins, jetpack, jira, kotlin, machine learning, mvvm
