Shamil's Blog. Software Engineering Processes, Android, CI, Devops.

Edit: This issue is possibly a known one: see "Podman can not create containers that bind to ports < 1024"

It's impossible to reach any exposed privileged port from external net when running podman

sudo podman run -p 80:80 -it nginx

Setup: dedicated Centos 8, clean install + docker, no firewalld, iptables at "allow-all" default ruleset. With docker however it works without problem.

What I was trying to run which needed to listen to privileged ports is a certbot inside docker. Motivation is to avoid installing python dependencies or else downloading and running unknown binary.

Labels: centos, centos8, certbot, docker, letsencrypt, linux, opensource, podman

Here's what I use for android instrumented tests pipeline: project with Android emulator in Docker image with VNC access. This is new variant of less convenient one (it was forked from Agoda). It wasn't nice to use, but gave me good example. This one is better. Usage:

$ make build-base && make build && make start

. This provides emulator device waiting to run your tests.

PS I am not a big fan of building in docker, however it's possible using avd-base image from same project. I will cover this later.

Labels: android, docker, emulator

Project repository. For how to install app see below "How to install app" section.

WhatsThat app privacy policy:

No user data is collected by app.

P.S. Actually it's completely offline app: contains no networking code at all.

Project summary

Summary of what's done is below, also see (more in repository readme):

• Android app in kotlin with MVVM and JetPack, TensorFlow image classifier for CameraX preview.
• Tests: unit (Robolectric JVM-based) and instrumented (uses Android emulator to run).
• Repository: Gitlab with safe workflow: development is done in feature branch, merge requests to master with Build/Test CI pipeline checks before able to merge to the master branch.
• CI: Gitlab integrated with Jenkins. Build is executed for any push to feature branch.
• Project tracking: Jira with Kanban. All commit messages are automapped to Jira tickets using "smart commit messages". Merge of MR leads to automatic Jira ticket state change (move to Done)
• CD: publishing is done to Artifactory (artefacts repository)
• Set up separate Jenkins slave to run project tests which require Android emulator for instrumented testing.

For my own projects I sometimes use non-standard coding conventions for XML (indents = 2 + special attributes order), and some for Java/Kotlin (indent = 2). However for team workflow I believe it's better to use a standards if there's no team decision otherwise. Also for larger teams >5 people it's better to always stick to standards (helps onboarding).

How to install app

• How to test: send me google account email to be added to internal testing group. Once confirmed a link will be sent and app can be installed. (Sorry for inconvenience, it's Google Play rules).
• How to install (for hackers: developer settings must be enabled and unsafe unknown sources installs option enabled there): download file, add .apk extension and upload to phone with adb install release.apk. Link to the latest master build: in Artifactory.

Update 10 Dec:
- Implemented unit and instrumented tests
- Added unit test to CI pipeline
- Started adding dockerized android for instrumented tests (problem: server does not provide /dev/kvm, so emulator needs to be tunes to use ARM system image)
- Added user to Jira, Jenkins, Artifactory. Will share credentials tomorrow in a task report.
- Todo: Add Jenkins-slave with kvm to run instrumented tests.
Update 9 Dec: 
- Added MVVM with Jetpack to easily bind image classification with UI output
- Added dagger2 to more simple load of components
- TensorFlow model now works fine
- Project repo: https://gitlab.gotalkmobile.com/shamilg1/image-classifier
Update 7 Dec: 
- Added Jira/Gitlab/Jenkins/Artifactory integrations and set up CI pipeline with CD to Artifactory.
- Added planned tasks to Jira (Kanban-based).
- Basic kotlin CameraX implementation is done.
- Tomorrow todos: set up basic quantized image analysis integration and further experiment with models.

5 Dec 2019. I've got assignment for the interview with a nice startup on ML image classification using existing pre-trained model. The aim of this project is to share my skills and experience in dealing with project, specifically an Android app.

Labels: android, artifactory, dagger2, docker, espresso, interview assignment, jenkins, jetpack, jira, kotlin, machine learning, mvvm

For a plantuml nice and simple diagrams text generation tool integration with the gitlab server setup, the plantuml server needs to be exposed to external network. Which is not good. Although some of security risks can be mitigated with running plantuml in docker but definitely not all (such as data exposure, DoS attacks) and mitigating them requires lots of careful manual tuning. Gitlab needs plantuml integration to host rendered images on its side. Maybe aa a plugin. I definitely want to investigate this.

Labels: devops, diagram, docker, git, gitlab, plantuml, security, uml, work process

When including any non-ASCII filenames into image one need to double-check filenames at char level! The reason is that docker can be creative on how to encode char (for example Й, 1081 (single char) vs И, 1080 + 774 (upper part) (two chars!) - docker makes 2 chars from single one). Thanks docker, you gave me 2 days of uking with this trash!

Labels: docker, docker build, file, linux, osx