Shamil's Blog. Software Engineering Processes, Android, CI, Devops.

Labels: blog

Permanent link. Posted by Shamil Gumirov @ 9:50 PM 0 comments

The new blog is sharing a CI/CD built for my self-hosted services (like Jabber server, Blog, Mail, Jitsi meet, Mumble and others). Currently I use a Gitlab-based deployment with an orchestrator repo, makefile per service and private docker registry for container images. All services are containerized for isolation and ease of moving around. It's grown evolutionary, but resembles a k8s, so I've plan to move to k8s, maybe in end of December.

I've patched blog2md to correctly export comments too, see details. All 220 posts in the hugo-based blog are exported from blogger using this, without much modifications afterwards. Some are still needed, mostly related to preformatted code from pre blocks. I did not made match to blog2md yet and not sure if I will ever as they're pretty minor.

Labels: blog, blogger, CD, deployment, gitlab, hugo, k8s, kubernetes, selfhosted, ungoogle

Permanent link. Posted by Shamil Gumirov @ 3:28 PM 0 comments

It's impossible to reach any exposed privileged port from external net when running podman

sudo podman run -p 80:80 -it nginx

What I was trying to run which needed to listen to privileged ports is a certbot inside docker. Motivation is to avoid installing python dependencies or else downloading and running unknown binary.

Labels: centos, centos8, certbot, docker, letsencrypt, linux, opensource, podman

Permanent link. Posted by Shamil Gumirov @ 2:39 PM 0 comments

Lately I'm trying out Centos 8 and move to openshift with podman and buildah as docker replacements. So far not without interesting problems!

In a freshly installed updated centos8 it's impossible to build any alpine-based repo or even pull alpine itself:

$ podman pull alpine
...
Error committing the finished image: error adding layer with blob "sha256:aad63a9339440e7c3e1fff2b988991b9bfb81280042fa7f39a5e327023056819": Error processing
tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:42 for /etc/shadow): lchown /etc/shadow: invalid argument

# dnf reinstall shadow-utils

$ getcap /usr/bin/newuidmap /usr/bin/newgidmap

/usr/bin/newuidmap = cap_setuid+ep
/usr/bin/newgidmap = cap_setgid+ep

Labels: buildah, centos, centos8, dnf, opensource, podman, yum

Permanent link. Posted by Shamil Gumirov @ 11:20 PM 0 comments

1. Lock account.
2. On log in, check for password and then ask for phone.
3. Accept OTP via any phone. Note: the phone is not stored as security phone! It's only one time random phone.

So I expect more problems with using gmail to send automatic notifications from my small service installations like personal devops. This means it's much better to discover and move to custom mail server from Gmail. Sounds like a paradise: no more worrying on email account locking, no more "look into spam folder". This is how email is supposed to work :)

The only real big problem is that Google is actually implements fake security, with fake like in "fake news". Something in the shell called "security" but not actually that. So it lies its users covering intentions with "security" topic, like "we've blocked your account to increase security". Or wait, that's not account's security, but a Google one (like secure our infrastructure with getting rid of bots)?

Labels: account, email, fake security, google, lock

Permanent link. Posted by Shamil Gumirov @ 4:55 PM 0 comments

The remark is very right, it's from Spotify's videos on the topic:
"healthy culture heals broken process".

So the organizational hierarchy of company values should be:

culture > process > work

The culture cannot be built with any amount of process, so it's on a higher level.

Also the process helps to make less mistakes in work, while not being work itself.

Permanent link. Posted by Shamil Gumirov @ 3:57 PM 0 comments

Also this is a great example of dymanic "web app" without javascript! Yep, sir, anchors only. I really love a pragmatic simplicity of this kind.

Labels: git, no javascript, reference, undo

Permanent link. Posted by Shamil Gumirov @ 4:01 AM 0 comments

$ make build-base && make build && make start

PS I am not a big fan of building in docker, however it's possible using avd-base image from same project. I will cover this later.

Labels: android, docker, emulator

Permanent link. Posted by Shamil Gumirov @ 3:23 AM 0 comments

The good overview of using kubernetes for large finance projects by a brilliant architect Oleg Chunikhin (now CTO at Kublr) and Terry Shea, with whom I was lucky to work during early days of my career:
Making Kubernetes a Reality for Financial Services

Here's a nice description of the key moments on way working on scalable cloud system:
How to sleep at night having a cloud service: common architecture do's

A must-read about choosing a technologies:
Choose boring technology.

Labels: architecture, bookmarks, cloud, kubernetes, scalability, software architecture

Permanent link. Posted by Shamil Gumirov @ 12:05 AM 0 comments

Do not create huge ViewModels, keep it simple. Yes, it's more classes per view, however your support and development costs are closer to linear in time.

Labels: android, architecture, mvvm, software architecture, view, viewmodel

Permanent link. Posted by Shamil Gumirov @ 3:52 PM 0 comments

1. Install latest hugo with snap
2. Find and install theme (with git submodule - see below)
3. Try blog with "hugo server ..."
4. Setup publishing pipeline on blog changes (for example, Jenkins or whatever is preferred).

1. Install blogger_to_hugo migration tool
   
2. (This one is not specific to anyone I suppose. I don't want to experiment with lots of new packages in my main macos system, so installing debian 9 as vbox VM. All further is done there).
3. Run migration from blogger backup with blogger_to_hugo. I got an error:
   

   TypeError: object of type 'NoneType' has no len()

   
   Seems like it expects only the english post titles (or maybe texts in english too) and crashes otherwise. Too bad.
4. Use blog2md which produces a bunch of .md files (not sure about non-english titles, but no crash at least).
5. Install and try to use zzo theme. It turns out that hugo 0.58 is not compatible. Although it's latest in debian sid (unstable) repo!
6. Use snap installer (another repo to think of..): snap install hugo
7. Hmm, zzo theme does still not work showing bunch of "render of 'page' failed..." errors and crashing in the end. Could be a non-english titles.
8. Trying a hugo theme pure.
9. Copy and edit config from example site of a chosen theme.

• Use git submodules for themes if your blog is in git (which itself is a great idea too!) like:

  git submodule add https://github.com/SOME/THEME themes/THEME

• Use snap (it's in bad ones too) - to experiment only.

• TL;DR: always use system packages, do not install binaries.
  I strongly advocate against using a zoo of repo managers (read: snap, pip, etc - whatever non-native to your OS updates subsystem) or binaries to anything running in runtime at server. It takes huge amount of maintenance to track of major updates. Most probably you would miss serious vulnerability at some point. Even old version of packages in debian stable receive security fixes (they are backported), which you would need to do yourself manually for any separate repo manager not added to your cron (most probably manually too).
• Test your hugo+chosen theme with all desired charsets before everything else. It's really hard to believe that modern software written in golang crashes for a non-ascii (read: utf-8 or cp-1251), but that's what happens to hugo with some themes.

• Good old backslash escaping is not what works in yaml out of the box. To escape apostrophe (single quote) use '' (yes, type it twice). Enjoy this good reading before going bed (do not): https://yaml.org/spec/1.2/spec.html

Labels: blog, blogger, debian, go, golang, hugo, vm

Permanent link. Posted by Shamil Gumirov @ 2:29 PM 0 comments

WhatsThat app privacy policy:

No user data is collected by app.

Project summary

Diagram. Data streams overview.

Summary of what's done is below, also see (more in repository readme):

• Android app in kotlin with MVVM and JetPack, TensorFlow image classifier for CameraX preview.
• Tests: unit (Robolectric JVM-based) and instrumented (uses Android emulator to run).
• Repository: Gitlab with safe workflow: development is done in feature branch, merge requests to master with Build/Test CI pipeline checks before able to merge to the master branch.
• CI: Gitlab integrated with Jenkins. Build is executed for any push to feature branch.
• Project tracking: Jira with Kanban. All commit messages are automapped to Jira tickets using "smart commit messages". Merge of MR leads to automatic Jira ticket state change (move to Done)
• CD: publishing is done to Artifactory (artefacts repository)
• Set up separate Jenkins slave to run project tests which require Android emulator for instrumented testing.

• How to test: send me google account email to be added to internal testing group. Once confirmed a link will be sent and app can be installed. (Sorry for inconvenience, it's Google Play rules).
• How to install (for hackers: developer settings must be enabled and unsafe unknown sources installs option enabled there): download file, add .apk extension and upload to phone with adb install release.apk. Link to the latest master build: in Artifactory.

Labels: android, artifactory, dagger2, docker, espresso, interview assignment, jenkins, jetpack, jira, kotlin, machine learning, mvvm

Permanent link. Posted by Shamil Gumirov @ 3:30 AM 0 comments

• https://blog.pragmaticengineer.com/the-product-minded-engineer/
• https://medium.com/@sherifmansour/product-engineers-f424da766871

Labels: engineer, hiring, product, work process

Permanent link. Posted by Shamil Gumirov @ 5:41 AM 0 comments

Labels: android studio, shit

Permanent link. Posted by Shamil Gumirov @ 8:02 PM 0 comments

Labels: devops, diagram, docker, git, gitlab, plantuml, security, uml, work process

Permanent link. Posted by Shamil Gumirov @ 1:33 PM 0 comments

Labels: ads, data leaks, facebook, security, user data

Permanent link. Posted by Shamil Gumirov @ 1:06 PM 0 comments

Labels: arch, centos, debian, java, java 8, laptop, linux, openjdk, opensource, x220

Permanent link. Posted by Shamil Gumirov @ 2:44 PM 0 comments

• If you want to move to Chromium smooth and save your opened tabs, the trick is to copy user profile while Chrome is opened (as Chromium resets some of settings while opening newly copied profile it starts with "Open with new tab" option effectively forgetting all your tabs
• In Chromium passwords cannot be imported. At all. Or saved. Until you delete "Login Data*" files (DB itself and -journal).
• Extensions cannot be set up from chrome store. Download process is manual using download URL below or here with pictures (if you don't mind some Japanese).
• Look through 

https://clients2.google.com/service/update2/crx?response=redirect&prodversion=[PRODVERSION]&acceptformat=crx2,crx3&x=id%3D[EXTENSIONID]%26uc

Labels: chrome, chromium, extensions, google, opensource, privacy, security, ublock

Permanent link. Posted by Shamil Gumirov @ 2:57 PM 0 comments

Flickr

Not sure yet where to host instead (and do I need a photo hosting at all). I seem to not like or need any of them, including instagram, fb, vk.

Thoughts on software subscription model

This year I stopped using Lightroom subscription after paying $180 - I just realized that paying more than infinite license cost before is wrong and I have a kinda mental right (although for sure not any kind of legal one, and for the record I do not do this!) to pirate it ;-).

And don't give me that bullshit on software updates that cost a lot. As a local software user I want stability and predictability more than any updates (OS is completely different substance as it's dealing with networking and security). Oh, I know that software engineers wants salaries to feed their families. But please just let your users decide if they want a new version or stay with this one. Limiting options to "pay for new version or stop using it" is just a wrong dichotomy. Company can hire contractors on monthly basis ;-)

Labels: flickr, photo

Permanent link. Posted by Shamil Gumirov @ 2:40 PM 0 comments

Hey. Stop. If this has nothing to do with your article, why the hell did you add it? I don't care about my bandwidth, but it still consumes screen space (even blocked - it's blank space) and it's noise as author just sincerely wrote.

This shall not be the modern web. Yet Medium is a profitable company! I now still use Blogger which shares founder with Medium. But Blogger was and is not that bad! Just to give some clue of how much noise is on the first page of a regular Medium-based blog here's a screenshot (hint: everything is a noise!):
UPD: typos, formatting.

Labels: blog, blogger, internet, marketing, medium, noise, web

Permanent link. Posted by Shamil Gumirov @ 12:36 AM 0 comments

1. dependency versions chain upgrade: studio -> gradle plugin -> kotlin version
2. solving mysterious problems with desugaring D8 (add "android.enableD8.desugaring=false" to gradle.properties)

I'm not very happy when I'm unexpectedly forced to spend 3h+ on infinite chain of tasks. This repeats itself for most of the Android Studio upgrades.

Labels: android, android studio, kotlin, upgrade

Permanent link. Posted by Shamil Gumirov @ 4:57 AM 1 comments

Labels: business, configuration, hosting, linux, management, server

Permanent link. Posted by Shamil Gumirov @ 7:53 PM 0 comments

Labels: Brian Cantrill, fun, humor, leadership, talk, tech, technology, video

Permanent link. Posted by Shamil Gumirov @ 4:02 AM 0 comments

    dataBinding {
        enabled = true
    }
    kapt {
        javacOptions {
            // Increase the max count of errors from annotation processors.
            // Default is 100.
            option("-Xmaxerrs", 500)
        }
    }

Labels: android, build, data binding, databinding, error, gradle, java, kotlin

Permanent link. Posted by Shamil Gumirov @ 10:39 PM 0 comments

Labels: email, regex, rfc, rfc 822, validation

Permanent link. Posted by Shamil Gumirov @ 3:14 PM 0 comments

Labels: bug, linux, mac, mc, shit

Permanent link. Posted by Shamil Gumirov @ 5:13 AM 0 comments

Labels: autoupdate, bugtracker, bugtracking, jira, linux, manual, no-autoupdate, shit, upgrade

Permanent link. Posted by Shamil Gumirov @ 3:13 PM 0 comments

Of course sometimes companies use any little reason to get more pieces of personal information, like the mobile phone number. For example, it makes absolutely no sense to ask for a phone number after "unusual account activity" has been already detected. Asking the one user who is logging in to provide his phone number (and possibly linking it to the account as 2FA or account recovery channel) actually is very bad idea in this situation! Because if attacker already has password on hand he can also link his phone number thus effectively preventing the account owner from logging into account forever.

Threat modelling can help understand such situations and prevent them.

Labels: account, security, threat model

Permanent link. Posted by Shamil Gumirov @ 1:27 AM 0 comments

Labels: bluetooth, headset, knock-knock, osx, resume, sleep, wakeup

Permanent link. Posted by Shamil Gumirov @ 3:02 AM 0 comments

Labels: linux, security, ssh

Permanent link. Posted by Shamil Gumirov @ 11:17 PM 0 comments

Labels: android, emulator, gui, linux, UI

Permanent link. Posted by Shamil Gumirov @ 8:13 PM 0 comments

Labels: marketing, photo, photography, sensor

Permanent link. Posted by Shamil Gumirov @ 1:02 PM 0 comments

Labels: automated tests, integration tests, memo, tests

Permanent link. Posted by Shamil Gumirov @ 10:16 AM 0 comments

Labels: docker, docker build, file, linux, osx

Permanent link. Posted by Shamil Gumirov @ 4:59 PM 0 comments

Labels: blog, content, homepage, http, httpd, site

Permanent link. Posted by Shamil Gumirov @ 11:51 PM 0 comments

Labels: katakana, linux, smile, unicode

Permanent link. Posted by Shamil Gumirov @ 7:51 PM 0 comments

Labels: cpu, memory, opensource, ram, resources, telegram, winword, word

Permanent link. Posted by Shamil Gumirov @ 3:33 PM 0 comments

Labels: ads, autoplay, chrome, chromium, linux, opensource, patch, video, webkit

Permanent link. Posted by Shamil Gumirov @ 3:23 PM 0 comments

Labels: brave, browser, crypto, linux, opensource, security

Permanent link. Posted by Shamil Gumirov @ 4:06 PM 0 comments

Labels: arch, linux, mate, mate-scresnsaver, opensource, screensaver, xscreensaver

Permanent link. Posted by Shamil Gumirov @ 11:05 PM 0 comments

man pcrepattern

Labels: ag, linux, man, opensource

Permanent link. Posted by Shamil Gumirov @ 7:32 PM 0 comments

$ git clone ssh://servername/reponame.git 
Cloning into 'reponame'...
fatal: '/reponame.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Labels: ssh

Permanent link. Posted by Shamil Gumirov @ 7:50 PM 0 comments

$ ssh ...
The authenticity of host '... (...)' can't be established.
RSA key fingerprint is SHA256:qSj/We80nmhTc8Pc4XPazxlpGrnv95EZNXSZB1YsGl0.

The authenticity of host '...' can't be established.
RSA key fingerprint is 77:5f:3b:1f:84:eb:d2:0c:ca:bd:b3:5d:f7:3c:c3:6f

Grouped hex digits, no trash.
Maybe I'm wrong, but it looks for me that ssh maintainers in this case are blindly following a modern fashion to use sha256 instead of considering which UX leads to less human errors.
I'm not sure if that's somehow connected to using ECDSA keys or not. Anyway this change to sha256 increases human mistakes in using crypto tools like SSH which is very bad.

Labels: ecdsa, md5, opensource, sha, sha256, ssh

Permanent link. Posted by Shamil Gumirov @ 4:54 PM 0 comments

To finish setting up this Microsoft account, 
we just need to make sure this email address is yours.

The thing is that I'm pretty sure I did not create any microsoft emails by myself and this email is a secondary one and I do not use it frequently.

So be careful when using Neocities as they could have some data leak.

Labels: hacking, neocities, spam

Permanent link. Posted by Shamil Gumirov @ 5:15 PM 0 comments

Labels: apache spark, software architecture, software engineering, spark

Permanent link. Posted by Shamil Gumirov @ 4:33 PM 0 comments

Labels: apache cassandra, big data, cassandra, software architecture

Permanent link. Posted by Shamil Gumirov @ 4:31 PM 0 comments

Labels: business logics, data logics, engineering, software, software engineering

Permanent link. Posted by Shamil Gumirov @ 8:51 PM 0 comments

Labels: meetings, work process

Permanent link. Posted by Shamil Gumirov @ 8:13 PM 0 comments

Want to compile small kernel.

As a first step, as I cannot yet compose the working initrd, I will use the existing pmagic iso which is 2.6.34 for i686 and replace kernel image.

As for package dependencies/toolchain - too many, search for error or ask me in comments.

$ git clone -v git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
$ git branch v2.6.34.1
Then patch something:
$ cd linux-stable ; patch -p0 < patch.diff

Patch is below:

Labels: kernel, linux, patch

Permanent link. Posted by Shamil Gumirov @ 4:49 PM 0 comments