PHP lifecycle management at Combell

How do we at Combell go from a release by the PHP maintainers to a deploy on production machines? How involved is such an update and can we go fast if it is really needed.

Testing Brotli with Nginx

Most webbrowsers support brotli for a while now, but outside the big players it’s not so widely adopted, or so it seems. Once you start looking outside the really big websites and sites not behind something like CloudFlare, there’s not that many responses with brotli compressed.

MySQL, local vs remote

The series of Proxying MySQL left one thing unanswered. How large is the difference between using local TCP or the socket. Yesterday I learned about socat, shame on me that I did not knew about this tool, so lets drop another remote test in there as well.

Proxying MySQL, conclusion

After running some tests with different proxies:

What can we decide and how do we will deploy and use MySQL.

Proxying MySQL, WordPress and Magneto performance

We have previously done several synthetic benchmarks. To finalize the tests with different proxies we will test if we see impact on real life - yet demo - applications.

Therefore we have choosen to test with sample e-commerce sites, one based on WordPress, another based on Magento. Note we will not try to compare both platforms but measure the impact of the place of our mysql on the application.

Proxying MySQL, benchmarking on production hardware

When connecting to a remote MySQL server there can be a pretty big performance impact, just due the fact we are using TCP to transfer our data. Where connecting over a socket is usually blazingly fast, introducing TCP to do that might give us a performance penalty on our application. We have seen some information on local tests, but how do these behave in the real world. So we will need to test how these sysbench tests will behave on actual production hardware. How much performance drop we see when we actually connect remotely to MySQL and is the difference between the proxies still pretty big.

Proxying MySQL, setting things up

If we want to host our MySQL dabase on other machines, is there a way to easily passthrough our mysql traffic, and how much performance impact can we expect?

We’ll have to figure that out. But therefore we first have to start by understanding what the proxies can do and how to set those up.

We will try some proxies with sysbench and see what that gives us.

LiteSpeed LiteMage vs Nginx + Varnish cache

There is a enterprise counterpart of OpenLiteSpeed, LiteSpeed. LiteSpeed has the big advantage that you can just point it to your existing Apache httpd configuration and it should all work fine. That is not wat we are going to test. The statement is that LiteSpeed + LiteMage is a lot faster compared to a Varnish cache setup for Magento 2. The added statement is also its a lot easier to setup. We are comparing a paid product with an Open Source product, but they are technically competing in the same space.

OpenLiteSpeed vs Apache HTTP Server vs Nginx

OpenLiteSpeed looks like a nice and fancy webserver, but how does it compare in terms of performance to the webservers we usually use. We are mostly interested in how it compares to Apache httpd and Nginx.

After this blogpost was posted, the nice people of Litespeedtech reached out to verify some things. They found there is a big difference between siege and other stress test tools. So I’ll run the tests again with an updated siege configuration and see if it makes a huge difference. Once the new tests are completed, the blogpost will be updated and thisone will stay available for reference.

Getting started with OpenLiteSpeed

Since we all get used to use one piece of software, Lets stir up the pot. We are mostly using nginx and apache httpd for webserver. But there is a replacement for apache httpd which supports .htaccess files, meaning we can switch very easily to it, called OpenLitesSpeed. OpenLiteSpeed is the free and Open Source “counterpart” of the LiteSpeed server by litespeedtech. It claims to be faster than Nginx and Apache httpd. And as an extra, it can handle .htaccess files, so many applications will work just out-of-the-box.

Kconfig hardening tests

Ever since the repo “kconfig-hardening-check” appeared I was interested to know what the potential impact was on performance when you apply all those changes. You can find the repo on Alexander Popov’s github: kconfig-hardening-check

Toying around with firecracker

Firecracker “Secure and fast microVMs for serverless computing”. That triggers a lot, secure, fast and serverless, so something with containers? So Lets play around with firecracker and see what it can do.

MySQL (or percona) memory usage tests in real life

Following up on what was written about mysql / percona memory usage tests which were basically benchmarks on a local machine. Do these results even hold up with real production data and production usage? Since we already had some issues where the memory consumption was very high, we sort of jumped fast to use jemalloc since that was the preferred way of running mysql. We alreay had experience with jemalloc for other workloads on mysql and there we had good results.

MySQL (or percona) memory usage tests

Recently we did some upgrades at work which were also updating the base OS. In general we did not expect too many issues since we were running tests and the operation of all parts was looking good. After the upgrades we noticed something was off with mysql. Some mysql machines were killing mysqld because of OOM (out-of-memory). You’ll see the kills with journalctl or in your syslog. kernel: [1412367.803012] Killed process 98651 (mysqld) total-vm:61264768kB, anon-rss:25002552kB, file-rss:0kB, shmem-rss:0kB kernel: [1412369.

Postponing maintenance and then doing too much at once

The “Oh I forgot” moment

21 october, late in the afternoon, suddenly it pops in my mind - Damn some letsencrypt certificates are about to expire and I ignored all notifications. That was 17.00h and the certificates were expiring at 19.00h. First ofcourse have diner, do something with the kids, get them to bed, …

So yeah 20.00h when I started looking at it. Somewhere due lack of updates or a configuration bug, the script that was meant to update my letsencrypt certificates failed. And I blatantly ignored multiple notifications the certificates were about to expire. So all entirely my fault.

My browser is not *my* media player

A recent update in Chrome (Blink) based browsers added the functionality to control media playing via the media buttons on your keyboard. As a side effect you can also control media on a remote system if you use something like KDE Connect. But what if you don’t want this behaviour and want your media controls to just control your actual media player?

Is the underlying filesystem performance important if you use containers for everything?

Filesystems I’m only going to talk about Linux filesystems here. In general most people use ext4 for their Linux installations, because that is the default in the biggest distributions. I personally have always been a fan of xfs because I always was under the impression xfs has always been very fast for the things I wanted to do. Another filesystem I very much like is btrfs because of the features it provides, this can be controversial because there are a lot of people who have had some negative experience with btrfs and performance or sudden dataloss.

How I manage Arch Linux updates

My goals

I want to be able to update my machine at any time without having to waste a lot of time waiting for stuff to download. Over the years I’ve had my fair share of small issues occuring when doing an update on a system running a “desktop”. So for a few years now I do my updates when logged out of a “desktop” in a tty.

Arch Linux reinstall script

To suit my personal preferences and diverting package choices compared to stock arch linux I have created a simple reinstall script to suit my needs.


The goal is to have a somewhat uniform way of installing my machines and have full disk encryption for root. Here the unlock key is stored on a portable usb device for additional security. You can argue about the added value over a password, but I like it this way.

The script should also enable me to install a new machine fairly quickly without having to do all the things manually. So if I want to use Deepin desktop, Plasma desktop, i3 or fluxbox, I want to get a working set of packages which I can start working with. Eventually there might be packages I need to do something extra, but I just tried to have a sane default for myself.

Meet Magento NL 2018

Surprise, I went to Meet Magento NL 2018. Actually this was a little surprise for me too. Originally I had submitted a few talks for this conference, but I did not know how it went. Since we already discussed at work to go to DPC 2018 I thought, maybe next year. Around 2 weeks before the event I got an email from Sander telling me something went wrong with the feedback on my proposals and they were offering me a ticket to attend the conference. I was happy with the proposal but still doubted shortly if I would go or not since that would be 2 weeks in a row going away, and I still have work to to in our house. But I really wanted to see some people speak and was interested to hear their experiences. So there I went, to Meet Magento NL 2018.

From Bash to Python for scripting

I am a huge fan of shell scripting. But recently it did give me very unexpected behaviour which lead me to start using python for “shell” scripts.

What happened? I was trying to create helpers that called other helpers and passed the arguments with “$@”. This worked fine for a very long time until you pass along strings like ‘-e “CREATE DATABASE foo”’ to a shell script.

Why I like containers for everything

I want to explain why running everything in containers is an improvement over install all your required software on a server. How containers can help us build better quality software faster.

Self signed multi-domain certificate

When you are developing a complex website with multiple subdomains and full https, it can be hard to mimic it in your development environment. For this purpose we will create a CA we will trust for development and that will allow us to generate multi-domain ssl keys.

Make Opera obey KDE activities

The Problem

When I installed opera-devel on my machine it did not play well with the activities I use in KDE. Opera was available in all my activities at all the time.

SOHO DHCP DNS with dhcpd and bind

Lets explain in clear and short how to setup a dhcp/dns with dhcpd and bind.

The goal is to have a predefined dns where the dhcp connected hosts are automatically added so you get a more convenient way to connect to other machines on your network. Especially not having to remember the ip address of the others.

Starting a blog

In the past i would have said, I have nothing interesting to tell people worthy of putting in a blog. Now I changed my mind and I kinda starting a blog to keep myself from forgetting how I got something done :) This blog is based on sculpin so everything is written in markdown. The choice for a static site generator is fairly simple, blogs are in general not really dynamic content and so it will be FAST :).