Magento on Kubernetes

With the skyrocketing popularity of online shopping, ensuring that your e-commerce websites run fast and smooth is a must for every successful business. In this guide, we'll show you how to Dockerize a Magento website and run it on Kubernetes, a container orchestration platform that provides stable performance regardless of the workload.

1. Prepare the repo

1. Download Magento v2 sources and create the main repo.

2. Create a new project in Buddy and push your code.

3. Magento requires running some jobs in the cron from time to time (e.g. every minute) for tasks like sending a newsletter or reindexing a database. This means we need to enable it on our own by adding a crontab file to the repo with the following content:

default
* * * * * www-data /usr/local/bin/php /var/www/html/bin/magento cron:run >> /var/www/html/var/log/magento.cron.log 2>&1

The file will execute cron:run job via the Magento CLI. The user that executes it will be added to the Docker image (to be created later on).

4. Copy auth.json.sample to auth.json and fill the placeholders with the keys received from Magento. The keys are used to pull the sources from PHP Composer (Magento repos are private and they are required to pull all the sources).

5. Create a Dockerfile with the following contents:

docker
# image
FROM php:7.1-apache

# envs
ENV INSTALL_DIR /var/www/html

# install composer
RUN curl -sS https://getcomposer.org/installer | php \
&& mv composer.phar /usr/local/bin/composer

# install libraries
RUN requirements="cron libpng-dev libmcrypt-dev libmcrypt4 libcurl3-dev libfreetype6 libjpeg62-turbo libjpeg62-turbo-dev libfreetype6-dev libicu-dev libxslt1-dev" \
 && apt-get update \
 && apt-get install -y $requirements \
 && rm -rf /var/lib/apt/lists/* \
 && docker-php-ext-install pdo_mysql \
 && docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ \
 && docker-php-ext-install gd \
 && docker-php-ext-install mcrypt \
 && docker-php-ext-install mbstring \
 && docker-php-ext-install zip \
 && docker-php-ext-install intl \
 && docker-php-ext-install xsl \
 && docker-php-ext-install soap \
 && docker-php-ext-install bcmath

# add magento cron job
COPY ./crontab /etc/cron.d/magento2-cron
RUN chmod 0644 /etc/cron.d/magento2-cron
RUN crontab -u www-data /etc/cron.d/magento2-cron

# turn on mod_rewrite
RUN a2enmod rewrite

# set memory limits
RUN echo "memory_limit=2048M" > /usr/local/etc/php/conf.d/memory-limit.ini

# clean apt-get
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# www-data should own /var/www
RUN chown -R www-data:www-data /var/www

# switch user to www-data
USER www-data

# copy sources with proper user
COPY --chown=www-data . $INSTALL_DIR

# set working dir
WORKDIR $INSTALL_DIR

# composer install
RUN composer install
RUN composer config repositories.magento composer https://repo.magento.com/

# chmod directories
RUN chmod u+x bin/magento

# switch back
USER root

# run cron alongside apache
CMD [ "sh", "-c", "cron && apache2-foreground" ]

In the Dockerfile, we are using the official PHP image from the Docker Hub in which we install all required libraries, copy all sources, install sources from Composer, and run cron and Apache. All layers are optimized to cache as many things as possible.

2. Create a pipeline that builds the image

1. Add a pipeline, select the branch that you want to Dockerize, and set the trigger mode (manual or on-push)

2. Add the Build Docker Image action and save it as it is (do not change anything at the moment)

   Image loading...

Once we save the action, run the pipeline to check if the Docker image is building successfully. If yes, we can keep going:

Image loading...

3. Push the image to Google Container Registry

1. Log in to GCloud.
2. Add a new project or use an existing one.

3. Create a service account for this project (you can use an existing one if you already have any). To create it, enter the API credentials in that project by clicking Create credentials → Service account key. Next, give it a name and a proper role (Editor is enough, but you can select any role that is able to push to GCR and manage GKE). Once you click Create, GCloud will create the account and send you a JSON file with authorization data:

   Image loading...

4. Go back to Buddy and add a new action at the end of the pipeline: Push Docker Image. Since we've already built the image earlier, leave Image built in the previous action selected. Select Google GCR as the Docker registry, and choose the location (the best idea is that you choose the same location where you want to run the cluster and Magento). In JSON key, paste the content of the file downloaded in step 3.3. In Repository, paste the name of the repo that you created in GCloud (e.g. magento-test-223207) with the image name that should be pushed (e.g. magento):

   default
    magento-test-223207/magento

   In Tags, enter the tag of the image. However, do not select any constant value like latest or dev, because it will result with an issue in the K8s cluster – in Kuberenetes deployment, if the YAML file with the deployment doesn't change (and it won't if the tag remains the same), the platform will not update our image on the cluster. To prevent that, we need to give our image a different tag every time. You don't have to do it manually before every deployment—Buddy has a set of predefined variables that you can use. For example:

   default
    $BUDDY_EXECUTION_ID

   Once you enter it, Buddy will automatically increment the tag with the number of the most recent execution.

   Image loading...

Now it's time to test your pipeline. If we configured everything correctly in our project in GCR, you should see an image with a proper tag:

Image loading...

Image loading...

4. GKE deployment

4.1 Prepare the cluster

The image is created and pushed to the registry. Now it's time to run it on a K8s cluster.

1. If you don't have any K8s cluster yet, you need to create it:

   Image loading...

2. Now, create 2 config files for Kubernetes in your repo:

   • mysql.yaml that will contain MySQL configuration (Magento requires it to work)
   • buddy.yaml that will contain Magento configuration

4.2 Apply MySQL deployment

Commit mysql.yaml to the repository:

yaml
apiVersion: v1
kind: Service
metadata:
  name: magento-mysql
  labels:
    app: magento
spec:
  ports:
    - port: 3306
  selector:
    app: magento
    tier: mysql
  clusterIP: None
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mysql-volumeclaim
  labels:
    app: magento
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: magento-mysql
  labels:
    app: magento
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magento
      tier: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: magento
        tier: mysql
    spec:
      containers:
        - image: mysql:5.6
          name: mysql
          args:
            - "--ignore-db-dir=lost+found"
          env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql
                  key: password
            - name: MYSQL_DATABASE
              value: magento
          ports:
            - containerPort: 3306
              name: mysql
          volumeMounts:
            - name: mysql-persistent-storage
              mountPath: /var/lib/mysql
      volumes:
        - name: mysql-persistent-storage
          persistentVolumeClaim:
            claimName: mysql-volumeclaim

This file consist of 3 components:

• the service that shares MySQL 3306 port to other applications run in the cluster (by default all apps are run in isolation). This type of service does not share the MySQL to the outside, only inside the cluster.
  • PersistentVolumeClaim. In the example, we gave it 10GB, but you can increase it and GKE will automatically create a proper storage volume. This storage is constant so the data won't be lost even when you create a MySQL container afresh.
  • MySQL deployment in which you:
    • use the image from Docker Hub (mysql:5.6)
    • share the 3306 port
    • mount an available volume in the path where MySQL keeps the data /var/lib/mysql
    • fetch the password from the kubernetes secret. We'll show you how to create a secret below.

Such configuration assures that even when the MySQL service stops or the container is killed, we'll still have the data that we saved in MySQL available for the next run. To create the secret password to MySQL, you need a properly configured kubectl:

shell
$ kubectl create secret generic mysql --from-literal=password=YOUR_PASSWORD

Now, we can finally add the Apply Deployment action to the pipeline (you will find it in the Kubernetes section). In the action details, select Google Kubernetes Engine as the cluster type and define the details: account, project, zone and cluster name. Change the Authentication mode to Service account, and paste the content of the JSON file that you downloaded. For the Configuration file, select mysql.yaml:

Image loading...

Now it's time to run the pipeline. If everything has passed successfully, you should see a runnning mysql container with the storage service in GKE:

Image loading...

4.3 Apply Magento deployment

Commit magento.yaml to the repository:

yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: magento
  name: magento
spec:
  type: LoadBalancer
  ports:
    - port: 80
  selector:
    app: magento
    tier: frontend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: magento
  labels:
    app: magento
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magento
      tier: frontend
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: magento
        tier: frontend
    spec:
      containers:
        - image: us.gcr.io/magento-test-223207/magento:$TAG_NAME
          name: magento
          ports:
            - containerPort: 80
              name: magento

The file consist of 2 components:

• A LoadBalancer type of service that shares our application to the outside so that it's available from the internet. Kubernetes assigns it an IP address and maps the IP's port 80 to the port 80 in the container
• Magento deployment in which we use the built image and share the 80 port (it's used by Apache)

In the image deployment, we use the $TAG_NAME variable but as for now nothing is replacing it, which means the deployment will not run if nothing has changed. Instead of this variable, we should use the ID of the execution (BUDDY_EXECUTION_ID). Okay, but how do we replace one variable with another? The answer is pretty easy: the Find & Replace action.

Summing up, add the Find & Replace action that will replace $TAG_NAME in the magento.yaml file with $BUDDY_EXECUTION_ID:

Image loading...

With everything prepared, we can finally add the most important action in the pipeline: Apply deployment. Just like before with MySQL, select Google Kubernetes Engine as the cluster type together with the rest of details: Google account, project, zone and target cluster. Make sure to change the Authentication mode to Service account and paste the content of the JSON file that you downloaded. In Configuration file, however, select magento.yaml:

Image loading...

5. Final tweaks

In its final form, our pipeline should look like this:

Image loading...

1. Let's see if everything's okay. Click the Run button or make a push to the repository to execute the pipeline. If all actions pass correctly, you will see a running container in your Google Cloud console:

   Image loading...

   To locate the address at which your Magento is available, go to the Services tab and look up the endpoint in the Magento line:

   Image loading...

   Once we open it, we should see a familiar Magento site:

   Image loading...

   To finish the installation process for the database, enter the following:

   • Host: magento-mysql
   • Username: root
   • Password: our secret password
   • Database Name: anything (you can leave the default one)
   • Table prefix: anything (you can leave the default one)

   Image loading...

2. At this moment, Kubernetes kills the Magento container and creates a brand new one on every execution. This means we have to go through the installation process every time, which is not very convenient. To prevent that, we need to deliver 2 files that Magento creates after the installation: config.php and env.php.

   • Run the pipeline and go through the installation process.
   • Enter the terminal and run the following command on the configured kubectl: shell $ kubectl get pods

   • The command will display the list of pods that are running in our cluster:

     default
       NAME READY STATUS RESTARTS AGE
       magento-64f7bfc667-cfqmd 1/1 Running 0 5m
       magento-mysql-65886c4f6-4zsqh 1/1 Running 0 18h

   • Now we need to open the terminal directly in the Magento container:

     shell
       $ kubectl exec -it magento-64f7bfc667-cfqmd -- /bin/bash

   • Once inside the container, you can view and copy the content of the 2 files that we need. You can find them in these paths:

     default
       /var/www/html/app/etc/config.php
       /var/www/html/app/etc/env.php

   • Go back to your pipeline in Buddy and switch to the Filesystem tab. Go to this directory:

     default
       app/etc

   • Paste both files or create them via Buddy'a interface:

   Image loading...

3. If you run the pipeline now and open the Magento site, you'll see that you didn't lose your configuration.

4. Dump / Import MySQL database

If you already have a database and you want to restore it, you can easily do that with configured kubectl.

To do that, we need to repeat the steps above, i.e. locate the proper pod in our cluster – this time with MySQL. Once you do that, run the following command in the terminal:

shell
kubectl -i POD_NAME -- mysql -u root -pPASSWORD < DUMP.sql

Where:

• POD_NAME
  • MySQL pod name. You can identify it by running kubectl get pods
• PASSWORD
  • our secret MySQL password
• DUMP.sql
  • path to the dump on our machine

Congratulations, you have just introduced advanced DevOps to your Magento delivery line—an achievement definitely worth showing to your CTO and fellow developers. 🙌