Magento on Kubernetes
In 2018 Magento has been rebranded to Adobe Commerce.
This guide is verified to work for Magento v1.0. We do not assume responsibility for compatibility errors that might result from using different software versions.
With the skyrocketing popularity of online shopping, ensuring that your e-commerce websites run fast and smooth is a must for every successful business. In this guide, we'll show you how to Dockerize a Magento website and run it on Kubernetes, a container orchestration platform that provides stable performance regardless of the workload.
Actions used in this guide:
Although there are a lot of Magento versions, in this guide we are going to use Magento v2.
1. Prepare the repo
Download Magento v2 sources and create the main repo.
Create a new project in Buddy and push your code.
Magento requires running some jobs in the cron from time to time (e.g. every minute) for tasks like sending a newsletter or reindexing a database. This means we need to enable it on our own by adding a
crontabfile to the repo with the following content:
* * * * * www-data /usr/local/bin/php /var/www/html/bin/magento cron:run >> /var/www/html/var/log/magento.cron.log 2>&1
The file will execute
cron:run job via the Magento CLI. The user that executes it will be added to the Docker image (to be created later on).
For more info on running cron with Magento, have a look at this article.
auth.jsonand fill the placeholders with the keys received from Magento. The keys are used to pull the sources from PHP Composer (Magento repos are private and they are required to pull all the sources).
Dockerfilewith the following contents:
# image FROM php:7.1-apache # envs ENV INSTALL_DIR /var/www/html # install composer RUN curl -sS https://getcomposer.org/installer | php \ && mv composer.phar /usr/local/bin/composer # install libraries RUN requirements="cron libpng-dev libmcrypt-dev libmcrypt4 libcurl3-dev libfreetype6 libjpeg62-turbo libjpeg62-turbo-dev libfreetype6-dev libicu-dev libxslt1-dev" \ && apt-get update \ && apt-get install -y $requirements \ && rm -rf /var/lib/apt/lists/* \ && docker-php-ext-install pdo_mysql \ && docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ \ && docker-php-ext-install gd \ && docker-php-ext-install mcrypt \ && docker-php-ext-install mbstring \ && docker-php-ext-install zip \ && docker-php-ext-install intl \ && docker-php-ext-install xsl \ && docker-php-ext-install soap \ && docker-php-ext-install bcmath # add magento cron job COPY ./crontab /etc/cron.d/magento2-cron RUN chmod 0644 /etc/cron.d/magento2-cron RUN crontab -u www-data /etc/cron.d/magento2-cron # turn on mod_rewrite RUN a2enmod rewrite # set memory limits RUN echo "memory_limit=2048M" > /usr/local/etc/php/conf.d/memory-limit.ini # clean apt-get RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* # www-data should own /var/www RUN chown -R www-data:www-data /var/www # switch user to www-data USER www-data # copy sources with proper user COPY --chown=www-data . $INSTALL_DIR # set working dir WORKDIR $INSTALL_DIR # composer install RUN composer install RUN composer config repositories.magento composer https://repo.magento.com/ # chmod directories RUN chmod u+x bin/magento # switch back USER root # run cron alongside apache CMD [ "sh", "-c", "cron && apache2-foreground" ]
In the Dockerfile, we are using the official PHP image from the DockerHub in which we install all required libraries, copy all sources, install sources from Composer, and run cron and Apache. All layers are optimized to cache as many things as possible.
2. Create a pipeline that builds the image
- Add a pipeline, select the branch that you want to Dockerize, and set the trigger mode (manual or on-push)
Add the Build Docker Image action and save it as it is (do not change anything at the moment)
Build Docker image action details
Once we save the action, run the pipeline to check if the Docker image is building successfully. If yes, we can keep going:
3. Push the image to Google Container Registry
- Log in to GCloud.
- Add a new project or use an existing one.
Create a service account for this project (you can use an existing one if you already have any). To create it, enter the API credentials in that project by clicking Create credentials → Service account key. Next, give it a name and a proper role (
Editoris enough, but you can select any role that is able to push to GCR and manage GKE). Once you click Create, GCloud will create the account and send you a JSON file with authorization data:
Google Cloud Platform account key
Go back to Buddy and add a new action at the end of the pipeline: Push Docker Image. Since we've already built the image earlier, leave
Image built in the previous actionselected. Select
Google GCRas the Docker registry, and choose the location (the best idea is that you choose the same location where you want to run the cluster and Magento). In JSON key, paste the content of the file downloaded in step 3.3. In Repository, paste the name of the repo that you created in GCloud (e.g.
magento-test-223207) with the image name that should be pushed (e.g.
In Tags, enter the tag of the image. However, do not select any constant value like
dev, because it will result with an issue in the K8s cluster – in Kuberenetes deployment, if the YAML file with the deployment doesn't change (and it won't if the tag remains the same), the platform will not update our image on the cluster. To prevent that, we need to give our image a different tag every time. You don't have to do it manually before every deployment—Buddy has a set of predefined variables that you can use. For example:
Once you enter it, Buddy will automatically increment the tag with the number of the most recent execution.
Push Docker image action details
Now it's time to test your pipeline. If we configured everything correctly in our project in GCR, you should see an image with a proper tag:
Google Cloud Platform container registry
By default, GCR works in private mode (all pushed images are private) and you can't pull images without authorization. For the purpose of this guide, our registry is set to public mode (you can change that in the settings—see the image below). If you want to use GCR in private mode, then in k8 deployment you have to use the
Read more about pulling images from private registries.
Container registry settings
4. GKE deployment
4.1 Prepare the cluster
The image is created and pushed to the registry. Now it's time to run it on a K8s cluster.
If you don't have any K8s cluster yet, you need to create it:
Creating Kubernetes cluster
Now, create 2 config files for Kubernetes in your repo:
mysql.yamlthat will contain MySQL configuration (Magento requires it to work)
buddy.yamlthat will contain Magento configuration
4.2 Apply MySQL deployment
mysql.yaml to the repository:
apiVersion: v1 kind: Service metadata: name: magento-mysql labels: app: magento spec: ports: - port: 3306 selector: app: magento tier: mysql clusterIP: None --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: mysql-volumeclaim labels: app: magento spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: magento-mysql labels: app: magento spec: replicas: 1 selector: matchLabels: app: magento tier: mysql strategy: type: Recreate template: metadata: labels: app: magento tier: mysql spec: containers: - image: mysql:5.6 name: mysql args: - "--ignore-db-dir=lost+found" env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql key: password - name: MYSQL_DATABASE value: magento ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysql-persistent-storage mountPath: /var/lib/mysql volumes: - name: mysql-persistent-storage persistentVolumeClaim: claimName: mysql-volumeclaim
This file consist of 3 components:
- the service that shares MySQL 3306 port to other applications run in the cluster (by default all apps are run in isolation). This type of service does not share the MySQL to the outside, only inside the cluster.
PersistentVolumeClaim. In the example, we gave it 10GB, but you can increase it and GKE will automatically create a proper storage volume. This storage is constant so the data won't be lost even when you create a MySQL container afresh.
- MySQL deployment in which you:
- use the image from DockerHub (
- share the 3306 port
- mount an available volume in the path where MySQL keeps the data
- fetch the password from the kubernetes secret. We'll show you how to create a secret below.
- use the image from DockerHub (
Such configuration assures that even when the MySQL service stops or the container is killed, we'll still have the data that we saved in MySQL available for the next run. To create the secret password to MySQL, you need a properly configured kubectl:
$ kubectl create secret generic mysql --from-literal=password=YOUR_PASSWORD
Click here for more info on configuring kubectl with gcloud
Now, we can finally add the Apply Deployment action to the pipeline (you will find it in the
Kubernetes section). In the action details, select
Google Kubernetes Engine as the cluster type and define the details: account, project, zone and cluster name. Change the Authentication mode to
Service account, and paste the content of the JSON file that you downloaded. For the Configuration file, select
Kubernetes action details
Now it's time to run the pipeline. If everything has passed successfully, you should see a runnning mysql container with the storage service in GKE:
4.3 Apply Magento deployment
magento.yaml to the repository:
apiVersion: v1 kind: Service metadata: labels: app: magento name: magento spec: type: LoadBalancer ports: - port: 80 selector: app: magento tier: frontend --- apiVersion: apps/v1 kind: Deployment metadata: name: magento labels: app: magento spec: replicas: 1 selector: matchLabels: app: magento tier: frontend strategy: type: Recreate template: metadata: labels: app: magento tier: frontend spec: containers: - image: us.gcr.io/magento-test-223207/magento:$TAG_NAME name: magento ports: - containerPort: 80 name: magento
The file consist of 2 components:
- A LoadBalancer type of service that shares our application to the outside so that it's available from the internet. Kubernetes assigns it an IP address and maps the IP's port 80 to the port 80 in the container
- Magento deployment in which we use the built image and share the 80 port (it's used by Apache)
In the image deployment, we use the
$TAG_NAME variable but as for now nothing is replacing it, which means the deployment will not run if nothing has changed. Instead of this variable, we should use the ID of the execution (
BUDDY_EXECUTION_ID). Okay, but how do we replace one variable with another? The answer is pretty easy: the Find & Replace action.
Summing up, add the Find & Replace action that will replace
$TAG_NAME in the
magento.yaml file with
Find and replace action
With everything prepared, we can finally add the most important action in the pipeline: Apply deployment. Just like before with MySQL, select Google Kubernetes Engine as the cluster type together with the rest of details: Google account, project, zone and target cluster. Make sure to change the Authentication mode to
Service account and paste the content of the JSON file that you downloaded. In Configuration file, however, select
Kubernetes action details
5. Final tweaks
In its final form, our pipeline should look like this:
Let's see if everything's okay. Click the Run button or make a push to the repository to execute the pipeline. If all actions pass correctly, you will see a running container in your Google Cloud console:
To locate the address at which your Magento is available, go to the Services tab and look up the endpoint in the Magento line:
Once we open it, we should see a familiar Magento site:
To finish the installation process for the database, enter the following:
- Password: our secret password
- Database Name: anything (you can leave the default one)
- Table prefix: anything (you can leave the default one)
Adding Magento database
At this moment, Kubernetes kills the Magento container and creates a brand new one on every execution. This means we have to go through the installation process every time, which is not very convenient. To prevent that, we need to deliver 2 files that Magento creates after the installation:
- Run the pipeline and go through the installation process.
- Enter the terminal and run the following command on the configured kubectl:
shell $ kubectl get pods
The command will display the list of pods that are running in our cluster:
NAME READY STATUS RESTARTS AGE magento-64f7bfc667-cfqmd 1/1 Running 0 5m magento-mysql-65886c4f6-4zsqh 1/1 Running 0 18h
Now we need to open the terminal directly in the Magento container:
$ kubectl exec -it magento-64f7bfc667-cfqmd -- /bin/bash
Once inside the container, you can view and copy the content of the 2 files that we need. You can find them in these paths:
Go back to your pipeline in Buddy and switch to the Filesystem tab. Go to this directory:
Paste both files or create them via Buddy'a interface:
If you run the pipeline now and open the Magento site, you'll see that you didn't lose your configuration.
Dump / Import MySQL database
If you already have a database and you want to restore it, you can easily do that with configured kubectl.
To do that, we need to repeat the steps above, i.e. locate the proper pod in our cluster – this time with MySQL. Once you do that, run the following command in the terminal:
kubectl -i POD_NAME -- mysql -u root -pPASSWORD < DUMP.sql
POD_NAME - MySQL pod name. You can identify it by running
kubectl get pods
PASSWORD - our secret MySQL password
DUMP.sql - path to the dump on our machine
Congratulations, you have just introduced advanced DevOps to your Magento delivery line—an achievement definitely worth showing to your CTO and fellow developers. 🙌
If you had any trouble on the way, send the URL to your pipeline to firstname.lastname@example.org and we'll help you out.
Customer Success Manager
With Buddy even the most complicated CI/CD workflows take minutes to create
Sign up for Buddy CI/CDStart a free trial
Trusted by customers from 170+ countries