Let’s Bulky WordPress Project Run On a Cloud With Galera, Nginx, Redis, HaProxy, and Varnish

Photo by Launchpresso on Unsplash

WordPress is an old but handy CMS available still in 2020. I am not a fan of WordPress, and to be honest, I never use it for my personal or professional purposes. However, there are many cases that I see the WordPress is still the best solution for medium and small businesses that are willing to launch their own website in the shortest time. Even in 2020, I still suggest the very WordPress to everyone who doesn’t want to suffer finding developers to create and maintain their websites and spend tiny sums for their “hello world” businesses online.

Amid one of my advisory jobs, I came across an unrelenting situation in which our WordPress-powered website was devouring server resources. We were barely able to serve 100 concurrent users. That was a big failure for my company’s marketing strategies. You are right, we could migrate from WordPress to a much faster technology stack such as Golang and Reactjs, but it was not palpable for the company I was working with. Instead, we went for building up our resources. We had some fair virtual machines and we decided to use them up in a cluster structure. We used 7 machines and I want to share my experience with you in solving our problem.

We use 8 servers with the following details:

  • front.example.com hosts load-balancer (and you can say WAF)
  • wp-1.example.com hosts 1st WordPress codes and its underlying contents equipped with php-fpm engine. This machine has a web and a cache server installed locally.
  • wp-2.example.com bears a replication of WordPress and php-fpm. This one has its own nginx and varnish servers as well.
  • cache.example.com this one hosts redis server and let wp-x servers to read/write their sessions on it.
  • file.example.com: This is where we put our WordPress wp-content folder on. We let wp-x servers to have an access to its content through NFS.
  • db-1, db-2,db-3.example.com: these three servers let would create your Galera cluster and wp-x servers would be able to connect to them.

Ubuntu 18.04 is installed on all 8 hosts. By the way, you can use any OS you want.

Installing haproxy on our front server:

$ sudo apt update
$ sudo apt install haproxy

haproxy, is a wonderfully fast open source load balancer that can be used as the entry point of your cluster. We would expose our front server to the world on the ports 80 and 443.

$ sudo nano /etc/haproxy/haproxy.cfg

In this file, I define a backend and a frontend called wordpress and loadbalancer, respectively.

frontend loadbalancer
bind *:80
bind *:443 ssl crt /etc/haproxy/certs/
http-request redirect scheme https unless { ssl_fc }
reqadd X-Forwarded-Proto:\ http
mode http
default_backend wordpress
backend wordpress
mode http
balance leastconn # or roundrobin
option forwardfor
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
option httpchk HEAD / HTTP/1.1\r\nHost:localhost
server wp-1.example.com [SERVER IP]:80 check
server wp-2.example.com [SERVER IP]:80 check

Side note: I would strongly recommend you to use CertBot to generate your SSL certificates. Create certificates for your domain and its subdomains with a reliable and free tool.

Test your config and restart the haproxy if it goes OK.

$ sudo haproxy -c -f /etc/haproxy/haproxy.cfg
$ sudo systemctl restart haproxy

Setting up Galera cluster on db-1, db-2 and db-3 servers. Galera, by mariadb is a good choice for clustering mysql/mariadb databases. Its installation is pretty straightforward and I invite you to follow this DigitalOcean article to create the cluster of yours.

Setup a cache server

We want to distribute the load of each WordPress server, thus we distribute the incoming requests on wp-1 and wp-2 servers. Each WordPress instance creates sessions for its clients, thus we need to let all the WordPress instances share the sessions with themselves. A cache server is the point wherein all the sessions are stored.

Redis is a fast key-value database that makes it a good choice to be used here. Install it, set a password for it, and bind it to the server IP to able others to connect it.

$ sudo apt install redis redis-server

Go to “/etc/redis/redis.conf”

bind 0.0.0.0 ::1 # find this line and change 127.0.0.1 to 0.0.0.0requirepass yourpassword # find this line, uncomment it

Then restart your redis server, write the following command on terminal:

sudo systemctl restart redis-server

Now, test if your configuration works:

$ redis-cli 
127.0.0.1> auth yourpassword
# OK
127.0.0.1> keys *
# (empty list or set)

So far so good, let’s go to setup the file server.

File Server:

WordPress uses various plugins and we shouldn't bother website admins to install plugins on every WordPress server. Moreover, the uploaded files are costly to save and we should not store them on each WordPress server. Thus, we make use of a shared file sever which uses NFS. Here is a good tutorial to do so:

https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-18-04

WordPress servers:

Here is the tough part, installing WordPress along with varnish, nginx, and php-fpm.

  • Setup database:

Connect to one of databases (db-1 … db-3) and create a user, database and grant remote access to wp-x servers.

CREATE DATABASE wp_database DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;use wp_database;
CREATE USER 'wpuser'@'localhost' IDENTIFIED BY 'password';
grant all on wp_database.* to 'newuser'@'wp-1.example.com';
grant all on wp_database.* to 'newuser'@'wp-2.example.com';

There you go, you have created the user and granted necessary remote access to WordPress servers.

  • Install WordPress:
cd /tmp
curl -O https://wordpress.org/latest.tar.gz
tar xzvf latest.tar.gz
sudo mv wordpress/* /var/www/html/sudo chown -R www-data:www-data /var/www/html
shod chmod -R 755 /var/www/html

Create wp-config.php and enter your database credentials.

cd /var/www/html
sudo cp wp-config-sample.php wp-config.php
sudo nano wp-config.php

It is recommended to create your own WordPress keys. To get your own, run the following command:

curl -s https://api.wordpress.org/secret-key/1.1/salt/

You wil recieve a response like this:

define('AUTH_KEY',         'eeQ=?Jr}FK_Y?O>K+b_+4Jk1P0re-ejVB5.z{ >|0RR8ap@##/=q,A)0:h^L|a3i');
define('SECURE_AUTH_KEY', 'gb.CJt:{aIf1#S)eH`cVpjr,8$qE@a9v=mifpd/;^LPX35i5mwnew^#o%|2t+5>4');
define('LOGGED_IN_KEY', '~Q=wG6D~<{+Z&|,8*wNyj~;x@Knrh_|n,d<8$3B=)PiB~jXTOPW yuJ|Rjs-t8*c');
define('NONCE_KEY', 'BzAF;|VZJo/*0(ZuCrI`0ePwk|[[x?wGCH2xI8Qcf@B,}3c9XMKn9^xB7eU~14J7');
define('AUTH_SALT', 'zo_Y0Cry+u&&JV9E8Xv$m,c5cUb6--:FU0X`f.|$BeSp?()y&#M%I?-)(2dOv::+');
define('SECURE_AUTH_SALT', 'rM`~(^i1|>;=cE!h<_v&@;y+,g?eCOrr5JSWIo>uZYH=QjjR:?^Yhpo:H,pq*,kE');
define('LOGGED_IN_SALT', '4O jUrK:UD&+El?zDa&H[C|>m6t.r;IY);habEh=cwhZS/g?tc08>q@ssa?GVTx*');
define('NONCE_SA

Keep these keys and import it in wp-config.php on all your WordPress instances.

Similarly, enter your database credentials against the following keys in the wp-config.php file.

define('DB_NAME', "wordpress_db");
define('DB_USER', "newuser");
define('DB_PASSWORD', "yourpassword");
define('DB_HOST', "db-1.example.com"); // it can be db-2, db-3
  • Install php-fpm:

Install php-fpm along with the necessary modules for WordPress.

sudo apt install php7.2-fpm php-mysql php-soap php-mbstring php-intl php-xml php-redis php-gd php-xmlrpc

Then, go to your php.ini file and change the following parameters:

sudo nano /etc/php/7.2/fpm/php.ini
  • Firstly, set the php-fpm to use redis to store its sessions:
session.save_handler = redis
session.save_path = "tcp://cache.example.com:6379?auth=yourpassword"

Then, let php-fpm to handle bulky scripts mainly involved with the prominent WooCommerce plugin,

max_execution_time = 60
max_input_time = 60
max_input_vars = 1000
memory_limit = 256M

The above configuration would prevent any future “Server Error” errors caused by php-fpm.

Restart your php-fpm and go to the next step:

sudo systemctl restart php7.2-fpm

Setup Nginx:

Nginx is a very powerful and the most beloved web server among server admins. you can easily install and configure it.

sudo apt install nginx

We will force nginx to listen to port 8080. To do so, edit the default server of

Nginx and paste the following content in it.

server {
listen 8080 default_server;
listen 127.0.0.1:8080 default_server;
root /var/www/html;
index index.php index.html index.htm index.html; server_name _; # your domain if you want to have multi sites location / {
try_files $uri $uri/ /index.php$is_args$args;
}

location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
}
# deny access to .htaccess files, if Apache's document root
location ~ /\.ht {
deny all;
}
}

Then, restart nginx and go for varnish server.

sudo systemctl restart nginx

Install Varnish:

Varnish is a powerful cache server that can be used above nginx webserver. To install it, do as follows:

sudo apt install varnish

Then, make it to listen to port 80. Run the following command and change the default varnish port (6081) to 80.

sudo nano /lib/systemd/system/varnish.service[Unit]
Description=Varnish HTTP accelerator
Documentation=https://www.varnish-cache.org/docs/4.1/ man:varnishd
[Service]
Type=simple
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
ExecReload=/usr/share/varnish/varnishreload
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
[Install]
WantedBy=multi-user.target

Then, set the varnish configuration at “/etc/varnish/default.vcl”. The following configuration works fine with WordPress:

vcl 4.0;# Default backend definition. Set this to point to your content server.
backend default {
.host = "localhost";
.port = "8080";
}
sub vcl_recv {
if (req.method == "PURGE") {
return (purge);
}
if (req.url ~ "&amp;amp;amp;quot;wp-admin|wp-login&amp;amp;amp;quot;") {
return (pass);
}
if ((req.url ~ "^/$" ||
req.url ~ "^/(product-category|product)(/.+)?" ||
req.url ~ "\.(css|js)\?.*" ||
req.url ~ "\.(png|gif|jpg|ico|js|css|xml|pdf|woff|ttf|mp4)$"
) && req.method == "GET") {
# Normalize the query parameters
# set req.url = std.querysort(req.url);
# Strip the cookies
unset req.http.cookie;
return(hash);
} else {
return(pass);
}
}
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
}

Restart your cache server

sudo systemctl restart varnish

Now, you are on the go to launch your distributed WordPress website.

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Google Summer of Code — Week 5 & 6

popshop customer care number//8584892738/8584892738/popshop customer care…

Introduction to Functional Programming with Clojure

CS373 Spring 2022: Tiago Grimaldi Rossi

STYLE Protocol and Koi Metaverse collaboration:

Memory Leak Issues in Java Application -Part 1

Are arrays better than linked lists?

How installing Arch Linux brought my old laptop back to life

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Davood Falahati

Davood Falahati

More from Medium

Deploy Nodejs app on Digital Ocean

Backend Server Hosting Services [Part-1]

Upload Mobile app to BrowserStack with GitHub Actions

Upload mobile app to BrowserStack with GitHub Actions

Let’s Build User Authentication with Express, Prisma, and JWTs