Introduction: Varnish Cache for the Fastest Drupal 8 Sites
This Tutorial will show how to install Varnish Cache on a VPS hosting plan to run a Drupal 8 website at high performance, including the configuration of required modules.
There are multiple versions of Varnish Cache available where this guide will use the 4.x platform as the recommended installation on LAMP servers with any Linux distribution.
There are competing Drupal 8 cache management modules available with Purge, Varnish Purge, & Acquia Purge used for Cache Tag systems on production websites.
Step One: Install Varnish Cache on the Web Server
The prerequisites for this Tutorial are a working Drupal 8 website already in production running on a LAMP server on VPS or Dedicated with command line access via SSH.
Use the Sudo or Apt-Get tools to install Varnish Cache via the command line depending on the Linux version running on the web server.
For Ubuntu Linux web servers:
sudo apt-get install -y varnish
For Debian Linux web servers:
apt-get install varnish
Next, navigate to the Varnish configuration file located at: /etc/default/varnish
Open the file in a text editor and add the following lines to the text:
DAEMON _OPTS="-a :80 -T localhost:6082 -f /etc/varnish/default.vcl -s malloc,512m" -S /etc/varnish/secret
It is required to create a new varnish.service file at: /etc/systemd/system/varnish.service with the following text:
[Unit] Description=Varnish HTTP accelerator Documentation=https://www.varnish-cache.org/docs/4.1/ man:varnishd [Service] Type=simple LimitNOFILE=131072 LimitMEMLOCK=82000 ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,512m ProtectSystem=full ProtectHome=true PrivateTmp=true PrivateDevices=true [Install] WantedBy=multi-user.target
Create the file & upload/save on the web server. Then reload the Varnish configuration:
systemctl daemon-reload
Next, navigate to: /etc/varnish/default.vcl and make a backup copy of the default.vcl file. Save the backup and create a new file with the following code:
backend default {
.host = "example.com";
.port = "8080";
}
# Access control list for PURGE requests.
# Here you need to put the IP address of your web server
acl purge {
"127.0.0.1";
}
# Respond to incoming requests.
sub vcl_recv {
# Add an X-Forwarded-For header with the client IP address.
if (req.restarts == 0) {
if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
}
else {
set req.http.X-Forwarded-For = client.ip;
}
}
# Only allow PURGE requests from IP addresses in the 'purge' ACL.
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "Not allowed."));
}
return (hash);
}
# Only allow BAN requests from IP addresses in the 'purge' ACL.
if (req.method == "BAN") {
# Same ACL check as above:
if (!client.ip ~ purge) {
return (synth(403, "Not allowed."));
}
# Logic for the ban, using the Cache-Tags header. For more info
# see https://github.com/geerlingguy/drupal-vm/issues/397.
if (req.http.Cache-Tags) {
ban("obj.http.Cache-Tags ~ " + req.http.Cache-Tags);
}
else {
return (synth(403, "Cache-Tags header missing."));
}
# Throw a synthetic page so the request won't go to the backend.
return (synth(200, "Ban added."));
}
# Only cache GET and HEAD requests (pass through POST requests).
if (req.method != "GET" && req.method != "HEAD") {
return (pass);
}
# Pass through any administrative or AJAX-related paths.
if (req.url ~ "^/status.php$" ||
req.url ~ "^/update.php$" ||
req.url ~ "^/admin$" ||
req.url ~ "^/admin/.*$" ||
req.url ~ "^/flag/.*$" ||
req.url ~ "^.*/ajax/.*$" ||
req.url ~ "^.*/ahah/.*$") {
return (pass);
}
# Removing cookies for static content so Varnish caches these files.
if (req.url ~ "(?i).(pdf|asc|dat|txt|doc|xls|ppt|tgz|csv|png|gif|jpeg|jpg|ico|swf|css|js)(?.*)?$") {
unset req.http.Cookie;
}
# Remove all cookies that Drupal doesn't need to know about. We explicitly
# list the ones that Drupal does need, the SESS and NO_CACHE. If, after
# running this code we find that either of these two cookies remains, we
# will pass as the page cannot be cached.
if (req.http.Cookie) {
# 1. Append a semi-colon to the front of the cookie string.
# 2. Remove all spaces that appear after semi-colons.
# 3. Match the cookies we want to keep, adding the space we removed
# previously back. (1) is first matching group in the regsuball.
# 4. Remove all other cookies, identifying them by the fact that they have
# no space after the preceding semi-colon.
# 5. Remove all spaces and semi-colons from the beginning and end of the
# cookie string.
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie, ";(SESS[a-z0-9]+|SSESS[a-z0-9]+|NO_CACHE)=", "; 1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
if (req.http.Cookie == "") {
# If there are no remaining cookies, remove the cookie header. If there
# aren't any cookie headers, Varnish's default behavior will be to cache
# the page.
unset req.http.Cookie;
}
else {
# If there is any cookies left (a session or NO_CACHE cookie), do not
# cache the page. Pass it on to Apache directly.
return (pass);
}
}
}
# Set a header to track a cache HITs and MISSes.
sub vcl_deliver {
# Remove ban-lurker friendly custom headers when delivering to client.
unset resp.http.X-Url;
unset resp.http.X-Host;
# Comment these for easier Drupal cache tag debugging in development.
unset resp.http.Cache-Tags;
unset resp.http.X-Drupal-Cache-Contexts;
if (obj.hits > 0) {
set resp.http.Cache-Tags = "HIT";
}
else {
set resp.http.Cache-Tags = "MISS";
}
}
# Instruct Varnish what to do in the case of certain backend responses (beresp).
sub vcl_backend_response {
# Set ban-lurker friendly custom headers.
set beresp.http.X-Url = bereq.url;
set beresp.http.X-Host = bereq.http.host;
# Cache 404s, 301s, at 500s with a short lifetime to protect the backend.
if (beresp.status == 404 || beresp.status == 301 || beresp.status == 500) {
set beresp.ttl = 10m;
}
# Don't allow static files to set cookies.
# (?i) denotes case insensitive in PCRE (perl compatible regular expressions).
# This list of extensions appears twice, once here and again in vcl_recv so
# make sure you edit both and keep them equal.
if (bereq.url ~ "(?i).(pdf|asc|dat|txt|doc|xls|ppt|tgz|csv|png|gif|jpeg|jpg|ico|swf|css|js)(?.*)?$") {
unset beresp.http.set-cookie;
}
# Allow items to remain in cache up to 6 hours past their cache expiration.
set beresp.grace = 6h;
}Make sure to edit the line to add the correct domain name address. Then navigate to: /etc/apache2/ports.conf and open the ports.conf file. Change the text:
Listen 80
To:
Listen 8080
Then navigate to: /etc/apache2/apache2.conf and open the apache2.conf file. Change the setting ‘AllowOverride None’ to ‘AllowOverride All’ and save the settings.
<Directory /var/www/> Options Indexes FollowSymLinks AllowOverrideAll Requireall granted </Directory>
To finish, navigate to /etc/apache2/sites-available & open all the available files, changing the port from 80 to 8080 on the virtual host settings.
In the last step, create a new virtual host file for the domain name at: /etc/apache2/sites-available/yourdomain.conf and add the following code:
<VirtualHost *:8080> ServerAdmin admin@example.com DocumentRoot /var/www/html/example.com ServerName example.com ServerAlias www.example.com <FilesMatch .php$> SetHandler application/x-httpd-php </FilesMatch> <Directory /> Options FollowSymLinks AllowOverrideAll </Directory> <Directory /var/www/html/example.com> DirectoryIndex index.php Options Indexes FollowSymLinks MultiViews AllowOverrideAll Requireall granted </Directory> ErrorLog /var/log/apache2/example.com-error_log CustomLog /var/log/apache2/example.com-access_log common </VirtualHost>
Restart Apache & Varnish Cache to completes the installation on LAMP. The Drupal 8 configuration settings can be managed with integration modules in the next step.
Step Two: Install Modules for Cache Management in Drupal 8
There is a lot of competition among Drupal 8 modules for Varnish Cache management with the most popular module for Drupal 7 still in development & not ready for production use. Older modules may not support Varnish Cache 4.x standards.
REQUIRED FILES – DOWNLOADS:
- Purge: (Download Files)
- Varnish Purger: (Download Files)
ALTERNATIVE CACHE MANAGEMENT MODULES:
- Generic HTTP Purger: (Download Files)
- Acquia Purge: (Download Files)
- Dropsolid Purge: (Download Files)
DRUPAL 8 MODULES IN DEVELOPMENT:
- Varnish: (Download Files)
For the purposes of this Tutorial, it is only required to install the Purge & Varnish Purger modules. The alternative cache management modules are for optional use only.
Monitor the status of the Varnish module for a Drupal 8 version or install & test the development version to provide feedback on bugs, issues, or required changes.
Drupal 8 publishers are recommended to have Drush installed for installing modules or clearing the Varnish Cache. For advanced use, build a site-wide Cache Tag system.
The Generic HTTP Purger module provides support for CDN integration with Varnish Cache & Drupal 8, which the other modules generally lack.
The Acquia Purge module is intended specifically for Acquia Cloud integration. The Dropsolid Purge module re-writes the Acquia version code to support all websites.
Recommended: Navigate to: /admin/modules/install and install the Purge & Varnish Purger modules. Enable the modules & review the global website cache settings.
Step Three: Configure the Global Varnish Cache Settings
To optimize the Drupal 8 CMS configuration for Varnish Cache including reverse-proxy caching support, open the settings.php file found in /sites/default folder and add:
$settings['reverse_proxy'] = TRUE;
$settings['reverse_proxy_addresses'] = array('127.0.0.1');In Drupal 8 administration, navigate to: /admin/config/development/performance and set the cache settings as required, (i.e. 15 minutes, 1 hour, or 24 hours depending on user website traffic and demand for system resources). Set 1 hour or 1 day as default.
Save the settings to “Aggregate CSS files†and “Aggregate JavaScript files†globally:
To finish, proceed to the next step to configure the Purge & Varnish Purger modules including any Cache Tags or BAN values required by section on high traffic websites.
Step Four: Configure the Drupal Cache Management Modules
Drupal 8 web publishers require the ability to clear the Varnish Cache, warm the cache with content, & set up Cache Tags for different content sections or content types to clear on command. Use the Purge & Varnish Purger modules for these capabilities.
Navigate to: /admin/config/development/performance/purge and select “Varnish Purger†as the default purger for the cache:
Save the settings and select a new label for the configuration. Choose “Tag†for the Type and enter 127.0.0.1 for the Hostname with Port 80 (Ubuntu) or 8080 (Debian) as the default values.
Make sure to set “BAN†as the request method with HTTP settings on the scheme:
Save the settings and click on the “Headers†tab to set “Purge-Cache-Tags†as the Header and “[invalidation:expression]†as Value:
Click on the “Configure logging†link and add checkmarks to all of the boxes for the purger_varnish_xxx value:
Drupal 8 developers can use Drush to clear the Varnish cache:
drush varnish-purge-all
Navigate to: /admin/config/development/performance/purge to clear the Varnish Cache or use the Drush command with CLI tools. Some D8 users prefer the Devel module.
Drupal 8 developers can use tools like isvarnishworking.uk or Pantheon’s Varnish Check to test the status of the installation on their web server & CMS pages.
Conclusion: Varnish Cache for High Traffic Drupal 8 Sites
Varnish Cache is an absolutely essential part of a Drupal 8 web server stack for high traffic websites which frequently have over 100 users at a time or millions of hits.
Varnish Cache works best with NGINX, the Adv Agg module, Redis, Memcached, Zend Opcache, APC, & a CDN service for optimizing a Drupal 8 web server for speed.
Drupal 8 web publishers will require a minimum of a VPS or Dedicated Server plan to run Varnish Cache or choose a Cloud Hosting plan with Varnish Cache pre-installed.
Check out these top 3 Drupal hosting services:
- Want to get top recommendations about best hosting? Just click this link!
