• Drupal CMS Developer
  • Website speed optimisation
  • Drupal 6 to Drupal 8 migrations


Welcome, I am a Web Developer based in Madrid, Spain originally from the UK. I studied Computer Science & eBusiness at Loughborough University. I specialise in Content Management System websites

Can't set custom log format for Varnishncsa with systemd

In the old days before systemd we updated /etc/default/varnishncsa enabling it and setting a custom format should we so choose. Now its a little more complicated. As this file says:

# Note: If systemd is installed, this file is obsolete and ignored.  You will
# need to copy /lib/systemd/system/varnishncsa.service to /etc/systemd/system/
# and edit that file.

Thats fine, so you go and create your /etc/systemd/system/varnishncsa.service file in said folder, enter the old configuration that you've always used and... whats this... it gives an error now??!?!

Basically it took me about 3 months and several attempts to realise that the reason it gives an error is because the log format contains special characters that the ExecStart line doesn't like. Here is my log format:

%h %u %t %r %s %{Referer}i %{User-agent}i %{Varnish:hitmiss}x

This basically replicates the apache2 log format but also tells me at the end if it was a cache hit or miss. The cool thing about this is you can get a live view of the URLs being requested which aren't cached (tail -f /var/log/varnish/varnishncsa.log | grep -v "js/\|sites/\|newsletters/\| hit" - notice that Im excluding any ajax calls, any images, and anything with "hit" at the end which means it was served by the cache, and it gives me a live view of just the pages/services being called).

Ok, so back to the problem. Every time I tried to write in my /etc/systemd/system/varnishncsa.service file:

ExecStart=/usr/bin/varnishncsa -a -F "%h %u %t %r %s %{Referer}i %{User-agent}i %{Varnish:hitmiss}x" -w /var/log/varnish/varnishncsa.log

it would give me an error and not tell me why. As it turns out, all you need to do is escape the % symbol by prefixing it with another. Making this line look like this:

ExecStart=/usr/bin/varnishncsa -a -F "%%h %%u %%t %%r %%s %%{Referer}i %%{User-agent}i %%{Varnish:hitmiss}x" -w /var/log/varnish/varnishncsa.log

And 3 months of frustration turned into celebrations!