varnish-formula
A simple SaltStack formula to install and configure Varnish.
This formula has been developed distributing id and state declarations in different files to make it usable in most situations. It should be useful in scenarios ranging from a simple install of the packages (without any special configuration) to a more complex set-up.
1. General notes
See the full SaltStack Formulas installation and usage instructions.
If you are interested in writing or contributing to formulas, please pay attention to the Writing Formula Section.
If you want to use this formula, please pay attention to the FORMULA
file and/or git tag, which contains the currently released version.
This formula is versioned according to Semantic
Versioning.
See Formula Versioning Section for more details.
If you need (non-default) configuration, please pay attention to the
pillar.example file and/or Special notes section.
2. Contributing to this repo
Commit message formatting is significant!!
Please see How to contribute for more details.
4. General customization strategies
First, see if providing pillar data is enough for your customization
needs That’s the recommended way and should be enough for most cases.
See that sometimes there’s a key named extra_conf that’s used to add
arbitrary configuration lines in the templates provided.
When providing pillar data is not enough for your needs, you can apply
the Template Override and Files Switch (TOFS) pattern as described in
the documentation file TOFS_pattern.md.
6. Next-generation, alternate approach
The following states provide an alternate approach to managing Varnish. Tested in Ubuntu 14/16 and CentOS 6/7.
6.3. varnish.ng.repo
Install the packagecloud.io varnish repo. Repos available can be found in packagecloud.io
7. Varnish NG considerations
Because of the different OSes with different init programs (ubuntu14 - init, ubuntu 16 - systemd, centos 6 - init, centos 7 - systemd), the following considerations must be taken into account:
-
VCL files must be copied to minions using
salt://(see pillar.example vcl section):
varnish:
ng:
vcl:
files:
default:
path: /etc/varnish/default.vcl
source_path: salt://hostname/default.vcl.jinja
mobile_detect:
enabled: False
path: /etc/varnish/mobile_detect.vcl
source_path: salt://hostname/mobile_detect.vcl-
Config files (
/etc/default/varnish,/etc/sysconfig/varnishor/etc/varnish/varnish.params) can be copied withsalt://or just creating the variables in theconfig_fileoption:
varnish:
ng:
config_source_path: salt://hostname/varnish.jinjaor
varnish:
ng:
config_file:
NFILES: "131072"
MEMLOCK: "82000"
NPROCS: "unlimited"
RELOAD_VCL: "1"
VARNISH_VCL_CONF: "/etc/varnish/default.vcl"
VARNISH_LISTEN_PORT: "6081"
VARNISH_ADMIN_LISTEN_ADDRESS: "127.0.0.1"
VARNISH_ADMIN_LISTEN_PORT: "6082"
VARNISH_SECRET_FILE: "/etc/varnish/secret"
VARNISH_MIN_THREADS: "50"
VARNISH_MAX_THREADS: "1000"
VARNISH_STORAGE_SIZE: "100M"
VARNISH_STORAGE: "malloc,${VARNISH_STORAGE_SIZE}"
VARNISH_TTL: "120"
DAEMON_OPTS: "-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} -f ${VARNISH_VCL_CONF} -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} -p thread_pool_min=${VARNISH_MIN_THREADS} -p thread_pool_max=${VARNISH_MAX_THREADS} -S ${VARNISH_SECRET_FILE} -s ${VARNISH_STORAGE}"-
In CentOS 6, it must be explicitely defined the
install_from_repooption because in the CentOS 6 repos it is the 2 version:
varnish:
ng:
lookup:
repo: 'varnish41'
install_from_repo: True-
For the Ubuntu 16 there is a problem regarding the usage of the systemd; there is no other way than editing the systemd varnish unit file to modify the init options (CentOS 7 has also systemd but there is the
varnish.paramsto bypass the edition of the unit file directly). A little trick to automate this, is using the following pillar:
varnish:
ng:
lookup:
repo: 'varnish41'
config: /etc/systemd/system/varnish.service
config_source_path: salt://hostname/varnish-systemdBeing varnish-systemd the varnish systemd unit file:
[Unit]
Description=Varnish Cache, a high-performance HTTP accelerator
[Service]
Type=forking
# Maximum number of open files (for ulimit -n)
LimitNOFILE=131072
# Locked shared memory - should suffice to lock the shared memory log
# (varnishd -l argument)
# Default log size is 80MB vsl + 1M vsm + header -> 82MB
# unit is bytes
LimitMEMLOCK=85983232
# On systemd >= 228 enable this to avoid "fork failed" on reload.
#TasksMax=infinity
# Maximum size of the corefile.
LimitCORE=infinity
# Set WARMUP_TIME to force a delay in reload-vcl between vcl.load and vcl.use
# This is useful when backend probe definitions need some time before declaring
# configured backends healthy, to avoid routing traffic to a non-healthy backend.
#WARMUP_TIME=0
ExecStart=/usr/sbin/varnishd -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,120m
ExecReload=/usr/share/varnish/reload-vcl
[Install]
WantedBy=multi-user.targetSo you can edit and have the file managed by salt.
-
Varnish versions from 6 (including it) have change the systemd unit service; are OS agnostic (see packaging changes).
For Debian Family OSes can use the previous trick (changing the unit service).
For RedHat Family OSes there is a new variable version that setup an
override.conf for the default varnish systemd service that allow to
use the varnish.params file. For example, to install varnish 6 LTS:
varnish:
ng:
lookup:
repo: 'varnish60lts'
version: '6'
install_from_repo: True
enabled: True
config_source_path: salt://hostname/varnish.params.jinja
vcl:
files:
default:
path: /etc/varnish/default.vcl
source_path: salt://hostname/default.vcl.jinjaThis change is compatible with other varnish versions (there is no error if not set).
8. Testing
Linux testing is done with kitchen-salt.
8.1. Requirements
-
Ruby
-
Docker
$ gem install bundler
$ bundle install
$ bin/kitchen test [platform]Where [platform] is the platform name defined in kitchen.yml, e.g.
debian-9-2019-2-py3.
8.2. bin/kitchen converge
Creates the docker instance and runs the template main state, ready
for testing.