Problem

You want to investigate salt internals a possible even make some changes to the salt codebase. How to set up a development environment?

Solution

This article is based on following the saltstack hacking page.

As I work on a wide variety of projects, using all manner of different setups, I prefer to set each one up in as isolated an environment as practicable. My ‘go to choices’ being either a Docker container or a vargant driven virtual machine environment. For this setup I decided a VM was more appropriate.

You can download the system described in this article (then use and abuse it as you please) from my gitlab repo.

To use this repo:

1
2
3
4
git clone https://gitlab.com/sv.saltstack/salt-dev.git
cd salt-dev
vagrant up
vagrant ssh
 

Once on the development machine you need to start the virtualenv so that all salt commands work correctly using the development version of salt found under the /home/vagrant/salt directory.

1
source /home/vagrant/venv/bin/activate
 

You can now use salt as you would normally (there will be a running salt-master and salt-minion.

The Vagrantfile

So far, so simple.

To discuss the detail, let’s examine the Vagrantfile.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.

vagrantfiledir=File.expand_path(File.dirname(__FILE__))

saltrepourl=ENV['SALTREPO'] || ''

salttooling=<<-SHELL
apt-get install -y vim git tmux python-virtualenv python3-virtualenv python-dev python3-dev psmisc build-essential
SHELL

saltclone=<<-SHELL
if [ -z "$1" ]; then
  git clone https://github.com/saltstack/salt
else
  mv /tmp/secure/id_rsa /home/vagrant/.ssh/id_rsa
  cat >>/home/vagrant/.ssh/config <<-SSHCONF
Host gitlab.com
  StrictHostKeyChecking no
  UserKnownHostsFile=/dev/null
SSHCONF
  git clone $1
  cd salt
  git remote add upstream https://github.com/saltstack/salt
  git fetch --tags upstream
  cd ..
fi
SHELL

saltdev=<<-SHELL
# Setup virtualenv and devtools
virtualenv -p python3 venv
source /home/vagrant/venv/bin/activate
pip install pyzmq PyYAML pycrypto msgpack-python jinja2 psutil futures tornado pylint saltpylint

# Setup for document generation
pip install Sphinx==1.3.1

# Setup for tests
pip install -r salt/requirements/dev_python34.txt
pip install -r salt/requirements/zeromq.txt

# Setup local configuration
mkdir -p ./venv/etc/salt/pki/{master,minion}
cp ./salt/conf/master ./salt/conf/minion ./venv/etc/salt/
sed -i -e 's/^#master: salt/master: localhost/' -e 's/#id:.*/id: saltdev/' ./venv/etc/salt/minion

# Install salt development
GENERATE_SALT_SYSPATHS=1 pip install --global-option='--salt-root-dir=/home/vagrant/venv' -e ./salt

mkdir -p ./venv/srv

# Startup salt
salt-master -d
sleep 5
# Run the minion as root (otherwise it will not be able to run many states/modules)
sudo bash -c "source $(pwd)/venv/bin/activate && salt-minion -d"
sleep 15
salt-key -A -y
sleep 5

# Quick sanity check
salt '*' test.ping
SHELL

Vagrant.configure("2") do |config|
  config.vm.box = "debian/stretch64"
  config.vm.synced_folder ".", "/vagrant", disabled: true
  config.vm.provider "virtualbox" do |vm|
    vm.name = "salt.dev"
    vm.memory = 4096
    vm.cpus = 2
  end

  if File.exists?("%s/secure" % vagrantfiledir)
    config.vm.provision "file", source: "%s/secure" % vagrantfiledir, destination: "/tmp/secure"
  end

  if File.exist?("%s/files" % vagrantfiledir)
    config.vm.provision "file", source: "%s/files" % vagrantfiledir, destination: "/tmp/files"
  end
  config.vm.provision "shell", inline: salttooling
  config.vm.provision "shell", privileged: false, inline: saltclone, args: saltrepourl
  config.vm.provision "shell", privileged: false, inline: saltdev

  scripts = Dir.glob("%s/scripts/*" % vagrantfiledir).reject {|f| File.directory? f}
  for script in scripts do
    config.vm.provision "shell", privileged: false, path: script
  end

end
 

As line 11 suggested, you can specify an environment variable SALTREPO in order to replace the main saltstack repository with your own fork. So a simple command line tweak such as the following allows my to use my own fork for ‘playing’ with salt development.

1
SALTREPO=git@gitlab.com:sv.saltstack/salt.git vagrant up
 

The saltclone script (lines 17–33) ensures GitLab is exluded from SSH key checking (note to self, really ought to make this use the SALTREPO url). Once your repository is cloned the saltstack repository is added as the upstream, providing an easy way to stay up to date with the salt development team.

The saltdev script (lines 35-69) does most of the work to set up for salt development. I’ve assumed you’re using python3 since we should all be making that move now. This script also sets up for documentation generation and sets the virtualenv up so you don’t have to constantly supply -c /home/vagrant/venv/etc/salt to salt commands.

Once everything is installed the salt-master and salt-minion are started.

The VM and its provisioning

The VM is a base Debian installation, 4Gb RAM, 2 cpus.

If you supply a secure/ directory (alongside the Vagrantfile) it will be loaded into the VM as /tmp/secure. This simplifies providing things like your repository SSH key and gpg signing key. Obviously, it’s important to not include this information in your repository (so this directory is explicitly ignored in the .gitignore).

Similarly any files in files/ directory are uploaded to /tmp/files. These are ‘insecure’ files and should be safe to put into your repository.

After the internal scripts (see above) have run, the Vagrantfile will run any scripts it finds in the scripts/ directory. These are run in directory listing order, so it’s best to precede names with 01, 02, and so on if you need to a clear order of provisioning.

Discussion

This is a very simple setup. It provides a quick method to set up a saltstack development environment (and yes, although somewhat general, it is set up to my taste—installing vim for example).

The user scripts/ provide maximum flexibility though, so I don’t feel my choices are particularly problematic. One day I’ll clean up a bit and make it even more general.

I put into my scripts/ two key scripts that you may also find useful.

Load test salt files

scripts/01install_salt_files

1
2
3
4
5
6
7
8
9
10
#!/usr/bin/env bash

mkdir -p /srv
sudo chown vagrant:vagrant /srv
mv /tmp/files/salt /srv/

[ -d /home/vagrant/venv/srv ] && [ ! -e /home/vagrant/venv/srv/salt ] && [ -d /srv/salt ] && ln -s /srv/salt /home/vagrant/venv/srv/salt
[ -d /home/vagrant/venv/srv ] && [ ! -e /home/vagrant/venv/srv/pillar ] && [ -d /srv/pillar ] && ln -s /srv/pillar /home/vagrant/venv/srv/pillar

exit 0
 

This just uploads any salt state/pillar data I need for whatever part of salt I’m testing/developing. Making them owned by the vagrant user is laziness on my part; it makes editing them easy from the default vagrant user context and the security implications don’t matter as this is a scratch (and isolated) environment).

scripts/02gitconf

1
2
3
4
5
6
7
 #!/usr/bin/env bash

git config --global user.name "Fred Flintstone"
git config --global user.email "fred@example.com"
git config --global user.signingkey 05047339A0FC99BEBFBEA2B56B7AF8F8A77935B3
git config --global commit.gpgsign true
git config --global gpg.program gpg
 

This finalises configuring git ready for signing all commits. Note however that you will still need to install your gpg signing key! I do this using the secure/ directory (see above) to upload my signingkey.asc file and then simply install it using gpg --import /tmp/secure/signingkey.asc the first time I log in to the VM. This step can’t be conveniently automated as you signing key (should) be passphrase protected.