Decentralized team passwords with Keepass and Syncthing

At synyx we constantly try to improve the quality of the work of the Operations team.

We found we were missing certain security requirements. Although we were already using ssh-key-authentication for all our linux-servers, we still had some devices and assets lying around, that were (only) accessible with the vendor-assigned default credentials.

Sadly it’s impossible to use key-authentication for all devices and some appliances don’t allow for creation of multiple personalized accounts with admin rights. Sometimes it can also come in handy to have a user/password fallback in case you can’t access the machine with your ssh keys. Therefore we decided we could improve our security by replacing the default credentials with strong passwords that won’t be reused. Since those passwords would have to be known to 7 people, we had to find a way to share them without the need to manually update everytime someone provided or changed any data.

We evaluated multiple tools under certain criteria:

• encrypted password container
• automatic sync on updates from others
• simple configuration
• decentralized sharing (no single point of failure)
• Open Source software preferred
• run on Linux, MacOS and probably Windows

We came up with a solution involving a keepass2 password container and Syncthing, a decentralized file sync framework.

Keepass

Keepass is an encrypted database for storing sensitive information like user/password-credentials and notes.

We quickly figured that we’d have to use a keepass2-compatible container to allow usage on all required OSs. Debian and Ubuntu offer official packages for keepass2, MacOS users can use keepassX, a port also available for Debian/Ubuntu. Aside of the client choice we run a pretty “standard” container with a single master password known to all admins. As of today, keepass containers can be encrypted with a (single) key. Admins can be given this key instead or in addition to the master password. Make sure you never share the key with the container!

Syncthing

Syncthing is a tool that allows you to share and synchronize files with multiple devices that form a cluster. All traffic between devices is encrypted with TLS1.2. It is typically installed as a service on every device. You can add other devices by specific ID. Once a device comes online it broadcasts a discovery announcement package on the local network (multicast on ipv6). Devices that have been “introduced by Device-ID” authenticate to each other to allow index comparison and possible updates. To allow for synchronization outside of the local network, its is possible to set up global device search through a discovery server. To relay actual traffic (syncing files) you can setup a syncthing relay-server or use existing servers. It is also possible to run and connect to other syncthing devices through SOCKS5 proxies.

Syncthing also allows you to keep a history of changes, so you have a fallback if someone accidently breaks the container.

Devices can be configured to be an “introducer”. New devices added to an introducer get shared to the whole group of linked devices. This prevents you from adding all new Device-IDs manually.

Our setup

We manage syncthing and keepass installations through puppet, and also provision the admin cluster config with all the Device-IDs automatically. There are two utility servers in the cluster that are always on to provide the share in case no admin laptop is reachable and someone misses updates. We also run an instance in the Datacenter in case an admin forgets his laptop

I tried to visualize the setup here, with every device being able to talk to every other device that has been added to the cluster.