Building your `programming helper` pet server with docker, on ubuntu

Nowadays there are quite many tools out there to help the process of developing apps. To easily manage them, I decided to go with a docker-compose approach on a VPS running ubuntu.

Since more or less last year, I have been trying to get “acquainted” with Jenkins (automating stuff, I mean) and his friends, but in the process stumbled upon other useful tools – Sonarqube, Sentry, Nextcloud, OnlyOffice.

The base

I have used and Ubuntu 18.04.1 lts with the latest docker version at this time (18.09.1), even if the .yml file is with v2. Also, for volumes, I have used the most basic implementation possible. Obviously, we also make use of git for this project.

Suggestion: you might want to setup your server to initialise your project at startup.

I initially wanted an nginx approach for the setup, but in the end found it rather complicated – or just more complicated that I wanted from something designed to help me do developer stuff. Therefore, I decided nginx and its performance will be the solution for projects, while this will be powered by traefik , as a reverse proxy solution/helper.

docker-compose.yml

This is the pretty straightforward common code for docker-compose.yml.

version: '2'
services:
  traefik:
  # bla, bla :)Code language: PHP (php)

Your service definitions could contain naming as well: container_name: 'project-traefik'. Otherwise, by the docker-compose rules, your containers will be named based on the folder that contains the .yml file.

Environments

The intention was to create two different environments: one for the server where the tooling will actually run and another for testing/managing the tool list. A few reasons for this:

  • the obvious one: editing on the server and testing there is more complicated than doing it locally;
  • https: traefik has nice support for automating the certificate retrieval from letsencrypt, therefore it would be a pity not to take advantage of it.

.env files

The solution for configuration was a .env files approach. The file stored in the git repository will be .env.example and will contain the configuration for the local environment. Also, we use the same file for all the services.

.env.example

This is example code – as the subtitle states – and also includes the assumptions for this article.

BASE_URL="my.host.local"
BASE_EMAIL="no-email@my.host.local"Code language: JavaScript (javascript)

A file structure

  • ./.data/
    This folder will contain the volumes for each of the containers. I like to start by creating subfolders for each of the services
    • .gitignore to ignore the contents of this folder
  • ./services/
    This contains folders for each of the services
  • .env.example
  • docker-compose.yml
  • .gitignore
    Has usual and basic rules

The proxy

As I said above, the proxy is provided by traefik. The content of the docker-compose.yml file is as follows:

traefik:
  build: ./services/traefik
  env_file: .env
  restart: always
  command:
    - "--configFile=/traefik.toml"
  ports:
    - "80:80"
    - "443:443"
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
    - ./.data/traefik/acme:/etc/traefik/acmeCode language: JavaScript (javascript)

Service files

As per our conventions above, the service files are stored in ./services/traefik with the following structure:

  • ./conf/
    • traefik.dev.toml.tmpl
    • traefik.toml.tmpl
  • Dockerfile
  • entrypoint.sh

Inside the .toml.tmpl files we are able to specify variables from the .env file, due to the fact that entrypoint.sh actually puts those values inside and renames the files to simply .toml .

traefik.dev.toml.tmpl

debug = true

logLevel = "ERROR"
defaultEntryPoints = ["http"]

[entryPoints]
  [entryPoints.http]
  address = ":80"

[retry]

[docker]
domain = "${BASE_URL}"
endpoint = "unix:///var/run/docker.sock"
watch = true
exposedByDefault = falseCode language: JavaScript (javascript)

traefik.toml.tmpl

debug = false

logLevel = "ERROR"
defaultEntryPoints = ["https","http"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]

[retry]

[docker]
domain = "${BASE_URL}"
endpoint = "unix:///var/run/docker.sock"
watch = true
exposedByDefault = false

[acme]
email = "${BASE_EMAIL}"
storage = "/etc/traefik/acme/acme.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"Code language: JavaScript (javascript)

Dockerfile

FROM traefik:v1.7-alpine

ENV BUILD_DEPS="gettext"  \
  RUNTIME_DEPS="libintl"

RUN set -x && \
  apk add --update $RUNTIME_DEPS && \
  apk add --virtual build_deps $BUILD_DEPS &&  \
  cp /usr/bin/envsubst /usr/local/bin/envsubst && \
  apk del build_deps

RUN mkdir -p /conf

COPY ./conf/traefik.dev.toml.tmpl /conf/traefik.dev.toml.tmpl
COPY ./conf/traefik.toml.tmpl /conf/traefik.toml.tmpl

COPY ./entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT [ "/bin/sh", "/entrypoint.sh" ]Code language: PHP (php)

entrypoint.sh

#!/bin/sh
set -e

# first arg is `-f` or `--some-option`
if [ "${1#-}" != "$1" ]; then
    set -- traefik "$@"
fi

# if our command is a valid Traefik subcommand, let's invoke it through Traefik instead
# (this allows for "docker run traefik version", etc)
if traefik "$1" --help | grep -s -q "help"; then
    set -- traefik "$@"
fi

# Define a variable to contain the dollar sign which needs to be escaped in the nginx config files
ESC=$

# enable default nginx configuration
if [ $TRAEFIK_MODE == 'prod' ]; then
  envsubst < /conf/traefik.toml.tmpl > /traefik.toml
else
  envsubst < /conf/traefik.dev.toml.tmpl > /traefik.toml
fi

# Run CMD in Dockerfile
exec $@Code language: PHP (php)

The certificates

For local development, I just went for http while for the public environment made use of traefik‘s letsencrypt support. In order to save the certificates, we have ./.data/traefik/acme:/etc/traefik/acme inside the service definition.

If having trouble with https, start by checking the permissions on the acme.jsonfile. Quick command from your project’s root: chmod 600 ./.data/traefik/acme/acme.json

Web

Traefik has a web interface. You might not want to use it, but in case you do, having some minimal protection is normally useful. Enabling it can be done by:

1) Changing the command to:

command:
  - "--web"
  - "--configFile=/traefik.toml"Code language: JavaScript (javascript)

2) The following labels put on the service definition:

labels:
  - "traefik.enable=true"
  - "traefik.frontend.rule=Host:traefik.${BASE_URL}"
  - "traefik.frontend.auth.basic=user:pass-already-encrypted"
  - "traefik.port=8080"Code language: JavaScript (javascript)

Tip: remember that your password should be encrypted as for a .htpasswd file. In your local dev, this interface will be accesible at http://traefik.my.host.local.

An example service

Portainer is a web interface designed to manage docker. The definition for our docker-compose.yml:

portainer:
  build: ./services/portainer
  env_file: .env
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - ./.data/portainer:/data
  restart: unless-stopped
  labels:
    - "traefik.enable=true"
    - "traefik.frontend.rule=Host:portainer.${BASE_URL}"
    - "traefik.port=9000"Code language: PHP (php)

./services/portainer/Dockerfile only contains FROM portainer/portainer:1.20.1.

The labels

Traefik manages the interaction with other containers using labels. You can read more about how to use them their website. However, for basic usage, the following three are important:

  • traefik.enable=true
  • traefik.frontend.rule=Host:portainer.${BASE_URL}
    This tells traefik what should the address of your service be.
  • traefik.port=9000
    This tells traefik which is the port where your service can be accessed. To the outside world, it will be the port of traefik.

Running it

Nothing special here, just common commands for docker-compose in your project folder:

  • docker-compose -f ./docker-compose.yml up -d --build
  • docker-compose -f ./docker-compose.yml up -d
  • docker-compose down

At a larger scale

When more services need to run, together with more complex configurations or components which have more frequent updates, I would recommend using separate repositories for each of the services.

Sources

Tags