Between Tuesday August 20th, 5:00pm and Thursday August 22nd, 8:00am git.uwaterloo.ca will be down for an upgrade to version 10.8.7.

W

wcms-docker

WCMS Docker setup

CAUTION! This Docker box is meant for local development only. This set up is not hardened and is not secure, so only us it as a local development environment.

Before You Start

You will need to have Administrator rights on your machine to be able to install this software.

You will need to install Docker Desktop CE.

You will need to create a Docker ID (if you don't already have one) to download the latest Docker Desktop.

On a Windows machine, Docker Desktop CE requires Hyper-V to be enabled which isn't compatible with VirtualBox, we recommend running Docker Desktop CE. If you need to run VirtualBox and Docker you will need to install the Docker Toolbox instead, Docker Toolbox for Windows or Docker Toolbox for Mac. Just note that we have not tested this install against Docker Toolbox.

You will also need to install Docker-sync (instructions are below).

Setting Up Docker

During setup you will be asked if you want to use Linux or Windows containers, for consistency (between Mac and Windows users) we will choose linux containers.

Note On a Windows machine during install, if you have VirtualBox installed you will get a warning: "Hyper-V and Containers are not enabled. Do you want to enable them for Docker to be able to work properly? Your computer will restart automatically. Note: VirtualBox will no longer work."

We want to enable Hyper-V so you can accept this (but as it said Virtualbox will no longer work).

When your computer restarts, Docker should also start and it will be asking you to log in with your Docker ID. If you don't want to log in, you can choose Privacy settings and the uncheck "send usage statistics..." (You can also optionally uncheck "start Docker Desktop when you log in"

Note If you had Docker Toolbox installed previously you should uninstall it and you might need to remove User Environmental Variables. If this is the case, you need to go to System > System Info > Advanced Settings > Environmental Variables and remove and Docker settings from the User Variables. Once you've done this you should reboot your machine.

You should update your local machines host file to include:

       127.0.0.1 wcms7 wcms7alt wcms8 wcms8alt

Install docker-sync

On a Windows machine to install Docker-sync you will need to enable the Windows Subsystem for Linux (WSL) and download and install a Linux distribution from the MS Store (We recommend Ubuntu).

To get setup (for both Windows and Mac), follow the instructions here.

On a Windows machine, two changes to their instructions, this is the link to install Docker on Ubuntu and you can install the OCAML 4.07.0 instead of 4.06.0. More information here.

On a Mac you may need to upgrade your copy of XCode (if you already have it installed) to the latest version, currently 10.2.1. You will need to remove the old version first.

WSL (for Windows machines only)

Following the specific instructions for docker-sync on WSL here, there is some additional information to help with the setup:

On a fresh install of Ubuntu WSL, a quick check using ls /mnt/c should show the contents of the C: drive. If not, then there is an issue with WSL.

To have the C: drive accessible to docker, it needs to be in /c, therefore as of 1803 the following entry in /etc/wsl.conf is needed:

[automount]
enabled = true
root = /
options = "metadata,umask=22,fmask=11"
mountFsTab = false

Following the instructions, there is no need to reboot/logout of the machine for the changes to take effect. Instead the WSL service can be restarted (from admin CMD):

sc query LxssManager        //check status
sc stop LxssManager         //stop the service
sc start LxssManager        //start the service

A simple stop then start should be enough to have the settings take effect.

Launching wsl is as simple as calling the wsl command in CMD or Powershell, where it will launch the default linux distro. For git-bash, use winpty wsl.

Check if C: drive is mounted as /c.

Linking directly to the windows binaries for docker is hit or miss, the safer option is to follow the install guide and installing the Ubuntu docker and docker-compose.

Navigating to a user folder requires cd /c/Users/'!user' if the user has a ! prefix.

Cloning the repo from within WSL or Windows does not seem to matter, but it is safer to do it from within WSL, else there might be issues mounting volumes.

The unison and caml parts can be automated a little by simply adding && \ to every line except the last to chain the commands.

To avoid permission errors when launching docker, ensure that the user used when sharing the C: drive with docker also has access to the folder created by the git clone.

Clone wcms-docker (for all OS'):

You should now be ready to clone the repo:

       git clone https://git.uwaterloo.ca/wcms/wcms-docker.git wcms-docker

Go into the directory you cloned Docker into:

       cd wcms-docker

First we need to build the images, this takes at least 10 minutes or so to build:

       docker-compose build --no-cache

Note The first time you run this on a Windows machine, you may be asked to share C: drive and you may need to add a Windows Firewall permission. Click yes/okay for both.

Also Note that docker caches images, if you update wcms-docker from Git and there are changes to the Dockerfiles (or anything that gets built by them) for any container you will need to rebuild. To do this you should use "docker-compose build --no-cache" which builds the image from whatever Dockerfile is referenced in the docker-compose.yml. You will then need to do a "docker-compose up -d" for it to use the new builds.

On a Windows machine, before you start the containers you will need to start your Ubuntu Shell. Once it's started, and you've logged in you will need to go to the directory you cloned wcms-docker into. An example might be "cd /mnt/c/Users//docker/wcms-docker".

For both Windows and Mac's, once you are in the folder you need to change one of the OS specific docker-sync files to be docker-sync.yml:

       mv docker-sync_<my OS>.yml docker-sync.yml

Then you can run:

       docker-sync start

Which will create the sync folders that are required when you run docker-compose up -d

Note On a Mac, if you get an error mkmf.rb can't find header files for ruby at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/include/ruby.h It means you need to update your copy of XCode to the latest version (see Install docker-sync above). Once you've updated XCode, continue below.

Once that's done you can create the containers:

       docker-compose up -d

** Note** on a Windows machine if you get an error running docker-compose related to wincred go to ~/.docker/config.json and remove the "credstore":"wincred" but leave the curly braces {} and save the file.

To check that all the boxes are running:

       docker-compose ps

If all containers say up, you are good to proceed to the next step.

Note If you are on a Windows machine and after running docker-compose ps all the drupal machines have an Exit status of 127, you will need to run git config --global core.autocrlf false. You will then need to delete the repo, re-clone it and run docker-compose build again.

Running "docker-compose up -d" will have created some folders within "wcms-docker/build-scripts". To populate these folders you need to log in to the container you want to use.

To "ssh" into the docker container e.g. docker-compose exec --user vagrant drupal7 bash:

       docker-compose exec --user vagrant <service name> bash

The current list of containers are:

  • drupal7 - drupal7 container running PHP 5.6
  • wcms7.2 - drupal7 container running PHP 7.2
  • drupal7os - Open Scholar container
  • drupal8 - drupal8 container running PHP 7.2

How to Install Your Site

Once you have logged in to the container (if you are not already logged in as the vagrant user, enter su vagrant). As the vagrant user you will need to run the "wcms-settings-d.sh" file which is located at /var/www/wcms-settings:

       cd /var/www/wcms-settings
       sudo ./wcms-settings-d7.sh

You will may be asked for the vagrant password, which is vagrant.

This file will then install drupal, clone our profile (for Drupal7), and setup the sites in the correct locations.

Once this is done you can access all the files in the local folder for the profile you are using e.g."\wcms-docker\build-scripts\drupal7\drupal7_local\"

You will need to do this for each container before you can install any sites.

Now, in a web browser you will need to go to https://wcms7:4443/fdsu1/install.php to build your drupal site.

Each container is assigned a specfic port:

Troubleshooting

If you are running Portainer and you get this error:

        un\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'

ERROR: for portainer  Cannot create container for service portainer: b'Mount denied:\nThe source path "\\\\var\\\\run\\\\docker.sock:/var/run/docker.sock"\nis not a valid Windows path'
Encountered errors while bringing up the project.

You will need to use Windows Powershell (as Admin) and enter:

        $Env:COMPOSE_CONVERT_WINDOWS_PATHS=1

Potential issue with Firefox and being able to resolve localhost: https://www.reddit.com/r/sysadmin/comments/94r885/firefox_will_soon_be_sending_all_dns_requests_to/

Fix (go to about:config in Firefox URL bar): change network.trr.mode=0 to network.trr.mode=5

Things of note

Running docker-compose down or docker-compose down --rmi all will destroy the contents of the container, and in the case of --rmi all all the images associated with the docker-compose.yml file (which means you will need to run a docker-compose build to rebuild the images the next time you want to use it.

If you have your containers running, but are done your project you can shut them down and delete the volumes and images used to create the containers (to start fresh) with one command:

        docker-compose down --rmi all --volumes

You will also need to manually remove the contents of the _local folders as well if you are starting fresh.

If you just want to remove just the images so you can rebuild them (you will have to have stopped the containers using docker-compose down or the images won't delete), you can run:

       docker image prune -a

Since we are using local volumes with our containers the contents of the /build-scripts/drupal7/drupal7_local/,/build-scripts/drupal7/drupal7alt_local/, /build-scripts/drupal8/drupal8_local/ and /build-scripts/drupal8/drupal8alt_local/ will remain after you have shut down Docker because they are being stored locally. The databases associated with these containers are also being saved so unless you remove the volumes your data should remain. This means that you can shutdown docker without losing things you are working on in the drupal folders or the database. But, anything that is stored exclusively in container will disappear. So if you add a ssh key or a github key those things only exist in the container and will disappear if you do a docker-compose down or restart your computer.

Having the folders exist means that you also need to be aware that if you create a new set of containers to start a new project you will still have the existing content in the volumes. This means that you will need to clean out local site module folders and make sure you run a ./rebuild.sh on the profile. Alternatively, you can just delete the content of the _local folders and start fresh.

For Patternlab in Drupal 8, you can access it (once you have Patternlab built) at https://localhost:4446/uw_wcms_pattern_lab/pattern-lab/public/

Most of the commands within the containers should be run as the vagrant user, as long as you log in using docker-compose exec --user vagrant drupal7 bash things should run okay.

Xdebug and PHPStorm

Xdebug is installed as part of the containers, the only setup that should need to be done is in PHPStorm.

** Note ** these instructions are still a work in progress, some additional steps may be required to get xdebug working in PHPStorm.

Open PHPStorm, go to:

File -> Settings -> PHP -> Debug

Set the debug port to the port indicated in the xdebug settings in the dockerfile (which is currently 9011).

Then, in the Top right corner you will see:

Run / Debug Configurations

You will need to create a PHP Remote Debug profile (from the template) and name it something simple, such as 'xdebug' or "WCMS xdebug'.

To start listening, enable the IDE to listen to the port, set breakpoints and run the 'xdebug' profile debug.