Docker for SCOT¶
Updated - 12/10/2019
Table of Contents¶
- Docker-SCOT containers
- Managing the containers
- FAQ / Common Issues
SCOT’s primary deployment technology is now via docker.
Backup your database via the backup.pl in the /opt/scot/bin/ directory before upgrading to the docker version of SCOT. If you are upgrading, you will also need to turn off all services that the older version of SCOT uses such as Apache, Activemq, Mongodb, ElasticSearch and SCOT (i.e. sudo service stop scot). Also as far as upgrading, we have not tested upgrading from any version before 3.4. Upgrade from versions prior to 3.4 to 3.5 first before upgrading to Docker-SCOT.
SCOT is comprised of the following services:
- SCOT - SCOT Application and associated API
- MongoDB - Storage for SCOT
- ActiveMQ - Message broker for servies interested in SCOT data
- Apache - Proxy for traffic between some services
- ElasticSearch - Search engine
- Flair Engine - ‘Entities’ found within SCOT are highlighted with a count of the number of times SCOT has ‘seen’ them before
- Game Engine - Used for homepage statistics
- Stretch - Used for adding data to ElasticSearch
- Mail - Used as a reslient mechanism for importing data to SCOT (not enabled by default - See configuration section)
- Reflair Similar to flair
To get started, refer to the Docker Community Edition documentation for installing the Docker engine on your respective OS: https://docs.docker.com/engine/installation/
Next, Docker-SCOT relies on docker-compose to build, run and manage services. Docker-compose does not ship with Docker engine, so you will need to refer to the following documentation for installation of Docker-Compose: https://docs.docker.com/compose/install/
Note These steps will most likely change slightly in 2019 as the SCOT team will be working on making the install script more robust, but easier to use so you can begin using (and even developing) with SCOT quickly.
There are two methods for getting started with SCOT. Run the SCOT/restart-build-deploy.sh script (will be promopted to enter sudo credentials) and follow the on screen prompts for either.
- Quick mode (fast) - this mode will pull all necessary docker images from from Dockerhub (preconfigured). As for 12/10/2019, this is the preferred method for SCOT installation. If you need to enable LDAP auth, configure TLS certificates, or other changes, these can be done in this mode but with volume mounts and possibly some manipulation of compose files / configs.
- Custom Mode (slow) - This mode should only be chosen if you are wanting to rebuild docker images for further customization.
Managing the containers¶
The restart-build-deploy.sh script will handle stopping and then restarting containers automaticaly. However if you need more granular control run the following:
To stop Docker-SCOT:
sudo docker-compose stop
To start a specific service:
sudo docker-compose up --build name_of_service
To stop a specific service:
sudo docker-compose stop name_of_of_service
To restart a specific service and build in any particular changes you have made to source:
sudo docker-compose up -d --build name_of_service
SCOT’s implementation of docker relies on the docker-compose.yml or docker-compose-custom.yml file to define the execution of the services, the DockerFiles that define the dependencies for each container, and two directories (docker-scripts & docker-configs).
The docker-compose.yml references the prebuilt images from Dockerhub.
The docker-compose-custom.yml file will build the SCOT docker images from source.
The docker-scripts directory contains scripts for backing up the data contained in MongoDB container and will eventually house other scripts that are similar.
The following scripts are currently supported:
To execute one of the above scripts, simply connect to the scot container via:
sudo docker exec -i -t -u scot scot /bin/bash
cd to /opt/scot/bin/
Restoring a database
For any questions about backing up and restoring databases, please contact the SCOT development team.
The docker-configs directory contains modified config files, perl modules, scripts, etc. that allow SCOT to function properly in a containerized environment. Most changes are references to localhost in the standard SCOT codebase where we modify those addresses to reference the ip addresses on the scot_docker subnet.
MongoDB Default password
MongoDB default password (also used for logging in to SCOT if local auth is enabled (by default)), is:
- Username: admin
- Password: admin
Note: If by chance you ever go to wipe your mongo database and would like to start fresh, you would need to delete the file /var/lib/mongodb/.mongodb_password_set.
You can view which data is being persisted by viewing the docker-compose.yml script and referring to the various ‘Volumes’. With regard to MongoDB (where SCOT records are persisted), the data from mongodb is persisted via nmaed volumes to /var/lib/docker/volumes/mongodb_data..
To begin using mail, you will need to uncomment the ‘mail’ service in the docker-compose.yml file and also add any of your organization’s mail configurations into the docker-configs/mail/alert.cfg.pl file.
By default, LDAP configuration is not enabled in docker-configs/scot/scot.cfg.pl. To enable, simply uncomment the LDAP configuration lines in docker-configs/scot/scot.cfg.pl and edit the necessary information to begin checking LDAP for group membership / auth.
Docker-SCOT’s Apache instance comes configured with a self-signed SSL cert baked into the container. However, if you wish to use your own ceritifcates, do the following:
Remove the SSL cert creation lines from the Dockerfile-Apache file.
In docker-configs/apache/ directory, there is a scot-revproxy-Ubuntu.conf. Replace the following line:
In the same file, replace the following lines:
SSLCertificateFile /etc/apache2/ssl/scot.crt SSLCertificateKeyFile /etc/apache2/ssl/scot.key
with the path and name of the eventual location where you will map your certs to via a shared data volume. 4. Next, as mentioned above, you need to pump your certs from your host machine into the container via a data volume (you can also copy them into the container at build time via COPY directive). In order to map them in via a data volume, add a new data volume under the apache service in the docker-compose.yml file. Eg.:
volumes: - "/path/to/your/cert:/path/to/file/location/you/defined/in/step/3 - "/path/to/your/key:/path/to/file/location/you/defined/in/step/3
- Re-run the restart-build-deploy.sh script and you should be set!
FAQ / Common Issues¶
- Apache frequently will throw an error on run time that the process is already running and will subequently die. In the event this happens, simply re-run the script.
- Complete backup and restore scripts in bash
- Update docs - better examples