Streamline EMR

Packaging and installation tools for Streamline's "Streamline EMR" application.

Building

1. ensure you have docker access to this project's container registry on GitLab
2. run package.sh to generate a new application package
3. package will be stored to build-packages/

Alternative Builds

Some variables can be overridden to support alternative builds. For example, an option exists to package the application WITHOUT docker images. This can be done by setting the appropriate environment variables (e.g. VAR=VALUE ./package.sh).

Options that can be overridden are:

• PACKAGE_NAME: base name of the output package. (default: streamline-emr)
• OUTPUT_DIR: directory where packages will be stored. (default: $PROJECT_ROOT/build-packages)
• PACKAGE_IMAGES: if set to a value other than 1, do NOT package images. (default: 1)
• IMAGE_TAG: if set, used to specify a source image tag. (default: uses git-based version name)
• IMAGES_LIST: if set, specifies an alternate image list file. (default: $PROJECT_ROOT/images.csv)

For example, to build using the absolute latest images available:

IMAGE_TAG=latest package.sh

Installing

1. extract the generated archive file in a temporary folder on the target system
2. run <package-name>/install.sh
3. add a instance.conf file at the root of the install directory (see notes below)
4. (optional) restart the systemd service if running on a system with systemd installed: systemctl restart streamline-emr
5. (optional) configure an app.conf file for integration with Signalytic services (e.g. dbsync)

Running

Production (systemd)

On a production system, systemd will manage the lifecycle of the application, e.g.

# enable and start the the application service
systemctl daemon-reload
systemctl enable streamline-emr
systemctl start streamline-emr

# stop the application service
systemctl stop streamline-emr

# restart the application service
systemctl restart streamline-emr

Monitoring

The script monitor can be used at any time to query the current status of all instances. If the bgsystemlog program is found, status data will be written to the Signalytic system log under the streamline_emr location (json format).

Docker

In a docker environment, the application must be started manually. This is done by running the entry script, for example:

/var/signalytic/clientapps/streamline/streamline-emr/entry

Note that the entry script will block until the first service stops or until interrupted.

If multiple instances are run from the same machine, additional configuration may be required to differentiate the instances. This is generally the case when using the "sibling container" method for running docker from within a docker container. See the Configuration section for more details.

Configuration

Certain configuration options can be adjusted by setting environment variables or by defining these variables in a .env file stored in the same directory as the docker-compose.yml file. These variables are: APP_PORT: port mapping for the web app server CONTAINER_NAME: name of the application container DATA_DIR: directory to be mounted for peristant data COMPOSE_PROJECT_NAME: assign the project name associate with the compose file - this avoids conflicts when the "same" compose file is used to manage multiple instances (e.g. when using docker sibling containers) IMAGE: optional, overrides the default streamline-emr:2.0 image

Each instance of the app uses its own .env file to specify its configuration. This is automated through the start script using information stored in a file instances.conf. This file defines two lists: INSTANCES: a list of unique instance names APP_PORTS: a corresponding list of port numbers to map to the local host

Important note when using "sibling containers": the sibling container method uses the host docker socket to provide containers with access to docker. As a result, port mappings and volume mount points all refer to the host system rather than the container from which the docker command originates.

Example Configuration Files

The following snippet shows an instances.conf file with 3 instances defined. Instance test0001-_default will use port 3001, test0002-_default will use 3002, etc.

INSTANCES="test0001-_default" "test0002-_default" "test0003-_default"
APP_PORTS=3001 3002 3003

App Configuration

An app.conf file is required for integration with other Signalytic services. A template is provided which demonstrates integration with the dbsync service.

Scripts

Several scripts are used to manage app instances, including: entry: main entry script used to start all configured instances - does not return until a service stops or the process is interrupted. start: loads configuration details from instances.conf, then configures and starts each application instance in a dedicated subfolder stop: locates and stops all instances, even if no longer defined in instances.conf monitor: checks the status of each instance service and sends data to bgsystemlog if possible (loc: streamline_emr)

Docker Images

We want to use our own copy of the Docker images hosted on our GitLab Registry. The docker folder contains a set of Dockerfiles that should be used for generating images.

Automated

The script build-image can be used to automatically build, tag, and push images with standard naming conventions. To build and push all images, simple run build-image -p. To simply build images and skip pushing to a registry, omit the -p flag.

Building for another platform (e.g. building arm64 from amd64 host) may fail if the Dockerfile requires running certain commands. To build only specific images, dockerfiles may be specified with build-image -f <DOCKERFILE> [ -f <DOCKERFILE> ... ]

Other options can be specified to override the image name or version. See the help menu for full details: build-image -h

Manual

1. Make sure you are logged in to the Signalytic GitLab Registry:

   # Login to the registry
   docker login registry.gitlab.com
   

2. Build an image (note: we're using platforms linux/amd64 and linux/arm64)

   # Build an image
   docker build -f ${DOCKERFILE} --platform ${PLATFORM} -t ${IMAGE_TAG} .
   

3. Make a copy of an existing image:

   # Create a tag
   docker tag ${IMAGE_ID} registry.gitlab.com/signalytic/client-external/streamline/streamline-emr/${NAME}:${TAG}
   

4. Push the image to our GitLab Registry:

   # Push the image
   docker push registry.gitlab.com/signalytic/client-external/streamline/streamline-emr/${NAME}:${TAG}