Streamline EMR

Packaging and installation tools for the Streamline EMR web app.

Setup

1. login to the gitlab docker registry: docker login registry.gitlab.com
2. update submodules: git submodule update --init --recursive

Docker Images

We want to use our own copy of the Docker images hosted on our GitLab Registry. The docker folder contains a set of Dockerfiles that should be used for generating images.

If not already configured, ensure appropriate docker credentials are available:

# Login to the GitLab registry (Signalytic packages)
docker login registry.gitlab.com

# Login to an external Docker Hub registry
docker login -u <user> -p <credentials>

The script docker/build-image can be used to automatically build, tag, and push images with standard naming conventions. To build and push all images, simple run build-image -p. To simply build images and skip pushing to a registry, omit the -p flag.

Building for another platform (e.g. building arm64 from amd64 host) may fail if the Dockerfile requires running certain commands. To build only specific images, dockerfiles may be specified with build-image -f <DOCKERFILE> [ -f <DOCKERFILE> ... ]

Other options can be specified to override the image name or version. See the help menu for full details: build-image -h

Conventions

• We are building for two platforms: linux/amd64 (x86_64) and linux/arm64 (aarch64)
• One images is currently required: streamline-emr
• Images will be stored to GitLab with the name: registry.gitlab.com/signalytic/client-external/streamline/streamline-emr/<image>/<arch>:<tag>
  • example: registry.gitlab.com/signalytic/client-external/streamline/streamline-emr/streamline-emr/arm64:latest

Building

To build, simply run ./package to generate a new application package.

The package script will assign a few variables by default, specifically:

CONFIG=${CONFIG:-${LOCATION}/config.json}
SRC=${SRC:-${LOCATION}/src}
BUILD_DIR=${BUILD_DIR:-${LOCATION}/build-packages}

If necessary, each of these may be overridden by running with VAR=VALUE ./package.

The hosted-app-utils project provides additional customization via command line arguments. This can be used to modify the package name, skip packaging of images, build for only one platform and more. For a list of options, run ./package -h.

Helpful options during development:

• --no-compression: skip image compression to speed up packaging (packages will be much larger)
• --app: skip packaging of images altogether

Installing

Automated (Filetransfer)

1. Copy package to the management node (e.g. scp build-packages/<package> <user>@<mgmt-node-url>:)
2. SSH into the management node and become root (e.g. ssh <user>@<mgmt-node-url>, sudo su)
3. (optional) Prepare a filetransfer channel (e.g. mkdir -p /var/signalytic/filetransfer/<app-name>-<arch>)
4. Prepare a filetransfer update:

mkdir -p tmp/<update-number>
mv <package> tmp/<update-number>/
openssl sha256 -binary tmp/<update-number>/<package> | base64 - > tmp/<update-number>/_hash_

5. Copy the filetransfer update to the appropriate channel (e.g. mv tmp/1 /var/signalytic/filetransfer/<app-name>-<arch>/)
6. (optional) Restart filetransfer: systemctl restart filetransfer
7. If necessary, add the package to the target nodes' selective sync lists

bgkvs --cred admin:<pass> -B _conf -N _shared_ -d <node> -k filetransfer.selective_sync_packages.[0=<pkg1>
bgkvs --cred admin:<pass> -B _conf -N _shared_ -d <node> -k filetransfer.selective_sync_packages.[1=<pkg2>
bgkvs --cred admin:<pass> -B _conf -N _shared_ -d <node> -k filetransfer.selective_sync_packages.[2=<pkg3>

Manual

1. extract the generated archive file in a temporary folder on the target system
2. run <package-name>/install.sh