Duo Developerandadmin
b9d8c82f19
utils_dir, temp_dir, build_dir, and staging_dir were computed at module level but never referenced. tempfile.mkdtemp() ran unconditionally on every invocation (including --help and argument-parse errors), leaking an empty temp directory on disk each time. Remove all four variables and the now-unused tempfile import.
Streamline EMR
Packaging and installation tools for the Streamline EMR web app.
Setup
- login to the gitlab docker registry:
docker login registry.gitlab.com - update submodules:
git submodule update --init --recursive
Docker Images
We want to use our own copy of the Docker images hosted on our GitLab Registry. The docker folder contains a set of Dockerfiles that should be used for generating images.
If not already configured, ensure appropriate docker credentials are available:
# Login to the GitLab registry (Signalytic packages)
docker login registry.gitlab.com
# Login to an external Docker Hub registry
docker login -u <user> -p <credentials>
The script docker/build-image can be used to automatically build, tag, and push images with standard naming conventions. To build and push all images, simple run build-image -p. To simply build images and skip pushing to a registry, omit the -p flag.
Building for another platform (e.g. building arm64 from amd64 host) may fail if the Dockerfile requires running certain commands. To build only specific images, dockerfiles may be specified with build-image -f <DOCKERFILE> [ -f <DOCKERFILE> ... ]
Other options can be specified to override the image name or version. See the help menu for full details: build-image -h
Conventions
- We are building for two platforms:
linux/amd64(x86_64) andlinux/arm64(aarch64) - One images is currently required:
streamline-emr - Images will be stored to GitLab with the name:
registry.gitlab.com/signalytic/client-external/streamline/streamline-emr/<image>/<arch>:<tag>- example:
registry.gitlab.com/signalytic/client-external/streamline/streamline-emr/streamline-emr/arm64:latest
- example:
Building
To build, simply run ./package to generate a new application package.
The package script will assign a few variables by default, specifically:
CONFIG=${CONFIG:-${LOCATION}/config.json}
SRC=${SRC:-${LOCATION}/src}
BUILD_DIR=${BUILD_DIR:-${LOCATION}/build-packages}
If necessary, each of these may be overridden by running with VAR=VALUE ./package.
The hosted-app-utils project provides additional customization via command line arguments. This can be used to modify the package name, skip packaging of images, build for only one platform and more. For a list of options, run ./package -h.
Helpful options during development:
--no-compression: skip image compression to speed up packaging (packages will be much larger)--app: skip packaging of images altogether
Installing
Automated (Filetransfer)
- Copy package to the management node (e.g.
scp build-packages/<package> <user>@<mgmt-node-url>:) - SSH into the management node and become root (e.g.
ssh <user>@<mgmt-node-url>,sudo su) - (optional) Prepare a filetransfer channel (e.g.
mkdir -p /var/signalytic/filetransfer/<app-name>-<arch>) - Prepare a filetransfer update:
mkdir -p tmp/<update-number>
mv <package> tmp/<update-number>/
openssl sha256 -binary tmp/<update-number>/<package> | base64 - > tmp/<update-number>/_hash_
- Copy the filetransfer update to the appropriate channel (e.g.
mv tmp/1 /var/signalytic/filetransfer/<app-name>-<arch>/) - (optional) Restart filetransfer:
systemctl restart filetransfer - If necessary, add the package to the target nodes' selective sync lists
bgkvs --cred admin:<pass> -B _conf -N _shared_ -d <node> -k filetransfer.selective_sync_packages.[0=<pkg1>
bgkvs --cred admin:<pass> -B _conf -N _shared_ -d <node> -k filetransfer.selective_sync_packages.[1=<pkg2>
bgkvs --cred admin:<pass> -B _conf -N _shared_ -d <node> -k filetransfer.selective_sync_packages.[2=<pkg3>
Manual
- extract the generated archive file in a temporary folder on the target system
- run
<package-name>/install.sh