Update Production CanDIG

As new stable versions of the CanDIGv2 stack are released, it is recommended that production deployments be updated as soon as possible as deployments at different versions are unlikely to be able to be federated.

The process for updating depends on the changes contained in a particular release. The release notes for a particular release should indicate whether an existing deployment needs a full reinstall or can be updated without deleting any existing data. The steps in the process should follow the following general steps:

Step-by-step guide to updating CanDIG

Backup your existing deployment using the steps on the page: Backing up and restoring CanDIG data. This should be done regardless of whether you think the deployment will destroy any data, just in case anything goes wrong. Things you will want to back up are: - ingested genomic and clinical data - Guide here: Back up postgres databases - log files - Guide here: backup logfiles - user authorizations and federation information - Guide here: backup secrets and authorizations

git repos diffs - see expandable section below:

Example bash script of how to save git diffs of all CanDIGv2 repos

#!/bin/bash

current_date=$(date +%F)

# Set the output file path in the tmp/logs directory
output_file="tmp/logs/diff-$current_date.diff"

# Create the tmp/logs directory if it doesn't exist
mkdir -p tmp/logs

# Clear the output file if it exists
> $output_file

# Get the diff of the main repo and append to the file
echo "Diff for the main repo:" >> $output_file
git diff --no-ext-diff --submodule=diff >> $output_file
echo "---------------------------------------------------" >> $output_file

# Loop through each subrepo in the 'lib' directory
for dir in lib/*/; do
  if [ -d "$dir/.git" ]; then # Enter the subrepo directory and get the diff
    echo "Diff for subrepo $dir:" >> $output_file
    (cd "$dir" && git diff) >> $output_file
    echo "---------------------------------------------------" >> $output_file
  fi
done

echo "Git diff for main repo and subrepos has been saved to $output_file."

Pull the latest stable release from the CanDIGv2 repo and update submodules
Terminal window
```
cd CanDIGv2
git checkout stable
git pull
git submodule update --init --recursive
```
Ensure the candig conda environment is activated and update to the latest requirements
Terminal window
```
conda activate candig
pip install -r etc/venv/requirements.txt
```
Backup your current .env and modify the latest example.env
Terminal window
```
cp .env .env-bak
make migrate-env
```
The make target will copy the existing values to the newly formatted env file.

:::

You can use a file comparison tool or use the bash diff command to see any changes between the two files
Apply any other deployment specific changes such as those found when performing a git diff in step 1
If the release allows for a non-destructive rebuild, use the rebuild-keep-data target to clean and recompose services without destroying any data. It is still recommended you back up all data before proceeding with this method.
Terminal window
```
make rebuild-keep-data
```
If you have removed the default site admin, you will need to enter your admin credentials when prompted by the build process. Be sure to pay attention in the terminal, the build will be stuck until you enter the credentials.

This target will clean, build and compose services but will not delete any ingested data, user authorization information or logging information.

Proceed to step 9.
If the release needs a full rebuild, all services will need to be cleaned, rebuilt and recomposed and any existing data reingested or restored from backups. This is done by running the following commands

This destroys all data in the system so it is important to ensure there is a backup of all existing data before proceeding with this method.
Terminal window
```
make clean-all
make build-all
```
If you performed a destructive rebuild, it will be necessary to reingest and restore backed up data following the guide. This includes any clinical and genomic data as well as information about user authorizations. Federating to other nodes will also need to be reinitiated with all other nodes in the network. Federating with nodes at a different stable version may not be possible.

Depending on the changes that were made, it may not be possible to directly restore from backup, e.g. if the underlying structure of the postgres databases has changed, the data will need to be reingested through ingest, rather than restored directly to the database.
Run integration tests to ensure all the services are running as expected

In a production environment, use the make command for prod integration tests as follows:
Terminal window
```
make test-integration-prod SITE_ADMIN_TOKEN=<token>
```