NVIDIA Clara Parabricks Pipelines: Local Installation

Get NVIDIA Clara Parabricks Pipelines up and running on any linux system with NVIDIA GPUs.


Prerequisites

The following are required to install Parabricks:

  • Access to the internet
  • nvidia-driver that supports cuda-9.0 or higher
  • nvidia-driver that supports cuda-10.0 or higher if you want to run deepvariant or cnnscorevariants
  • nvidia-docker or singularity version 2.6.1 or higher
  • Python 3
  • curl (Most Linux systems will already have this installed)

The following are the hardware requirements

  • Run on any GPU that supports CUDA architecture 60, 61, 70, 75 and has 12GB GPU RAM or more. It has been tested on NVIDIA P100, NVIDIA V100, and NVIDIA T4 GPUs.
    • 1 GPU server should have 64GB CPU RAM, at least 16 CPU threads
    • 2 GPU server should have 100GB CPU RAM, at least 24 CPU threads
    • 4 GPU server should have 196GB CPU RAM, at least 32 CPU threads
    • 8 GPU server should have 392GB CPU RAM, at least 48 CPU threads

Installation Steps

 

The user must have received the parabricks.tar.gz file from NVIDIA. This package contains all items necessary to install and run the Parabricks software. Please request access from developer.nvidia.com/clara-parabricks.

Install the Parabricks package to your system:

# Step 1: Unzip the package.
tar -xzf parabricks.tar.gz

# Step 2: Run the installer
sudo ./parabricks/installer.py

# Step 3: verify your installation.
# This should display the parabricks version number:
pbrun version

Installation Options

The default installation option for the package received by you, should have the correct options. However, you can modify the following options.

Options

Details

--install-location

Location in which a folder by the name of parabricks will be created. The parabricks folder will contain everything needed to run the software. The user must have permission to write to this location. (default: /opt/)

--container

Container technology to use. Currently we support, nvidia-docker, docker with nvidia-container-runtime support and singularity version 2.6.1

--release

The parabricks release version you want to install. Contact parabricks-support@nvidia.com, if you do not want the default version and need more information

--cpu-only

Install on a CPU server. No GPU accelerated tools will run with this option

--symlink

Create a symlink at /usr/bin/pbrun. You can choose to do this during the installation

--arch

Default architecture of the hardware. Currently we support x64_64 or ppc64le

--uninstall

Uninstall parabricks. Removes all the images and scripts from --install-location

--force

Disable interactive installation (default: False)

--ngc

Pull image from NGC (default: False)

--disable-fakeroot

Disable the use of fakeroot for Singularity lower than version 3.3. You will need sudo to build for singularity then. (default: False)

--ext3-file EXT3_FILE

If mkfs.ext3 is not installed on your system, you will need to create a ext3 filesystem image for overlaying singularity containers. You can pass the ext3 filesystem image using this option. On most system you can create an ext3 filesystem images using dd if=/dev/zero of=ext3.img bs=1024 count=1048576 mkfs.ext3 ext3.img (default: None)

 

Package contents:

  • installer.py: Installer to download and install the software
  • license.bin: License file based on your license with NVIDIA Parabricks
  • EULA.txt: End User License Agreement

Installing with Singularity 3.0 or higher

1) Log in to a machine with sudo access and singularity 3.0 or higher.
2) To download the software:
$ wget -O parabricks.tar.gz " < DOWNLOAD_LINK > "
3) Unzip the package
$ tar -xvzf parabricks.tar.gz
4) Do a local installation:$ mkdir localdir
$ sudo ./parabricks/installer.py --install-location localdir --container singularity
5) Everything will be installed in localdir/parabricks folder. Tar this folder.
$ cd localdir
$ tar -cvzf parabricks_install.tar.gz parabricks
6) and copy it to the node which will be used for testing and has singularity v3.x.
7) Untar the tar.gz file in some <INSTALL_DIR>$ cd <INSTALL_DIR>
$ tar -xvzf parabricks_install.tar.gz


Uninstalling Parabricks

To uninstall, you can run the uninstall tool:

./parabricks/installer.py --uninstall