home-llm/Readme.md

home-llm Docker Compose

So far, just the docker-compose.yml file used to pull and run the containers for ollama and open-webui.

nvidia-container Installation

Step 1: Download the NVIDIA Docker Packages

Download the NVIDIA Container Toolkit and its dependencies:

You can use the following commands to download the necessary packages. Make sure to adjust the version numbers if needed:

bash

    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/nvidia-docker2_2.16.0-1_amd64.deb
    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/nvidia-container-runtime_3.11.0-1_amd64.deb
    wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/nvidia-container-toolkit_1.12.0-1_amd64.deb

Note: The version numbers may change, so you might want to check the NVIDIA website for the latest versions.

Step 2: Install the Downloaded Packages

Install the downloaded packages:

Run the following command to install the packages:

bash

sudo dpkg -i nvidia-container-runtime_3.11.0-1_amd64.deb nvidia-container-toolkit_1.12.0-1_amd64.deb nvidia-docker2_2.16.0-1_amd64.deb

If you encounter any dependency issues, you can resolve them by running:

bash

sudo apt-get install -f