Oobabooga - text-generation-webui auto installation (Ubuntu 22.04.3)

Life can be hectic, and we're all looking for ways to make it easier. One solution gaining momentum is automation. When it comes to installation processes, automating them can be a game-changer. In this blog, we'll demonstrate how automation can make a complex tool like Oobaboga accessible to a wider audience by providing an auto-install script in this post. Let's do this

First we need to install Oobabooga text-generation-webui, you will be prompted 2 times during this install, for your GPUs / CPUs and about using old GPUs (very old)

#!/bin/bash

# Install git if not present
sudo apt update -y
sudo apt upgrade -y
sudo apt install git -y

# Navigate to /opt directory to deploy Oobabooga
cd /opt

# Clone the repository
sudo git clone https://github.com/oobabooga/text-generation-webui.git

# Give some rights to the current user on the newly created directory
sudo chown your_user:xxx -R text-generation-webui/

# Initial deployment
cd text-generation-webui
sudo apt install curl
bash start_linux.sh

# Answer the two questions about your GPU / CPU

You can stop here, text-generation-webui is ready to go if you don't want /need to add the Text To Speech extension

Then we need to install Cuda Toolkit

#!/bin/bash

# Download the CUDA repository PIN file
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

# Download the CUDA repository package
wget https://developer.download.nvidia.com/compute/cuda/12.3.2/local_installers/cuda-repo-ubuntu2204-12-3-local_12.3.2-545.23.08-1_amd64.deb

# Install the CUDA repository package
sudo dpkg -i cuda-repo-ubuntu2204-12-3-local_12.3.2-545.23.08-1_amd64.deb

# Copy the CUDA keyring to the appropriate location
sudo cp /var/cuda-repo-ubuntu2204-12-3-local/cuda-*-keyring.gpg /usr/share/keyrings/

# Update the package list
sudo apt-get update

# Install CUDA toolkit version 12.3
sudo apt-get -y install cuda-toolkit-12-3

# cleanup
sudo apt autoremove

Now we need to install the Text To Speech extention

#!/bin/bash

# Navigate to the text-generation-webui/extensions directory
cd /opt/text-generation-webui/extensions

# Clone the repository
git clone https://github.com/erew123/alltalk_tts

# Enter the virtual environment
bash ../cmd_linux.sh

# Install requirements (use requirements_nvidia.txt if you have an NVIDIA GPU, requirements_other.txt if not)
pip install -r extensions/alltalk_tts/requirements_nvidia.txt
exit

# Launch the text-generation-webui script again
bash start_linux.sh --listen

(mandatory) if you want to enable deepspeed you need to execute these commands (manualy)

# Navigate back to the text-generation-webui directory
cd /opt/text-generation-webui

# Enter the virtual environment
bash cmd_linux.sh

# Change the environment variable for CUDA_HOME
conda env config vars set CUDA_HOME=/etc/alternatives/cuda

# Check if it's saved
export CUDA_HOME=/etc/alternatives/cuda
exit

# Enter the virtual environment
bash ./cmd_linux.sh

# Install Deepspeed
pip install deepspeed
exit

# Launch the text-generation-webui script again
bash start_linux.sh --listen

Boom you're good to go !

Happy LLM adventures !

Oobabooga Github repo :