Basic Usage
Deployment
The easiest way to deploy the Cargoship models on your own server is using Docker and Traefik. Traefik is a reverse proxy that can be used to route traffic to your models. It can also be used to automatically generate SSL certificates using Let's Encrypt. You can also use your own domain name and configure Traefik to use it.
Install Docker
We are starting with a clean Ubuntu 22.04 server. First, we need to install Docker. You can find the official installation instructions here: https://docs.docker.com/engine/install/ubuntu/
First make sure you have no other Docker installation on your system and remove it:
$ sudo apt-get remove docker docker-engine docker.io containerd runc
Then install Docker. First update your package index:
sudo apt-get update
Then install the required packages:
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
Add Docker's official GPG key:
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
Use then use the following command to set up the repository:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Update your package index again:
sudo apt-get update
Finally, install Docker:
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
Test your Docker installation:
sudo docker run hello-world
If everything went well, you should see an output downloading the hello-world image and then running it.
To make interacting with docker easier, you also can add your current user to the docker group (that way you can avoid using sudo with docker commands). First create the group:
sudo groupadd docker
Then add your user to the group:
sudo usermod -aG docker $USER
Run this command to activate the changes to groups:
newgrp docker
Install Traefik
We use Traefik as an easy to use webserver and to make all the models available to the outside under different domains. Traefik also takes care of automatically generating SSL certificates using Let's Encrypt.
First choose in which directory you want to install traefik and all the models we are hosting. We will use the root users home directory (/root
) in this example.
Switch to home directory:
cd ~
Create a directory for traefik and switch to it:
mkdir traefik && cd traefik
Create a docker-compose.yml file:
touch docker-compose.yml
Open the file with your favorite editor (e.g. nano docker-compose.yml
) and add the following content:
version: "3.5"
services:
traefik:
image: "traefik:v2.7"
restart: always
container_name: "traefik"
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /root/traefik/traefik.yaml:/traefik.yaml
- /root/traefik/acme.json:/acme.json
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- web
networks:
web:
external: true
If you use a different directory, make sure to change the paths in the volumes section.
Create a traefik.yaml file:
touch traefik.yaml
Open the file with your favorite editor (e.g. nano traefik.yaml
) and add the following content:
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: websecure
scheme: https
permanent: true
websecure:
address: ":443"
http:
tls:
certResolver: default
providers:
docker:
watch: true
exposedByDefault: false
network: web
certificatesResolvers:
default:
acme:
email: YOUR-EMAIL-ADDRESS # TODO: replace with your email address
storage: acme.json
caServer: "https://acme-v01.api.letsencrypt.org/directory"
tlsChallenge: {}
Make sure to replace YOUR-EMAIL-ADDRESS
with your email address. This is required by Let's Encrypt to send you notifications about your certificates.
To store your SSL certificates locally, create a file called acme.json
:
touch acme.json
And set the correct permissions:
chmod 600 acme.json
Now you can start traefik:
docker-compose up -d
You can check the status of traefik with:
docker-compose ps
You can also check the logs with:
docker-compose logs -f
Now that traefik is running, you can start other docker containers on your system and make them available to the outside world using traefik. Please make sure, that the ports 443 and 80 are accessible to the outside world and not blocked by your firewall or the firewall of your hoster.
Run your first model
Now that traefik is running, we can start our first model. We will use the Cargoship model cargoshipsh/language-detection
as an example.
First, move back to your base directory (in our case the roots home directory /root
or ~
):
cd ~
Create a directory for the model and switch to it:
mkdir language-detection && cd language-detection
Create a docker-compose.yml file:
touch docker-compose.yml
Open the file with your favorite editor (e.g. nano docker-compose.yml
) and add the following content:
version: "3.3"
services:
language-detection-api:
restart: unless-stopped
image: cargoshipsh/language-detection:latest
environment:
- API_KEY=CHANGE_ME # TODO: Change to your API key
networks:
- web
labels:
- "traefik.enable=true"
- "traefik.http.routers.language-detection.rule=Host(`api.example.com`)" # TODO: Change with your own domain
- "traefik.http.routers.language-detection.tls.certresolver=default"
- "traefik.http.routers.language-detection.entrypoints=websecure"
- "traefik.http.services.language-detection.loadbalancer.server.port=80"
networks:
web:
external: true
Make sure to replace CHANGE_ME
with your API key you want to use to access the API and api.example.com
with your own domain.
(Note: if you want to deploy more than one model, make sure the router names, e.g. traefik.http.routers.language-detection
, and the domain names are unique.)
Make sure your domain is pointing to the IP address of your server. You can check your IP address with:
curl ifconfig.me
If you want to host different models, you can also use subdomains, e.g. language-detection.api.example.com
and only set a wildcard DNS record for *.api.example.com
once.
Now you can start the model:
docker-compose up -d
You can check the status of the containers with:
docker-compose ps
You can also check the logs with:
docker-compose logs -f
Your model should now be available under your domain. You can test it with:
curl -X POST \
https://api.example.com \
-H 'Content-Type: application/json' \
-H 'X-API-KEY: CHANGE_ME' \
-d '{
"text": "This is a test"
}'
Make sure to replace the API key and also address with the domain you chose. You should get a response like this:
{
"language": "en"
}
If you get an error, make sure that your domain is pointing to the IP address of your server and that the ports 80 and 443 are accessible to the outside world. You can also check the logs of the traefik container with docker logs -f traefik
.
Have a lot of fun with your new machine learning API! 🎉🎉🎉