On Google Cloud Platform

This describes how to set up appfl gRPC server to run on Google Cloud platform, while the clients are training the model locally. In this tutorial, we describe the steps to run a simple example by using the following services:

Note

We recommend to use Endpoints, as we observed that gRPC server running on Cloud Run occasionally experienced connection issue. See https://github.com/APPFL/APPFL/issues/56.

For the tutorials, complete the following steps first.

Creating a Google project

We need to create a Google project and set the following environment variable for convenience:

export GCP_PROJECT=<YOUR_PROJECT_ID>

Preparing a Docker container

The tutorials on Google Cloud Platform use a Docker container to run a gRPC server. A simple Dockerfile can be written as follows:

1FROM python
2
3RUN python3 -m pip install --upgrade pip
4RUN python3 -m pip --no-cache-dir install appfl
5
6WORKDIR /APPFL
7COPY grpc_mnist_server.py .
8
9CMD ["python3", "grpc_mnist_server.py", "--nclients=1"]

A Docker container can also be built from the repository:

 1FROM ubuntu
 2
 3RUN apt-get update
 4RUN apt-get install git python3 python3-pip -yq
 5
 6# This avoids to cache git clone.
 7ADD https://api.github.com/repos/APPFL/APPFL/git/refs/heads/kkim/grpc version.json
 8RUN git clone -b kkim/grpc https://github.com/APPFL/APPFL.git
 9
10WORKDIR /APPFL
11COPY grpc_mnist_server.py .
12
13RUN python3 -m pip install --upgrade pip
14RUN python3 -m pip --no-cache-dir install .
15
16CMD ["python3", "grpc_mnist_server.py", "--nclients=1"]

Building a Docker container

We build a docker container by using the tag name gcr.io/$GCP_PROJECT/appfl-test:latest.

docker build -t gcr.io/$GCP_PROJECT/appfl-test:latest .

Test the Docker container on local machine

To run the docker image locally:

docker run -d -p 50051:50051 -e PORT=50051 gcr.io/$GCP_PROJECT/appfl-test:latest

Once the image running successfully, we can run the appfl client from the file examples/gcloud/grpc_mnist_client.py in the repository.

python grpc_mnist_client.py --host=localhost --port=50051 --nclients=1 --client_id=1

Deploying the Docker container to the cloud

To deploy the docker container to the Google cloud, we first need to configure gcloud:

gcloud auth configure-docker

Now we upload the docker container to the cloud:

docker push gcr.io/$GCP_PROJECT/appfl-test:latest