On Google Cloud Platform

This describes how to set up appfl gRPC server to run on Google Cloud platform, while the clients are training the model locally. In this tutorial, we describe the steps to run a simple example by using the following services:

• How to run with Endpoints for Compute Engine with ESP
• How to run on Cloud Run

Note

We recommend to use Endpoints, as we observed that gRPC server running on Cloud Run occasionally experienced connection issue. See APPFL/APPFL#56.

For the tutorials, complete the following steps first.

Creating a Google project

We need to create a Google project and set the following environment variable for convenience:

export GCP_PROJECT=<YOUR_PROJECT_ID>

Preparing a Docker container

The tutorials on Google Cloud Platform use a Docker container to run a gRPC server. A simple Dockerfile can be written as follows:

FROM python

RUN python3 -m pip install --upgrade pip
RUN python3 -m pip --no-cache-dir install appfl

WORKDIR /APPFL
COPY grpc_mnist_server.py .

CMD ["python3", "grpc_mnist_server.py", "--nclients=1"]

A Docker container can also be built from the repository:

FROM ubuntu

RUN apt-get update
RUN apt-get install git python3 python3-pip -yq

# This avoids to cache git clone.
ADD https://api.github.com/repos/APPFL/APPFL/git/refs/heads/kkim/grpc version.json
RUN git clone -b kkim/grpc https://github.com/APPFL/APPFL.git

WORKDIR /APPFL
COPY grpc_mnist_server.py .

RUN python3 -m pip install --upgrade pip
RUN python3 -m pip --no-cache-dir install .

CMD ["python3", "grpc_mnist_server.py", "--nclients=1"]

Building a Docker container

We build a docker container by using the tag name gcr.io/$GCP_PROJECT/appfl-test:latest.

docker build -t gcr.io/$GCP_PROJECT/appfl-test:latest .

Test the Docker container on local machine

To run the docker image locally:

docker run -d -p 50051:50051 -e PORT=50051 gcr.io/$GCP_PROJECT/appfl-test:latest

Once the image running successfully, we can run the appfl client from the file examples/gcloud/grpc_mnist_client.py in the repository.

python grpc_mnist_client.py --host=localhost --port=50051 --nclients=1 --client_id=1

Deploying the Docker container to the cloud

To deploy the docker container to the Google cloud, we first need to configure gcloud:

gcloud auth configure-docker

Now we upload the docker container to the cloud:

docker push gcr.io/$GCP_PROJECT/appfl-test:latest