How to set configuration#

In this notebook, we will provide details about how to set configurations for federated learning experiments.

Load default configuration#

APPFL empolys OmegaConf package, a hierarchical configuration system, for FL training configurations. OmegaConf package allows users to create a hierarchical configuration in DictConfig type from a python @dataclass. For example, we can load the APPFL default configuration dataclass using OmegaConf.structured() as shown below

[1]:

from appfl.config import Config
from omegaconf import OmegaConf
cfg = OmegaConf.structured(Config)

The configuration cfg is initialized with the default values. Let’s check the configuration values.

[2]:

print(OmegaConf.to_yaml(cfg))

fed:
  type: federated
  servername: ServerFedAvg
  clientname: ClientOptim
  args:
    server_learning_rate: 0.01
    server_adapt_param: 0.001
    server_momentum_param_1: 0.9
    server_momentum_param_2: 0.99
    optim: SGD
    num_local_epochs: 10
    optim_args:
      lr: 0.001
    use_dp: false
    epsilon: 1
    clip_grad: false
    clip_value: 1
    clip_norm: 1
device: cpu
device_server: cpu
num_clients: 1
num_epochs: 2
num_workers: 0
batch_training: true
train_data_batch_size: 64
train_data_shuffle: true
validation: true
test_data_batch_size: 64
test_data_shuffle: false
data_sanity: false
reproduce: true
pca_dir: ''
params_start: 0
params_end: 49
ncomponents: 40
use_tensorboard: false
load_model: false
load_model_dirname: ''
load_model_filename: ''
save_model: false
save_model_dirname: ''
save_model_filename: ''
checkpoints_interval: 2
save_model_state_dict: false
send_final_model: false
output_dirname: output
output_filename: result
logginginfo: {}
summary_file: ''
personalization: false
p_layers: []
config_name: ''
max_message_size: 104857600
operator:
  id: 1
server:
  id: 1
  host: localhost
  port: 50051
  use_tls: false
  api_key: null
client:
  id: 1
enable_compression: false
lossy_compressor: SZ2
lossless_compressor: blosc
compressor_sz2_path: ../.compressor/SZ/build/sz/libSZ.dylib
compressor_sz3_path: ../.compressor/SZ3/build/tools/sz3c/libSZ3c.dylib
compressor_szx_path: ../.compressor/SZx-main/build/lib/libSZx.dylib
error_bounding_mode: ''
error_bound: 0.0
flat_model_dtype: np.float32
param_cutoff: 1024

Most variables are self-explanatory. Specifically,

Variable fed sets the choice of FL algorithm and the algorithm-related parameters, and it is aslo defined as a python @dataclass. We provide the definition of those dataclasses at appfl.config.fed.*. In details,
- appfl.config.fed.federated is a general dataclass for all synchronous FL algorithms, where you can specify the server algorithm name at servername, client algorithm name at clientname, and all related arguments and paramters at args.
- appfl.config.fed.fedasync is a general dataclass for all asynchronous FL algorithms, whose args contains commonly-used parameters in asynchrnous FL.
- appfl.config.fed.iceadmm is a dataclass specifically wrote for the ICEADMM privacy-preserving FL algorithm, whose args contains all needed parameters for the ICEADMM algorithm.
- appfl.config.fed.iiadmm is a dataclass specifically wrote for the IIADMM privacy-preserving FL algorithm, whose args contains all needed parameters for the IIADMM algorithm.

Initialize configuration with arguments#

We can also initialize the configuration with other values. For example, the following code is loading the configuration with the algorithm choice of IIADMM.

[3]:

from appfl.config import fed
cfg = OmegaConf.structured(Config(
    fed = fed.iiadmm.IIADMM()
))
print(OmegaConf.to_yaml(cfg.fed))

type: iiadmm
servername: IIADMMServer
clientname: IIADMMClient
args:
  num_local_epochs: 1
  accum_grad: true
  coeff_grad: false
  optim: SGD
  optim_args:
    lr: 0.01
  init_penalty: 100.0
  residual_balancing:
    res_on: false
    res_on_every_update: false
    tau: 1.1
    mu: 10
  use_dp: false
  epsilon: 1
  clip_grad: false
  clip_value: 1
  clip_norm: 1

Change configuration values#

We can also change the configuration value after initialization. For example, we can change fed variable as follows:

[4]:

cfg = OmegaConf.structured(Config)
my_fed = OmegaConf.structured(fed.fedasync.FedAsync)
cfg.fed = my_fed
print(OmegaConf.to_yaml(cfg.fed))

type: fedasync
servername: ServerFedAsynchronous
clientname: ClientOptim
args:
  server_learning_rate: 0.01
  server_adapt_param: 0.001
  server_momentum_param_1: 0.9
  server_momentum_param_2: 0.99
  optim: SGD
  num_local_epochs: 10
  optim_args:
    lr: 0.001
  use_dp: false
  epsilon: 1
  clip_grad: false
  clip_value: 1
  clip_norm: 1
  K: 3
  alpha: 0.9
  staleness_func:
    name: constant
    args:
      a: 0.5
      b: 4
  gradient_based: false