Note
Go to the end to download the full example code.
Federated Node Classification with Homomorphic Encryption Example
Federated Node Classification with FedGCN and Homomorphic Encryption on the Cora dataset.
(Time estimate: 3 minutes)
Load libraries
import attridict
from fedgraph.federated_methods import run_fedgraph
Specify the Node Classification with Homomorphic Encryption configuration
config = {
# Task, Method, and Dataset Settings
"fedgraph_task": "NC",
"dataset": "cora",
"method": "FedGCN", # Federated learning method, e.g., "FedGCN"
"iid_beta": 10000, # Dirichlet distribution parameter for label distribution among clients
"distribution_type": "average", # Distribution type among clients
# Training Configuration
"global_rounds": 100,
"local_step": 3,
"learning_rate": 0.5,
"n_trainer": 2,
"batch_size": -1, # -1 indicates full batch training
# Model Structure
"num_layers": 2,
"num_hops": 1, # Number of n-hop neighbors for client communication
# Resource and Hardware Settings
"gpu": False,
"num_cpus_per_trainer": 1,
"num_gpus_per_trainer": 0,
# Logging and Output Configuration
"logdir": "./runs",
# Security and Privacy
"use_encryption": True, # Whether to use Homomorphic Encryption for secure aggregation
# Dataset Handling Options
"use_huggingface": False, # Load dataset directly from Hugging Face Hub
"saveto_huggingface": False, # Save partitioned dataset to Hugging Face Hub
# Scalability and Cluster Configuration
"use_cluster": False, # Use Kubernetes for scalability if True
}
Run fedgraph method
config = attridict(config)
run_fedgraph(config)
config: {'fedgraph_task': 'NC', 'dataset': 'cora', 'method': 'FedGCN', 'iid_beta': 10000, 'distribution_type': 'average', 'global_rounds': 100, 'local_step': 3, 'learning_rate': 0.5, 'n_trainer': 2, 'batch_size': -1, 'num_layers': 2, 'num_hops': 1, 'gpu': False, 'num_cpus_per_trainer': 1, 'num_gpus_per_trainer': 0, 'logdir': './runs', 'use_encryption': True, 'use_huggingface': False, 'saveto_huggingface': False, 'use_cluster': False}
File already exists: ./data/cora/raw/ind.cora.x
File already exists: ./data/cora/raw/ind.cora.tx
File already exists: ./data/cora/raw/ind.cora.allx
File already exists: ./data/cora/raw/ind.cora.y
File already exists: ./data/cora/raw/ind.cora.ty
File already exists: ./data/cora/raw/ind.cora.ally
File already exists: ./data/cora/raw/ind.cora.graph
File already exists: ./data/cora/raw/ind.cora.test.index
Initialization start time recorded.
(pid=2637) FedGraph Differential Privacy module loaded (v1.0.0)
(pid=2637) ✓ Differential Privacy support loaded
(Trainer pid=2637) Trainer 0 loaded HE context
Loaded HE context with secret key.
//Log init_time: 7962.413 ms //end
Pretrain start time recorded.
Starting encrypted feature aggregation...
(Trainer pid=2637) WARNING: The input does not fit in a single ciphertext, and some operations will be disabled.
(Trainer pid=2637) The following operations are disabled in this setup: matmul, matmul_plain, enc_matmul_plain, conv2d_im2col.
(Trainer pid=2637) If you need to use those operations, try increasing the poly_modulus parameter, to fit your input.
Pre-training Phase Metrics:
Total Pre-training Time: 19.28 seconds
Pre-training Upload: 604.41 MB
Pre-training Download: 604.43 MB
Total Pre-training Communication Cost: 1208.84 MB
//pretrain_time: 19276.741 ms//end
Train start time recorded.
global_rounds 100
(pid=2636) FedGraph Differential Privacy module loaded (v1.0.0)
(pid=2636) ✓ Differential Privacy support loaded
(Trainer pid=2636) Trainer 1 loaded HE context
(Trainer pid=2637) WARNING: The input does not fit in a single ciphertext, and some operations will be disabled. [repeated 2x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(Trainer pid=2637) The following operations are disabled in this setup: matmul, matmul_plain, enc_matmul_plain, conv2d_im2col. [repeated 2x across cluster]
(Trainer pid=2637) If you need to use those operations, try increasing the poly_modulus parameter, to fit your input. [repeated 2x across cluster]
Round 1: Global Test Accuracy = 0.3540
Round 1: Training Time = 0.16s, Communication Time = 0.19s
Round 2: Global Test Accuracy = 0.5970
Round 2: Training Time = 0.04s, Communication Time = 0.12s
Round 3: Global Test Accuracy = 0.7600
Round 3: Training Time = 0.04s, Communication Time = 0.12s
Round 4: Global Test Accuracy = 0.7760
Round 4: Training Time = 0.04s, Communication Time = 0.10s
Round 5: Global Test Accuracy = 0.8000
Round 5: Training Time = 0.04s, Communication Time = 0.10s
Round 6: Global Test Accuracy = 0.7970
Round 6: Training Time = 0.04s, Communication Time = 0.12s
Round 7: Global Test Accuracy = 0.8030
Round 7: Training Time = 0.04s, Communication Time = 0.12s
Round 8: Global Test Accuracy = 0.7990
Round 8: Training Time = 0.04s, Communication Time = 0.12s
Round 9: Global Test Accuracy = 0.7880
Round 9: Training Time = 0.04s, Communication Time = 0.13s
Round 10: Global Test Accuracy = 0.7940
Round 10: Training Time = 0.04s, Communication Time = 0.10s
Round 11: Global Test Accuracy = 0.7890
Round 11: Training Time = 0.04s, Communication Time = 0.13s
Round 12: Global Test Accuracy = 0.7940
Round 12: Training Time = 0.04s, Communication Time = 0.10s
Round 13: Global Test Accuracy = 0.7890
Round 13: Training Time = 0.05s, Communication Time = 0.13s
Round 14: Global Test Accuracy = 0.7940
Round 14: Training Time = 0.05s, Communication Time = 0.11s
Round 15: Global Test Accuracy = 0.7870
Round 15: Training Time = 0.04s, Communication Time = 0.10s
Round 16: Global Test Accuracy = 0.7910
Round 16: Training Time = 0.04s, Communication Time = 0.10s
Round 17: Global Test Accuracy = 0.7960
Round 17: Training Time = 0.04s, Communication Time = 0.11s
Round 18: Global Test Accuracy = 0.7870
Round 18: Training Time = 0.04s, Communication Time = 0.10s
Round 19: Global Test Accuracy = 0.7940
Round 19: Training Time = 0.04s, Communication Time = 0.10s
Round 20: Global Test Accuracy = 0.7950
Round 20: Training Time = 0.04s, Communication Time = 0.10s
Round 21: Global Test Accuracy = 0.7930
Round 21: Training Time = 0.04s, Communication Time = 0.11s
Round 22: Global Test Accuracy = 0.7760
Round 22: Training Time = 0.04s, Communication Time = 0.12s
Round 23: Global Test Accuracy = 0.7890
Round 23: Training Time = 0.04s, Communication Time = 0.10s
Round 24: Global Test Accuracy = 0.7890
Round 24: Training Time = 0.04s, Communication Time = 0.10s
Round 25: Global Test Accuracy = 0.7910
Round 25: Training Time = 0.04s, Communication Time = 0.10s
Round 26: Global Test Accuracy = 0.7940
Round 26: Training Time = 0.04s, Communication Time = 0.10s
Round 27: Global Test Accuracy = 0.7970
Round 27: Training Time = 0.04s, Communication Time = 0.10s
Round 28: Global Test Accuracy = 0.7920
Round 28: Training Time = 0.04s, Communication Time = 0.10s
Round 29: Global Test Accuracy = 0.7840
Round 29: Training Time = 0.04s, Communication Time = 0.10s
Round 30: Global Test Accuracy = 0.7960
Round 30: Training Time = 0.04s, Communication Time = 0.10s
Round 31: Global Test Accuracy = 0.7910
Round 31: Training Time = 0.05s, Communication Time = 0.10s
Round 32: Global Test Accuracy = 0.7940
Round 32: Training Time = 0.04s, Communication Time = 0.10s
(Trainer pid=2637) WARNING: The input does not fit in a single ciphertext, and some operations will be disabled. [repeated 64x across cluster]
(Trainer pid=2637) The following operations are disabled in this setup: matmul, matmul_plain, enc_matmul_plain, conv2d_im2col. [repeated 64x across cluster]
(Trainer pid=2637) If you need to use those operations, try increasing the poly_modulus parameter, to fit your input. [repeated 64x across cluster]
Round 33: Global Test Accuracy = 0.7940
Round 33: Training Time = 0.04s, Communication Time = 0.10s
Round 34: Global Test Accuracy = 0.7890
Round 34: Training Time = 0.05s, Communication Time = 0.10s
Round 35: Global Test Accuracy = 0.7910
Round 35: Training Time = 0.04s, Communication Time = 0.11s
Round 36: Global Test Accuracy = 0.7930
Round 36: Training Time = 0.04s, Communication Time = 0.10s
Round 37: Global Test Accuracy = 0.7860
Round 37: Training Time = 0.04s, Communication Time = 0.10s
Round 38: Global Test Accuracy = 0.7910
Round 38: Training Time = 0.04s, Communication Time = 0.10s
Round 39: Global Test Accuracy = 0.7920
Round 39: Training Time = 0.04s, Communication Time = 0.10s
Round 40: Global Test Accuracy = 0.7920
Round 40: Training Time = 0.04s, Communication Time = 0.13s
Round 41: Global Test Accuracy = 0.7890
Round 41: Training Time = 0.04s, Communication Time = 0.10s
Round 42: Global Test Accuracy = 0.7940
Round 42: Training Time = 0.04s, Communication Time = 0.10s
Round 43: Global Test Accuracy = 0.7900
Round 43: Training Time = 0.30s, Communication Time = 0.10s
Round 44: Global Test Accuracy = 0.7900
Round 44: Training Time = 0.04s, Communication Time = 0.10s
Round 45: Global Test Accuracy = 0.7940
Round 45: Training Time = 0.04s, Communication Time = 0.11s
Round 46: Global Test Accuracy = 0.7930
Round 46: Training Time = 0.04s, Communication Time = 0.10s
Round 47: Global Test Accuracy = 0.7890
Round 47: Training Time = 0.04s, Communication Time = 0.10s
Round 48: Global Test Accuracy = 0.7880
Round 48: Training Time = 0.04s, Communication Time = 0.10s
Round 49: Global Test Accuracy = 0.7940
Round 49: Training Time = 0.04s, Communication Time = 0.12s
Round 50: Global Test Accuracy = 0.7890
Round 50: Training Time = 0.04s, Communication Time = 0.10s
Round 51: Global Test Accuracy = 0.7930
Round 51: Training Time = 0.04s, Communication Time = 0.10s
Round 52: Global Test Accuracy = 0.7880
Round 52: Training Time = 0.04s, Communication Time = 0.10s
Round 53: Global Test Accuracy = 0.7900
Round 53: Training Time = 0.04s, Communication Time = 0.10s
Round 54: Global Test Accuracy = 0.7910
Round 54: Training Time = 0.04s, Communication Time = 0.10s
Round 55: Global Test Accuracy = 0.7930
Round 55: Training Time = 0.04s, Communication Time = 0.10s
Round 56: Global Test Accuracy = 0.7880
Round 56: Training Time = 0.04s, Communication Time = 0.11s
Round 57: Global Test Accuracy = 0.7880
Round 57: Training Time = 0.04s, Communication Time = 0.10s
Round 58: Global Test Accuracy = 0.7890
Round 58: Training Time = 0.04s, Communication Time = 0.10s
Round 59: Global Test Accuracy = 0.7890
Round 59: Training Time = 0.04s, Communication Time = 0.10s
Round 60: Global Test Accuracy = 0.7900
Round 60: Training Time = 0.04s, Communication Time = 0.10s
Round 61: Global Test Accuracy = 0.7930
Round 61: Training Time = 0.04s, Communication Time = 0.10s
Round 62: Global Test Accuracy = 0.7890
Round 62: Training Time = 0.04s, Communication Time = 0.10s
Round 63: Global Test Accuracy = 0.7870
Round 63: Training Time = 0.04s, Communication Time = 0.10s
Round 64: Global Test Accuracy = 0.7860
Round 64: Training Time = 0.04s, Communication Time = 0.10s
(Trainer pid=2637) WARNING: The input does not fit in a single ciphertext, and some operations will be disabled. [repeated 64x across cluster]
(Trainer pid=2637) The following operations are disabled in this setup: matmul, matmul_plain, enc_matmul_plain, conv2d_im2col. [repeated 64x across cluster]
(Trainer pid=2637) If you need to use those operations, try increasing the poly_modulus parameter, to fit your input. [repeated 64x across cluster]
Round 65: Global Test Accuracy = 0.7880
Round 65: Training Time = 0.04s, Communication Time = 0.10s
Round 66: Global Test Accuracy = 0.7910
Round 66: Training Time = 0.05s, Communication Time = 0.10s
Round 67: Global Test Accuracy = 0.7910
Round 67: Training Time = 0.04s, Communication Time = 0.10s
Round 68: Global Test Accuracy = 0.7860
Round 68: Training Time = 0.04s, Communication Time = 0.10s
Round 69: Global Test Accuracy = 0.7930
Round 69: Training Time = 0.04s, Communication Time = 0.10s
Round 70: Global Test Accuracy = 0.7900
Round 70: Training Time = 0.04s, Communication Time = 0.10s
Round 71: Global Test Accuracy = 0.7900
Round 71: Training Time = 0.04s, Communication Time = 0.10s
Round 72: Global Test Accuracy = 0.7900
Round 72: Training Time = 0.04s, Communication Time = 0.11s
Round 73: Global Test Accuracy = 0.7920
Round 73: Training Time = 0.04s, Communication Time = 0.10s
Round 74: Global Test Accuracy = 0.7900
Round 74: Training Time = 0.04s, Communication Time = 0.10s
Round 75: Global Test Accuracy = 0.7890
Round 75: Training Time = 0.04s, Communication Time = 0.10s
Round 76: Global Test Accuracy = 0.7860
Round 76: Training Time = 0.04s, Communication Time = 0.10s
Round 77: Global Test Accuracy = 0.7900
Round 77: Training Time = 0.04s, Communication Time = 0.10s
Round 78: Global Test Accuracy = 0.7900
Round 78: Training Time = 0.04s, Communication Time = 0.10s
Round 79: Global Test Accuracy = 0.7900
Round 79: Training Time = 0.04s, Communication Time = 0.10s
Round 80: Global Test Accuracy = 0.7910
Round 80: Training Time = 0.04s, Communication Time = 0.10s
Round 81: Global Test Accuracy = 0.7880
Round 81: Training Time = 0.04s, Communication Time = 0.10s
Round 82: Global Test Accuracy = 0.7880
Round 82: Training Time = 0.04s, Communication Time = 0.10s
Round 83: Global Test Accuracy = 0.7880
Round 83: Training Time = 0.04s, Communication Time = 0.10s
Round 84: Global Test Accuracy = 0.7880
Round 84: Training Time = 0.04s, Communication Time = 0.10s
Round 85: Global Test Accuracy = 0.7870
Round 85: Training Time = 0.04s, Communication Time = 0.10s
Round 86: Global Test Accuracy = 0.7870
Round 86: Training Time = 0.04s, Communication Time = 0.10s
Round 87: Global Test Accuracy = 0.7880
Round 87: Training Time = 0.04s, Communication Time = 0.10s
Round 88: Global Test Accuracy = 0.7900
Round 88: Training Time = 0.04s, Communication Time = 0.10s
Round 89: Global Test Accuracy = 0.7910
Round 89: Training Time = 0.04s, Communication Time = 0.10s
Round 90: Global Test Accuracy = 0.7900
Round 90: Training Time = 0.04s, Communication Time = 0.10s
Round 91: Global Test Accuracy = 0.7890
Round 91: Training Time = 0.06s, Communication Time = 0.10s
Round 92: Global Test Accuracy = 0.7890
Round 92: Training Time = 0.04s, Communication Time = 0.10s
Round 93: Global Test Accuracy = 0.7920
Round 93: Training Time = 0.04s, Communication Time = 0.10s
Round 94: Global Test Accuracy = 0.7910
Round 94: Training Time = 0.04s, Communication Time = 0.10s
Round 95: Global Test Accuracy = 0.7930
Round 95: Training Time = 0.04s, Communication Time = 0.10s
Round 96: Global Test Accuracy = 0.7910
Round 96: Training Time = 0.04s, Communication Time = 0.10s
Round 97: Global Test Accuracy = 0.7900
Round 97: Training Time = 0.04s, Communication Time = 0.10s
Round 98: Global Test Accuracy = 0.7930
Round 98: Training Time = 0.04s, Communication Time = 0.10s
(Trainer pid=2637) WARNING: The input does not fit in a single ciphertext, and some operations will be disabled. [repeated 68x across cluster]
(Trainer pid=2637) The following operations are disabled in this setup: matmul, matmul_plain, enc_matmul_plain, conv2d_im2col. [repeated 68x across cluster]
(Trainer pid=2637) If you need to use those operations, try increasing the poly_modulus parameter, to fit your input. [repeated 68x across cluster]
Round 99: Global Test Accuracy = 0.7910
Round 99: Training Time = 0.04s, Communication Time = 0.10s
Round 100: Global Test Accuracy = 0.7910
Round 100: Training Time = 0.04s, Communication Time = 0.10s
//train_time: 15487.617999999999 ms//end
================================================================================
TIME BREAKDOWN (excluding initialization)
================================================================================
Total Pure Training Time (forward + gradient descent): 4.46 seconds
Total Communication Time (parameter aggregation): 10.32 seconds
Total Training + Communication Time: 15.49 seconds
Training Time Percentage: 28.8%
Communication Time Percentage: 66.6%
Average Training Time per Round: 0.04 seconds
Average Communication Time per Round: 0.10 seconds
================================================================================
[Pure Training Time] Dataset: cora, Batch Size: -1, Trainers: 2, Hops: 1, IID Beta: 10000 => Pure Training Time = 4.46 seconds
[Communication Time] Dataset: cora, Batch Size: -1, Trainers: 2, Hops: 1, IID Beta: 10000 => Communication Time = 10.32 seconds
Training Phase Metrics:
Total Training Time: 4.46 seconds
Training Upload: 0.00 MB
Training Download: 0.00 MB
Total Training Communication Cost: 0.00 MB
Overall Totals:
Total Execution Time: 38.18 seconds
Total Upload: 604.41 MB
Total Download: 604.43 MB
Total Communication Cost: 1208.84 MB
Pre-training Time %: 50.5%
Training Time %: 11.7%
Communication Time %: 27.0%
average_final_test_loss, 0.9379583668708801
Average test accuracy, 0.791
================================================================================
INDIVIDUAL TRAINER MEMORY USAGE
================================================================================
====================================================================================================
TRAINER MEMORY vs LOCAL GRAPH SIZE
====================================================================================================
Trainer Memory(MB) Nodes Edges Memory/Node Memory/Edge
----------------------------------------------------------------------------------------------------
0 1325.4 1355 9708 0.978 0.137
1 1450.6 1353 9604 1.072 0.151
====================================================================================================
Total Memory Usage: 2776.0 MB (2.71 GB)
Total Nodes: 2708, Total Edges: 19312
Average Memory per Trainer: 1388.0 MB
Average Nodes per Trainer: 1354.0
Average Edges per Trainer: 9656.0
Max Memory: 1450.6 MB (Trainer 1)
Min Memory: 1325.4 MB (Trainer 0)
Overall Memory/Node Ratio: 1.025 MB/node
Overall Memory/Edge Ratio: 0.144 MB/edge
====================================================================================================
//Log Theoretical Pretrain Comm Cost: 1208.84 MB //end
//Log Theoretical Train Comm Cost: 35.19 MB //end
================================================================================
CSV FORMAT RESULT:
DS,IID,BS,TotalTime[s],PureTrainingTime[s],CommTime[s],FinalAcc[%],CommCost[MB],PeakMem[MB],AvgRoundTime[s],ModelSize[MB],TotalParams
cora,10000,-1,38.2,4.5,10.3,0.79,1244.0,1450.6,0.045,0.088,0
================================================================================
================================================================================
EXPERIMENT SUMMARY
================================================================================
Dataset: cora
Method: FedGCN
Trainers: 2
IID Beta: 10000
Batch Size: -1
Hops: 1
Total Execution Time: 38.19 seconds
Pure Training Time: 4.46 seconds
Communication Time: 10.32 seconds
Pretrain Comm Cost: 1208.84 MB
Training Comm Cost: 35.19 MB
Total Comm Cost: 1208.84 MB
================================================================================
(Trainer pid=2636) WARNING: The input does not fit in a single ciphertext, and some operations will be disabled.
(Trainer pid=2636) The following operations are disabled in this setup: matmul, matmul_plain, enc_matmul_plain, conv2d_im2col.
(Trainer pid=2636) If you need to use those operations, try increasing the poly_modulus parameter, to fit your input.
Total running time of the script: (0 minutes 44.721 seconds)