Note
Go to the end to download the full example code.
Federated Node Classification Example
Federated Node Classification with FedGCN on the Cora dataset.
(Time estimate: 3 minutes)
Load libraries
import attridict
from fedgraph.federated_methods import run_fedgraph
Specify the Node Classification configuration
config = {
# Task, Method, and Dataset Settings
"fedgraph_task": "NC",
"dataset": "cora",
"method": "FedGCN", # Federated learning method, e.g., "FedGCN"
"iid_beta": 10000, # Dirichlet distribution parameter for label distribution among clients
"distribution_type": "average", # Distribution type among clients
# Training Configuration
"global_rounds": 100,
"local_step": 3,
"learning_rate": 0.5,
"n_trainer": 2,
"batch_size": -1, # -1 indicates full batch training
# Model Structure
"num_layers": 2,
"num_hops": 1, # Number of n-hop neighbors for client communication
# Resource and Hardware Settings
"gpu": False,
"num_cpus_per_trainer": 1,
"num_gpus_per_trainer": 0,
# Logging and Output Configuration
"logdir": "./runs",
# Security and Privacy
"use_encryption": False, # Whether to use Homomorphic Encryption for secure aggregation
# Dataset Handling Options
"use_huggingface": False, # Load dataset directly from Hugging Face Hub
"saveto_huggingface": False, # Save partitioned dataset to Hugging Face Hub
# Scalability and Cluster Configuration
"use_cluster": False, # Use Kubernetes for scalability if True
}
Run fedgraph method
config = attridict(config)
run_fedgraph(config)
config: {'fedgraph_task': 'NC', 'dataset': 'cora', 'method': 'FedGCN', 'iid_beta': 10000, 'distribution_type': 'average', 'global_rounds': 100, 'local_step': 3, 'learning_rate': 0.5, 'n_trainer': 2, 'batch_size': -1, 'num_layers': 2, 'num_hops': 1, 'gpu': False, 'num_cpus_per_trainer': 1, 'num_gpus_per_trainer': 0, 'logdir': './runs', 'use_encryption': False, 'use_huggingface': False, 'saveto_huggingface': False, 'use_cluster': False}
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.x to ./data/cora/raw/ind.cora.x...
Downloaded ./data/cora/raw/ind.cora.x
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.tx to ./data/cora/raw/ind.cora.tx...
Downloaded ./data/cora/raw/ind.cora.tx
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.allx to ./data/cora/raw/ind.cora.allx...
Downloaded ./data/cora/raw/ind.cora.allx
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.y to ./data/cora/raw/ind.cora.y...
Downloaded ./data/cora/raw/ind.cora.y
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.ty to ./data/cora/raw/ind.cora.ty...
Downloaded ./data/cora/raw/ind.cora.ty
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.ally to ./data/cora/raw/ind.cora.ally...
Downloaded ./data/cora/raw/ind.cora.ally
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.graph to ./data/cora/raw/ind.cora.graph...
Downloaded ./data/cora/raw/ind.cora.graph
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.test.index to ./data/cora/raw/ind.cora.test.index...
Downloaded ./data/cora/raw/ind.cora.test.index
Initialization start time recorded.
(pid=2092) FedGraph Differential Privacy module loaded (v1.0.0)
(pid=2092) ✓ Differential Privacy support loaded
//Log init_time: 11902.579 ms //end
Pretrain start time recorded.
server aggregates all local neighbor feature sums
clients received feature aggregation from server
//pretrain_time: 143.47099999999998 ms//end
Train start time recorded.
global_rounds 100
Round 1: Global Test Accuracy = 0.4740
Round 1: Training Time = 0.08s, Communication Time = 0.01s
Round 2: Global Test Accuracy = 0.5000
Round 2: Training Time = 0.04s, Communication Time = 0.00s
Round 3: Global Test Accuracy = 0.7710
Round 3: Training Time = 0.04s, Communication Time = 0.00s
Round 4: Global Test Accuracy = 0.7870
Round 4: Training Time = 0.04s, Communication Time = 0.00s
Round 5: Global Test Accuracy = 0.7920
Round 5: Training Time = 0.04s, Communication Time = 0.00s
Round 6: Global Test Accuracy = 0.7970
Round 6: Training Time = 0.04s, Communication Time = 0.00s
Round 7: Global Test Accuracy = 0.8080
Round 7: Training Time = 0.04s, Communication Time = 0.00s
Round 8: Global Test Accuracy = 0.7790
Round 8: Training Time = 0.04s, Communication Time = 0.01s
Round 9: Global Test Accuracy = 0.8010
Round 9: Training Time = 0.05s, Communication Time = 0.01s
Round 10: Global Test Accuracy = 0.8060
Round 10: Training Time = 0.04s, Communication Time = 0.00s
Round 11: Global Test Accuracy = 0.8010
Round 11: Training Time = 0.04s, Communication Time = 0.00s
Round 12: Global Test Accuracy = 0.7950
Round 12: Training Time = 0.04s, Communication Time = 0.00s
Round 13: Global Test Accuracy = 0.8040
Round 13: Training Time = 0.04s, Communication Time = 0.00s
Round 14: Global Test Accuracy = 0.8060
Round 14: Training Time = 0.04s, Communication Time = 0.00s
Round 15: Global Test Accuracy = 0.7890
Round 15: Training Time = 0.04s, Communication Time = 0.00s
Round 16: Global Test Accuracy = 0.7850
Round 16: Training Time = 0.04s, Communication Time = 0.00s
Round 17: Global Test Accuracy = 0.8080
Round 17: Training Time = 0.04s, Communication Time = 0.00s
Round 18: Global Test Accuracy = 0.7980
Round 18: Training Time = 0.04s, Communication Time = 0.00s
Round 19: Global Test Accuracy = 0.8000
Round 19: Training Time = 0.04s, Communication Time = 0.00s
Round 20: Global Test Accuracy = 0.7920
Round 20: Training Time = 0.04s, Communication Time = 0.01s
Round 21: Global Test Accuracy = 0.7880
Round 21: Training Time = 0.04s, Communication Time = 0.00s
Round 22: Global Test Accuracy = 0.7930
Round 22: Training Time = 0.04s, Communication Time = 0.00s
Round 23: Global Test Accuracy = 0.7920
Round 23: Training Time = 0.04s, Communication Time = 0.00s
Round 24: Global Test Accuracy = 0.7940
Round 24: Training Time = 0.04s, Communication Time = 0.00s
Round 25: Global Test Accuracy = 0.7900
Round 25: Training Time = 0.04s, Communication Time = 0.00s
Round 26: Global Test Accuracy = 0.7880
Round 26: Training Time = 0.04s, Communication Time = 0.00s
Round 27: Global Test Accuracy = 0.7900
Round 27: Training Time = 0.04s, Communication Time = 0.00s
Round 28: Global Test Accuracy = 0.7860
Round 28: Training Time = 0.04s, Communication Time = 0.00s
Round 29: Global Test Accuracy = 0.7890
Round 29: Training Time = 0.04s, Communication Time = 0.00s
Round 30: Global Test Accuracy = 0.7970
Round 30: Training Time = 0.04s, Communication Time = 0.00s
Round 31: Global Test Accuracy = 0.7860
Round 31: Training Time = 0.04s, Communication Time = 0.00s
Round 32: Global Test Accuracy = 0.7930
Round 32: Training Time = 0.04s, Communication Time = 0.00s
Round 33: Global Test Accuracy = 0.7880
Round 33: Training Time = 0.04s, Communication Time = 0.01s
Round 34: Global Test Accuracy = 0.7870
Round 34: Training Time = 0.04s, Communication Time = 0.01s
Round 35: Global Test Accuracy = 0.7850
Round 35: Training Time = 0.04s, Communication Time = 0.00s
Round 36: Global Test Accuracy = 0.7880
Round 36: Training Time = 0.04s, Communication Time = 0.00s
Round 37: Global Test Accuracy = 0.7930
Round 37: Training Time = 0.04s, Communication Time = 0.01s
Round 38: Global Test Accuracy = 0.7890
Round 38: Training Time = 0.04s, Communication Time = 0.00s
Round 39: Global Test Accuracy = 0.7900
Round 39: Training Time = 0.04s, Communication Time = 0.00s
Round 40: Global Test Accuracy = 0.7880
Round 40: Training Time = 0.04s, Communication Time = 0.00s
Round 41: Global Test Accuracy = 0.7900
Round 41: Training Time = 0.04s, Communication Time = 0.01s
Round 42: Global Test Accuracy = 0.7890
Round 42: Training Time = 0.06s, Communication Time = 0.00s
Round 43: Global Test Accuracy = 0.7870
Round 43: Training Time = 0.04s, Communication Time = 0.00s
Round 44: Global Test Accuracy = 0.7910
Round 44: Training Time = 0.07s, Communication Time = 0.00s
Round 45: Global Test Accuracy = 0.7870
Round 45: Training Time = 0.04s, Communication Time = 0.00s
Round 46: Global Test Accuracy = 0.7900
Round 46: Training Time = 0.04s, Communication Time = 0.00s
Round 47: Global Test Accuracy = 0.7970
Round 47: Training Time = 0.04s, Communication Time = 0.01s
Round 48: Global Test Accuracy = 0.7900
Round 48: Training Time = 0.04s, Communication Time = 0.00s
Round 49: Global Test Accuracy = 0.7900
Round 49: Training Time = 0.04s, Communication Time = 0.00s
Round 50: Global Test Accuracy = 0.7920
Round 50: Training Time = 0.04s, Communication Time = 0.00s
Round 51: Global Test Accuracy = 0.7910
Round 51: Training Time = 0.04s, Communication Time = 0.00s
Round 52: Global Test Accuracy = 0.7920
Round 52: Training Time = 0.04s, Communication Time = 0.00s
Round 53: Global Test Accuracy = 0.7920
Round 53: Training Time = 0.04s, Communication Time = 0.00s
Round 54: Global Test Accuracy = 0.7930
Round 54: Training Time = 0.04s, Communication Time = 0.00s
Round 55: Global Test Accuracy = 0.7920
Round 55: Training Time = 0.04s, Communication Time = 0.00s
Round 56: Global Test Accuracy = 0.7940
Round 56: Training Time = 0.04s, Communication Time = 0.00s
Round 57: Global Test Accuracy = 0.7950
Round 57: Training Time = 0.04s, Communication Time = 0.00s
Round 58: Global Test Accuracy = 0.7950
Round 58: Training Time = 0.04s, Communication Time = 0.01s
Round 59: Global Test Accuracy = 0.7950
Round 59: Training Time = 0.04s, Communication Time = 0.00s
Round 60: Global Test Accuracy = 0.7890
Round 60: Training Time = 0.04s, Communication Time = 0.00s
Round 61: Global Test Accuracy = 0.7910
Round 61: Training Time = 0.04s, Communication Time = 0.00s
Round 62: Global Test Accuracy = 0.7910
Round 62: Training Time = 0.04s, Communication Time = 0.00s
Round 63: Global Test Accuracy = 0.7900
Round 63: Training Time = 0.04s, Communication Time = 0.00s
Round 64: Global Test Accuracy = 0.7900
Round 64: Training Time = 0.04s, Communication Time = 0.00s
Round 65: Global Test Accuracy = 0.7890
Round 65: Training Time = 0.04s, Communication Time = 0.00s
Round 66: Global Test Accuracy = 0.7890
Round 66: Training Time = 0.04s, Communication Time = 0.00s
Round 67: Global Test Accuracy = 0.7880
Round 67: Training Time = 0.04s, Communication Time = 0.00s
Round 68: Global Test Accuracy = 0.7930
Round 68: Training Time = 0.04s, Communication Time = 0.00s
Round 69: Global Test Accuracy = 0.7890
Round 69: Training Time = 0.04s, Communication Time = 0.00s
Round 70: Global Test Accuracy = 0.7940
Round 70: Training Time = 0.04s, Communication Time = 0.00s
Round 71: Global Test Accuracy = 0.7940
Round 71: Training Time = 0.04s, Communication Time = 0.00s
Round 72: Global Test Accuracy = 0.7950
Round 72: Training Time = 0.04s, Communication Time = 0.00s
Round 73: Global Test Accuracy = 0.7910
Round 73: Training Time = 0.04s, Communication Time = 0.00s
Round 74: Global Test Accuracy = 0.7890
Round 74: Training Time = 0.04s, Communication Time = 0.00s
Round 75: Global Test Accuracy = 0.7930
Round 75: Training Time = 0.04s, Communication Time = 0.01s
Round 76: Global Test Accuracy = 0.7960
Round 76: Training Time = 0.04s, Communication Time = 0.00s
Round 77: Global Test Accuracy = 0.7890
Round 77: Training Time = 0.04s, Communication Time = 0.00s
Round 78: Global Test Accuracy = 0.7940
Round 78: Training Time = 0.04s, Communication Time = 0.01s
Round 79: Global Test Accuracy = 0.7960
Round 79: Training Time = 0.05s, Communication Time = 0.00s
Round 80: Global Test Accuracy = 0.7950
Round 80: Training Time = 0.04s, Communication Time = 0.00s
Round 81: Global Test Accuracy = 0.7940
Round 81: Training Time = 0.04s, Communication Time = 0.00s
Round 82: Global Test Accuracy = 0.7870
Round 82: Training Time = 0.04s, Communication Time = 0.00s
Round 83: Global Test Accuracy = 0.7890
Round 83: Training Time = 0.04s, Communication Time = 0.00s
Round 84: Global Test Accuracy = 0.7850
Round 84: Training Time = 0.04s, Communication Time = 0.00s
Round 85: Global Test Accuracy = 0.7860
Round 85: Training Time = 0.04s, Communication Time = 0.00s
Round 86: Global Test Accuracy = 0.7890
Round 86: Training Time = 0.04s, Communication Time = 0.00s
Round 87: Global Test Accuracy = 0.7920
Round 87: Training Time = 0.04s, Communication Time = 0.00s
Round 88: Global Test Accuracy = 0.7910
Round 88: Training Time = 0.04s, Communication Time = 0.00s
Round 89: Global Test Accuracy = 0.7900
Round 89: Training Time = 0.04s, Communication Time = 0.00s
Round 90: Global Test Accuracy = 0.7870
Round 90: Training Time = 0.04s, Communication Time = 0.00s
Round 91: Global Test Accuracy = 0.7890
Round 91: Training Time = 0.04s, Communication Time = 0.00s
Round 92: Global Test Accuracy = 0.7910
Round 92: Training Time = 0.04s, Communication Time = 0.00s
Round 93: Global Test Accuracy = 0.7920
Round 93: Training Time = 0.04s, Communication Time = 0.00s
Round 94: Global Test Accuracy = 0.7910
Round 94: Training Time = 0.04s, Communication Time = 0.00s
Round 95: Global Test Accuracy = 0.7910
Round 95: Training Time = 0.04s, Communication Time = 0.00s
Round 96: Global Test Accuracy = 0.7920
Round 96: Training Time = 0.04s, Communication Time = 0.00s
Round 97: Global Test Accuracy = 0.7900
Round 97: Training Time = 0.04s, Communication Time = 0.00s
Round 98: Global Test Accuracy = 0.7880
Round 98: Training Time = 0.05s, Communication Time = 0.01s
Round 99: Global Test Accuracy = 0.7900
Round 99: Training Time = 0.04s, Communication Time = 0.00s
Round 100: Global Test Accuracy = 0.7870
Round 100: Training Time = 0.04s, Communication Time = 0.00s
//train_time: 5273.04 ms//end
================================================================================
TIME BREAKDOWN (excluding initialization)
================================================================================
Total Pure Training Time (forward + gradient descent): 4.08 seconds
Total Communication Time (parameter aggregation): 0.45 seconds
Total Training + Communication Time: 5.27 seconds
Training Time Percentage: 77.4%
Communication Time Percentage: 8.5%
Average Training Time per Round: 0.04 seconds
Average Communication Time per Round: 0.00 seconds
================================================================================
[Pure Training Time] Dataset: cora, Batch Size: -1, Trainers: 2, Hops: 1, IID Beta: 10000 => Pure Training Time = 4.08 seconds
[Communication Time] Dataset: cora, Batch Size: -1, Trainers: 2, Hops: 1, IID Beta: 10000 => Communication Time = 0.45 seconds
average_final_test_loss, 0.9797245250940323
Average test accuracy, 0.787
================================================================================
INDIVIDUAL TRAINER MEMORY USAGE
================================================================================
====================================================================================================
TRAINER MEMORY vs LOCAL GRAPH SIZE
====================================================================================================
Trainer Memory(MB) Nodes Edges Memory/Node Memory/Edge
----------------------------------------------------------------------------------------------------
0 508.7 1347 9704 0.378 0.052
1 509.0 1361 9844 0.374 0.052
====================================================================================================
Total Memory Usage: 1017.7 MB (0.99 GB)
Total Nodes: 2708, Total Edges: 19548
Average Memory per Trainer: 508.8 MB
Average Nodes per Trainer: 1354.0
Average Edges per Trainer: 9774.0
Max Memory: 509.0 MB (Trainer 1)
Min Memory: 508.7 MB (Trainer 0)
Overall Memory/Node Ratio: 0.376 MB/node
Overall Memory/Edge Ratio: 0.052 MB/edge
====================================================================================================
//Log Theoretical Pretrain Comm Cost: 56.55 MB //end
//Log Theoretical Train Comm Cost: 35.19 MB //end
================================================================================
CSV FORMAT RESULT:
DS,IID,BS,TotalTime[s],PureTrainingTime[s],CommTime[s],FinalAcc[%],CommCost[MB],PeakMem[MB],AvgRoundTime[s],ModelSize[MB],TotalParams
cora,10000,-1,12.2,4.1,0.4,0.79,91.7,509.0,0.041,0.088,0
================================================================================
================================================================================
EXPERIMENT SUMMARY
================================================================================
Dataset: cora
Method: FedGCN
Trainers: 2
IID Beta: 10000
Batch Size: -1
Hops: 1
Total Execution Time: 12.15 seconds
Pure Training Time: 4.08 seconds
Communication Time: 0.45 seconds
Pretrain Comm Cost: 56.55 MB
Training Comm Cost: 35.19 MB
================================================================================
(pid=2093) FedGraph Differential Privacy module loaded (v1.0.0)
(pid=2093) ✓ Differential Privacy support loaded
Total running time of the script: (0 minutes 20.865 seconds)