Skip to content

benchmark_pareto_hard_summary.md

Latest commit

 

History

History
162 lines (145 loc) · 35.9 KB

benchmark_pareto_hard_summary.md

File metadata and controls

162 lines (145 loc) · 35.9 KB
Raw

Pareto Hard Benchmark Summary

Dataset: {'difficulty': 'hard', 'noise_std': 0.08, 'train_samples': 2500, 'test_samples': 700, 'classes': 10, 'image_size': 96, 'device': 'cuda', 'epochs': 14}

BackpropResNet50

Pareto front size: 5

Top-10 Balanced Score

  1. acc=0.2157, train_sps=2939.6, infer_sps=4504.3, balanced=0.9471, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.9}
  2. acc=0.1900, train_sps=2943.5, infer_sps=4505.1, balanced=0.8363, params={'backprop_learning_rate': 0.05, 'backprop_momentum': 0.9}
  3. acc=0.1900, train_sps=2929.7, infer_sps=4543.3, balanced=0.8276, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.95}
  4. acc=0.1700, train_sps=2926.1, infer_sps=4534.3, balanced=0.7153, params={'backprop_learning_rate': 0.005, 'backprop_momentum': 0.9}
  5. acc=0.2057, train_sps=2932.7, infer_sps=4286.4, balanced=0.7105, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.95}
  6. acc=0.1043, train_sps=2947.7, infer_sps=4530.9, balanced=0.4608, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.9}
  7. acc=0.1029, train_sps=2940.8, infer_sps=4539.1, balanced=0.4410, params={'backprop_learning_rate': 0.03, 'backprop_momentum': 0.9}
  8. acc=0.1029, train_sps=2943.7, infer_sps=4515.4, balanced=0.4306, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.85}
  9. acc=0.1043, train_sps=2899.4, infer_sps=4490.2, balanced=0.2941, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.85}
  10. acc=0.1000, train_sps=2858.2, infer_sps=4439.5, balanced=0.1192, params={'backprop_learning_rate': 0.003, 'backprop_momentum': 0.9}

Top-10 Accuracy

  1. acc=0.2157, train_sps=2939.6, infer_sps=4504.3, balanced=0.9471, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.9}
  2. acc=0.2057, train_sps=2932.7, infer_sps=4286.4, balanced=0.7105, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.95}
  3. acc=0.1900, train_sps=2943.5, infer_sps=4505.1, balanced=0.8363, params={'backprop_learning_rate': 0.05, 'backprop_momentum': 0.9}
  4. acc=0.1900, train_sps=2929.7, infer_sps=4543.3, balanced=0.8276, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.95}
  5. acc=0.1700, train_sps=2926.1, infer_sps=4534.3, balanced=0.7153, params={'backprop_learning_rate': 0.005, 'backprop_momentum': 0.9}
  6. acc=0.1043, train_sps=2947.7, infer_sps=4530.9, balanced=0.4608, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.9}
  7. acc=0.1043, train_sps=2899.4, infer_sps=4490.2, balanced=0.2941, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.85}
  8. acc=0.1029, train_sps=2943.7, infer_sps=4515.4, balanced=0.4306, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.85}
  9. acc=0.1029, train_sps=2940.8, infer_sps=4539.1, balanced=0.4410, params={'backprop_learning_rate': 0.03, 'backprop_momentum': 0.9}
  10. acc=0.1000, train_sps=2858.2, infer_sps=4439.5, balanced=0.1192, params={'backprop_learning_rate': 0.003, 'backprop_momentum': 0.9}

Top-10 Training Speed

  1. acc=0.1043, train_sps=2947.7, infer_sps=4530.9, balanced=0.4608, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.9}
  2. acc=0.1029, train_sps=2943.7, infer_sps=4515.4, balanced=0.4306, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.85}
  3. acc=0.1900, train_sps=2943.5, infer_sps=4505.1, balanced=0.8363, params={'backprop_learning_rate': 0.05, 'backprop_momentum': 0.9}
  4. acc=0.1029, train_sps=2940.8, infer_sps=4539.1, balanced=0.4410, params={'backprop_learning_rate': 0.03, 'backprop_momentum': 0.9}
  5. acc=0.2157, train_sps=2939.6, infer_sps=4504.3, balanced=0.9471, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.9}
  6. acc=0.2057, train_sps=2932.7, infer_sps=4286.4, balanced=0.7105, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.95}
  7. acc=0.1900, train_sps=2929.7, infer_sps=4543.3, balanced=0.8276, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.95}
  8. acc=0.1700, train_sps=2926.1, infer_sps=4534.3, balanced=0.7153, params={'backprop_learning_rate': 0.005, 'backprop_momentum': 0.9}
  9. acc=0.1043, train_sps=2899.4, infer_sps=4490.2, balanced=0.2941, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.85}
  10. acc=0.1000, train_sps=2858.2, infer_sps=4439.5, balanced=0.1192, params={'backprop_learning_rate': 0.003, 'backprop_momentum': 0.9}

Top-10 Inference Speed

  1. acc=0.1900, train_sps=2929.7, infer_sps=4543.3, balanced=0.8276, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.95}
  2. acc=0.1029, train_sps=2940.8, infer_sps=4539.1, balanced=0.4410, params={'backprop_learning_rate': 0.03, 'backprop_momentum': 0.9}
  3. acc=0.1700, train_sps=2926.1, infer_sps=4534.3, balanced=0.7153, params={'backprop_learning_rate': 0.005, 'backprop_momentum': 0.9}
  4. acc=0.1043, train_sps=2947.7, infer_sps=4530.9, balanced=0.4608, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.9}
  5. acc=0.1029, train_sps=2943.7, infer_sps=4515.4, balanced=0.4306, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.85}
  6. acc=0.1900, train_sps=2943.5, infer_sps=4505.1, balanced=0.8363, params={'backprop_learning_rate': 0.05, 'backprop_momentum': 0.9}
  7. acc=0.2157, train_sps=2939.6, infer_sps=4504.3, balanced=0.9471, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.9}
  8. acc=0.1043, train_sps=2899.4, infer_sps=4490.2, balanced=0.2941, params={'backprop_learning_rate': 0.02, 'backprop_momentum': 0.85}
  9. acc=0.1000, train_sps=2858.2, infer_sps=4439.5, balanced=0.1192, params={'backprop_learning_rate': 0.003, 'backprop_momentum': 0.9}
  10. acc=0.2057, train_sps=2932.7, infer_sps=4286.4, balanced=0.7105, params={'backprop_learning_rate': 0.01, 'backprop_momentum': 0.95}

PredictiveCodingResNet50

Pareto front size: 4

Top-10 Balanced Score

  1. acc=0.1029, train_sps=2807.7, infer_sps=4534.2, balanced=0.9336, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  2. acc=0.1043, train_sps=2778.0, infer_sps=4500.1, balanced=0.8808, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 14, 'predictive_inference_learning_rate': 0.12}
  3. acc=0.1029, train_sps=2786.6, infer_sps=4473.9, balanced=0.8453, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  4. acc=0.1029, train_sps=2790.2, infer_sps=4458.5, balanced=0.8406, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  5. acc=0.1057, train_sps=2748.5, infer_sps=4431.9, balanced=0.8025, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  6. acc=0.0957, train_sps=2793.7, infer_sps=4527.6, balanced=0.7390, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  7. acc=0.1043, train_sps=2713.0, infer_sps=4393.1, balanced=0.6701, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.1}
  8. acc=0.0857, train_sps=2798.8, infer_sps=4514.6, balanced=0.5127, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  9. acc=0.0857, train_sps=2789.9, infer_sps=4536.4, balanced=0.5118, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  10. acc=0.0857, train_sps=2748.9, infer_sps=4497.5, balanced=0.4009, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}

Top-10 Accuracy

  1. acc=0.1057, train_sps=2748.5, infer_sps=4431.9, balanced=0.8025, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  2. acc=0.1043, train_sps=2778.0, infer_sps=4500.1, balanced=0.8808, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 14, 'predictive_inference_learning_rate': 0.12}
  3. acc=0.1043, train_sps=2713.0, infer_sps=4393.1, balanced=0.6701, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.1}
  4. acc=0.1029, train_sps=2807.7, infer_sps=4534.2, balanced=0.9336, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  5. acc=0.1029, train_sps=2790.2, infer_sps=4458.5, balanced=0.8406, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  6. acc=0.1029, train_sps=2786.6, infer_sps=4473.9, balanced=0.8453, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  7. acc=0.0957, train_sps=2793.7, infer_sps=4527.6, balanced=0.7390, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  8. acc=0.0857, train_sps=2798.8, infer_sps=4514.6, balanced=0.5127, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  9. acc=0.0857, train_sps=2789.9, infer_sps=4536.4, balanced=0.5118, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  10. acc=0.0857, train_sps=2748.9, infer_sps=4497.5, balanced=0.4009, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}

Top-10 Training Speed

  1. acc=0.1029, train_sps=2807.7, infer_sps=4534.2, balanced=0.9336, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  2. acc=0.0857, train_sps=2798.8, infer_sps=4514.6, balanced=0.5127, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  3. acc=0.0957, train_sps=2793.7, infer_sps=4527.6, balanced=0.7390, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  4. acc=0.1029, train_sps=2790.2, infer_sps=4458.5, balanced=0.8406, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  5. acc=0.0857, train_sps=2789.9, infer_sps=4536.4, balanced=0.5118, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  6. acc=0.1029, train_sps=2786.6, infer_sps=4473.9, balanced=0.8453, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  7. acc=0.1043, train_sps=2778.0, infer_sps=4500.1, balanced=0.8808, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 14, 'predictive_inference_learning_rate': 0.12}
  8. acc=0.0857, train_sps=2748.9, infer_sps=4497.5, balanced=0.4009, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  9. acc=0.1057, train_sps=2748.5, infer_sps=4431.9, balanced=0.8025, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  10. acc=0.1043, train_sps=2713.0, infer_sps=4393.1, balanced=0.6701, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.1}

Top-10 Inference Speed

  1. acc=0.0857, train_sps=2789.9, infer_sps=4536.4, balanced=0.5118, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  2. acc=0.1029, train_sps=2807.7, infer_sps=4534.2, balanced=0.9336, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  3. acc=0.0957, train_sps=2793.7, infer_sps=4527.6, balanced=0.7390, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}
  4. acc=0.0857, train_sps=2798.8, infer_sps=4514.6, balanced=0.5127, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  5. acc=0.1043, train_sps=2778.0, infer_sps=4500.1, balanced=0.8808, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.02, 'predictive_inference_steps': 14, 'predictive_inference_learning_rate': 0.12}
  6. acc=0.0857, train_sps=2748.9, infer_sps=4497.5, balanced=0.4009, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  7. acc=0.1029, train_sps=2786.6, infer_sps=4473.9, balanced=0.8453, params={'predictive_head_hidden_dim': 256, 'predictive_learning_rate': 0.008, 'predictive_inference_steps': 10, 'predictive_inference_learning_rate': 0.09}
  8. acc=0.1029, train_sps=2790.2, infer_sps=4458.5, balanced=0.8406, params={'predictive_head_hidden_dim': 384, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  9. acc=0.0814, train_sps=2681.3, infer_sps=4435.8, balanced=0.1226, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.015, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.12}
  10. acc=0.1057, train_sps=2748.5, infer_sps=4431.9, balanced=0.8025, params={'predictive_head_hidden_dim': 512, 'predictive_learning_rate': 0.01, 'predictive_inference_steps': 12, 'predictive_inference_learning_rate': 0.1}

CircadianPredictiveCodingResNet50

Pareto front size: 5

Top-10 Balanced Score

  1. acc=0.1900, train_sps=2506.4, infer_sps=4180.1, balanced=0.6848, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  2. acc=0.1243, train_sps=2574.4, infer_sps=4251.9, balanced=0.5357, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.72, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 3, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  3. acc=0.1000, train_sps=2634.9, infer_sps=4180.6, balanced=0.4606, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.65, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  4. acc=0.1043, train_sps=2595.4, infer_sps=4220.9, balanced=0.4429, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  5. acc=0.1057, train_sps=2572.8, infer_sps=4171.0, balanced=0.3612, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  6. acc=0.1171, train_sps=2590.6, infer_sps=4031.8, balanced=0.3296, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  7. acc=0.0957, train_sps=2591.9, infer_sps=4148.8, balanced=0.3254, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  8. acc=0.1043, train_sps=2553.0, infer_sps=4156.2, balanced=0.3016, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  9. acc=0.1043, train_sps=2540.4, infer_sps=4136.4, balanced=0.2590, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  10. acc=0.1014, train_sps=2559.0, infer_sps=4108.4, balanced=0.2547, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}

Top-10 Accuracy

  1. acc=0.1900, train_sps=2506.4, infer_sps=4180.1, balanced=0.6848, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  2. acc=0.1243, train_sps=2574.4, infer_sps=4251.9, balanced=0.5357, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.72, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 3, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  3. acc=0.1171, train_sps=2590.6, infer_sps=4031.8, balanced=0.3296, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  4. acc=0.1057, train_sps=2572.8, infer_sps=4171.0, balanced=0.3612, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  5. acc=0.1043, train_sps=2595.4, infer_sps=4220.9, balanced=0.4429, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  6. acc=0.1043, train_sps=2553.0, infer_sps=4156.2, balanced=0.3016, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  7. acc=0.1043, train_sps=2540.4, infer_sps=4136.4, balanced=0.2590, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  8. acc=0.1014, train_sps=2559.0, infer_sps=4108.4, balanced=0.2547, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  9. acc=0.1000, train_sps=2634.9, infer_sps=4180.6, balanced=0.4606, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.65, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  10. acc=0.0957, train_sps=2591.9, infer_sps=4148.8, balanced=0.3254, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}

Top-10 Training Speed

  1. acc=0.1000, train_sps=2634.9, infer_sps=4180.6, balanced=0.4606, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.65, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  2. acc=0.1043, train_sps=2595.4, infer_sps=4220.9, balanced=0.4429, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  3. acc=0.0957, train_sps=2591.9, infer_sps=4148.8, balanced=0.3254, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  4. acc=0.1171, train_sps=2590.6, infer_sps=4031.8, balanced=0.3296, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  5. acc=0.1243, train_sps=2574.4, infer_sps=4251.9, balanced=0.5357, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.72, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 3, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  6. acc=0.1057, train_sps=2572.8, infer_sps=4171.0, balanced=0.3612, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  7. acc=0.1014, train_sps=2559.0, infer_sps=4108.4, balanced=0.2547, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  8. acc=0.0857, train_sps=2556.6, infer_sps=4108.7, balanced=0.1675, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  9. acc=0.1043, train_sps=2553.0, infer_sps=4156.2, balanced=0.3016, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  10. acc=0.0957, train_sps=2547.4, infer_sps=4145.5, balanced=0.2358, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.65, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}

Top-10 Inference Speed

  1. acc=0.1243, train_sps=2574.4, infer_sps=4251.9, balanced=0.5357, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.72, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 3, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  2. acc=0.1043, train_sps=2595.4, infer_sps=4220.9, balanced=0.4429, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  3. acc=0.1000, train_sps=2634.9, infer_sps=4180.6, balanced=0.4606, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.65, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  4. acc=0.1900, train_sps=2506.4, infer_sps=4180.1, balanced=0.6848, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  5. acc=0.1057, train_sps=2572.8, infer_sps=4171.0, balanced=0.3612, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 14, 'circadian_inference_learning_rate': 0.18, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.78, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  6. acc=0.1043, train_sps=2553.0, infer_sps=4156.2, balanced=0.3016, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.01, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.82, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  7. acc=0.0957, train_sps=2591.9, infer_sps=4148.8, balanced=0.3254, params={'circadian_head_hidden_dim': 512, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}
  8. acc=0.0957, train_sps=2547.4, infer_sps=4145.5, balanced=0.2358, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.02, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.12, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.65, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  9. acc=0.1043, train_sps=2540.4, infer_sps=4136.4, balanced=0.2590, params={'circadian_head_hidden_dim': 256, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 2, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.68, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 96, 'circadian_max_hidden_dim': 768}
  10. acc=0.0857, train_sps=2556.6, infer_sps=4108.7, balanced=0.1675, params={'circadian_head_hidden_dim': 384, 'circadian_learning_rate': 0.03, 'circadian_inference_steps': 12, 'circadian_inference_learning_rate': 0.15, 'circadian_sleep_interval': 1, 'circadian_split_threshold': 0.8, 'circadian_prune_threshold': 0.7, 'circadian_max_split_per_sleep': 2, 'circadian_max_prune_per_sleep': 2, 'circadian_min_hidden_dim': 128, 'circadian_max_hidden_dim': 1024}

Global Winners

  • Best accuracy: {'model_name': 'BackpropResNet50', 'epochs_ran': 14, 'final_metric_name': 'loss', 'final_metric_value': 325.91778564453125, 'test_accuracy': 0.21571428571428572, 'train_seconds': 11.906183200000001, 'train_samples_per_second': 2939.649038828833, 'mean_train_step_ms': 14.104383749999938, 'inference_latency_mean_ms': 14.14512857142926, 'inference_latency_p95_ms': 14.412194999997396, 'inference_samples_per_second': 4504.327082821833, 'total_parameters': 23528522, 'trainable_parameters': 20490, 'circadian_hidden_dim_start': None, 'circadian_hidden_dim_end': None, 'circadian_total_splits': 0, 'circadian_total_prunes': 0, 'accuracy_per_train_second': 0.01811783693318995, 'accuracy_per_million_trainable_params': 10.52778358781287, 'balanced_score': 0.9471485503269111, 'params': {'backprop_learning_rate': 0.01, 'backprop_momentum': 0.9}}
  • Best training speed: {'model_name': 'BackpropResNet50', 'epochs_ran': 14, 'final_metric_name': 'loss', 'final_metric_value': 3553.029296875, 'test_accuracy': 0.10428571428571429, 'train_seconds': 11.873553199999996, 'train_samples_per_second': 2947.72755976703, 'mean_train_step_ms': 14.073830714285814, 'inference_latency_mean_ms': 14.062157142857094, 'inference_latency_p95_ms': 14.451199999999531, 'inference_samples_per_second': 4530.904118551223, 'total_parameters': 23528522, 'trainable_parameters': 20490, 'circadian_hidden_dim_start': None, 'circadian_hidden_dim_end': None, 'circadian_total_splits': 0, 'circadian_total_prunes': 0, 'accuracy_per_train_second': 0.008783024973999722, 'accuracy_per_million_trainable_params': 5.0895907411280765, 'balanced_score': 0.46075780981674863, 'params': {'backprop_learning_rate': 0.02, 'backprop_momentum': 0.9}}
  • Best inference speed: {'model_name': 'BackpropResNet50', 'epochs_ran': 14, 'final_metric_name': 'loss', 'final_metric_value': 604.1658325195312, 'test_accuracy': 0.19, 'train_seconds': 11.946449000000015, 'train_samples_per_second': 2929.7408794864446, 'mean_train_step_ms': 14.177403928571714, 'inference_latency_mean_ms': 14.023942857142353, 'inference_latency_p95_ms': 14.42687999999066, 'inference_samples_per_second': 4543.2505225758405, 'total_parameters': 23528522, 'trainable_parameters': 20490, 'circadian_hidden_dim_start': None, 'circadian_hidden_dim_end': None, 'circadian_total_splits': 0, 'circadian_total_prunes': 0, 'accuracy_per_train_second': 0.015904307631497843, 'accuracy_per_million_trainable_params': 9.272816007808686, 'balanced_score': 0.8275778219799201, 'params': {'backprop_learning_rate': 0.02, 'backprop_momentum': 0.95}}
  • Best balanced score: {'model_name': 'BackpropResNet50', 'epochs_ran': 14, 'final_metric_name': 'loss', 'final_metric_value': 325.91778564453125, 'test_accuracy': 0.21571428571428572, 'train_seconds': 11.906183200000001, 'train_samples_per_second': 2939.649038828833, 'mean_train_step_ms': 14.104383749999938, 'inference_latency_mean_ms': 14.14512857142926, 'inference_latency_p95_ms': 14.412194999997396, 'inference_samples_per_second': 4504.327082821833, 'total_parameters': 23528522, 'trainable_parameters': 20490, 'circadian_hidden_dim_start': None, 'circadian_hidden_dim_end': None, 'circadian_total_splits': 0, 'circadian_total_prunes': 0, 'accuracy_per_train_second': 0.01811783693318995, 'accuracy_per_million_trainable_params': 10.52778358781287, 'balanced_score': 0.9471485503269111, 'params': {'backprop_learning_rate': 0.01, 'backprop_momentum': 0.9}}