src package

Subpackages

• src.core package
  • Submodules
  • src.core.data_types module
    • ParameterServerStatus
      • ParameterServerStatus.ACCEPTED
      • ParameterServerStatus.REJECTED
      • ParameterServerStatus.SHUTDOWN
  • src.core.parameter_server module
    • ParameterServer
      • ParameterServer.get_hist()
      • ParameterServer.get_staleness_stats()
      • ParameterServer.get_time_push()
      • ParameterServer.get_version()
      • ParameterServer.pull()
      • ParameterServer.push()
  • src.core.parameter_server_asap module
    • ParameterServerASAP_SGD
      • ParameterServerASAP_SGD.push()
  • src.core.parameter_server_dasgd module
    • ParameterServerDASGD
      • ParameterServerDASGD.push()
  • src.core.parameter_server_saasgd module
    • ParameterServerSAASGD
      • ParameterServerSAASGD.push()
  • src.core.train_runner module
    • PSManager
      • PSManager.ParameterServer()
      • PSManager.get_hist()
      • PSManager.get_staleness_stats()
      • PSManager.get_time_push()
    • run_training()
  • src.core.utils module
    • build_model()
    • evaluate_model()
    • exp_delay()
    • l1_norm()
    • l2_norm()
    • parse_args()
    • set_seed()
    • sgd_training()
    • sparsity_ratio()
    • weight_kurtosis()
  • src.core.worker module
    • worker()
  • Module contents
• src.data package
  • Submodules
  • src.data.base module
    • AbstractDataBuilder
      • AbstractDataBuilder.split_data()
  • src.data.datasets module
    • create_linear_data_loader()
    • create_linear_dataset()
    • create_poly_varied_data_loader()
    • create_poly_varied_dataset()
    • load_linear_data()
    • load_poly_varied_data()
    • split_data()
  • src.data.full module
    • FullDataLoaderBuilder
  • src.data.linear module
    • LinearRegressionBuilder
  • src.data.poly module
    • PolyVariedBuilder
  • Module contents
• src.experiments package
  • Submodules
  • src.experiments.asap_sgd module
    • main()
  • src.experiments.dasgd module
    • main()
  • src.experiments.saasgd module
    • main()
  • Module contents
• src.models package
  • Submodules
  • src.models.linear module
    • LinearNetModel
      • LinearNetModel.forward()
  • Module contents

Submodules

src.config module

class src.config.ConfigParameters(num_workers: int = 5, staleness: int = 50, lr: float = 0.01, local_steps: int = 1, batch_size: int = 10, device: str = 'cpu', log_level: int = 20, tol: float = 1e-08, Amplitude: float = 1)

Bases: object

Configuration for Stale Synchronous Parallel training for Asynchronous SGD (SSP-ASGD).

Parameters:

• num_workers (int) – Number of worker processes.

• staleness (int) – Staleness bound allowed for the workers during training. Represents the maximum number of versions a worker can be behind the latest version.

• lr (float) – Learning rate for the model. Represents the step size for updating the model parameters.

• local_steps (int) – Number of steps/updates each worker locally computes before pushing gradients to the server.

• batch_size (int) – Batch size for each training step and the data loader.

• device (str) – Device to use for training (e.g., “cuda” or “cpu”).

• log_level (int) – Logging verbosity level.

Amplitude: float = 1

batch_size: int = 10

device: str = 'cpu'

local_steps: int = 1

log_level: int = 20

lr: float = 0.01

num_workers: int = 5

staleness: int = 50

tol: float = 1e-08

src.run_tests module

src.run_tests.main()

Main function to run the experiment based on command line arguments. It accepts two arguments: test and overparameterization.

E.g. to run the script: python src/run_tests.py dasgd 200 python -m src.run_tests saasgd 110

Parameters:

• test – The test/experiment to run. Options are ‘saasgd’, ‘dasgd’, ‘asap_sgd’.

• overparam – The overparameterization size. Options are ‘110’, ‘150’, ‘200’.

Returns:

None

Module contents

Package Initialization Module

The file initializes the initializes the package and imports necessary modules.