src.core.parameter_server module

class src.core.parameter_server.ParameterServer(model, param)

Bases: object

Parameter Server for Stale Synchronous Parallel training. The server manages the global model parameters and coordinates the gradient updates from multiple workers. Each worker computes gradients locally and with a push operation sends the result to the server, which aggregates the gradients and updates the model parameters. Each worker can receive the latest model parameters with a pull operation.

Arguments: :param model: PyTorch model instance :type model: nn.Module :param param: Configuration parameters :type param: ConfigParameters

get_hist() → list[int]

Return the raw counts of staleness occurrences for this run.

get_staleness_stats()

Returns a dict: {“per_worker”: { wid: { “mean”:…, “median”:…, “std”:…, “pct_over_bound”:…}, …},”combined”: {“mean”:…,”median”:…,”std”:…,”pct_over_bound”:…}}

get_time_push()

Return the time spent in push and pull operations.

get_version()

Return the current version of the model parameters.

pull()

push(wid, w_version: int, grads: list[Tensor]) → ParameterServerStatus