hyper module

The hyper module provides powerful hyperparameter search and sweep functionality for params-proto configurations.

Core Classes

Sweep

The main class for creating parameter sweeps and hyperparameter searches.

class params_proto.v2.hyper.Sweep

Bases: object

each(fn)

items()

property list

returns self as a list. Currently not idempotent. Might become idempotent in the future.

property dataframe

property noot

property snack

property original

set_param(name, params, prefix=None)

property product: ContextManager[None, bool | None]

property zip: ContextManager[ParamsProto, bool | None]

property set: ContextManager[ParamsProto, bool | None]

property chain: ContextManager[ParamsProto, bool | None]

save(filename='sweep.jsonl', overwrite=True, verbose=True)

static log(deps, filename)

append deps object to a JSONL log file, used as a helper function

static read(filename)

Read JSONL log files, used as a helper function

file = None

load(file='sweep.jsonl', strict=True, silent=False)

Loading sweep state from a jsonl file:

Note: Important Caveat When multiple prefix-free ParamsProto objects are present,

We sweep through all of the proto objects and sets the attribute to the first proto with the correct key. This first-attr approach works because the ParamsProto object also generates argparse parameters, which means repetitive arguments are not possible.

However this would fail in cases where attributes are dynamically added to an argument object. The sweep.jsonl file loses this type of information, therefore there is no way to recover this type of attributes. So the user should try to either use PrefixProto, or explicitly define the attributes.

Usage Pattern 1: Loading from a file:

sweep = Sweep(Args, RUN).load('sweep.jsonl')
for i, deps in enumerate(sweep):
    assert RUN.job_id == i + 1, "the job_id in that sweep.json should be 1-based."

Usage Pattern 2: Loading from a sweep list object or a pandas DataFrame:

sweep_list = Sweep.read(sweep.jsonl)
sweep = Sweep(Args, RUN).load(sweep_list)
for i, deps in enumerate(sweep):
    assert RUN.job_id == i + 1, "the job_id in that sweep.json should be 1-based."

Key Features

• Parameter Combinations: Generate all combinations of parameter values

• Custom Functions: Apply functions to each parameter combination

• Slicing Support: Access specific subsets of the sweep

• Save/Load: Persist sweep configurations to files

Usage Example

from params_proto.v2.proto import ParamsProto, Proto
from params_proto.v2.hyper import Sweep

class Config(ParamsProto):
    learning_rate = Proto("Learning rate", default=0.001)
    batch_size = Proto("Batch size", default=32)
    model = Proto("Model type", default="resnet")

# Create a sweep
sweep = Sweep(Config).product([
    Config.learning_rate << [0.001, 0.01, 0.1],
    Config.batch_size << [16, 32, 64],
    Config.model << ["resnet", "transformer"]
])

# Iterate through combinations
for config in sweep:
    print(f"lr={Config.learning_rate}, bs={Config.batch_size}, model={Config.model}")
    # Run your experiment here

# Save sweep to file
sweep.save("experiments.jsonl")

Parameter Search Methods

Product Sweep

Generate cartesian product of all parameter combinations:

sweep = Sweep(Config).product([
    Config.learning_rate << [0.001, 0.01],
    Config.epochs << [10, 20, 50]
])
# Results in 2×3 = 6 combinations

Zip Sweep

Combine parameters element-wise:

sweep = Sweep(Config).zip([
    Config.learning_rate << [0.001, 0.01, 0.1],
    Config.epochs << [10, 20, 30]
])
# Results in 3 combinations: (0.001,10), (0.01,20), (0.1,30)

Chain Sweep

Concatenate multiple sweeps:

sweep1 = Sweep(Config).product([Config.model << ["resnet"]])
sweep2 = Sweep(Config).product([Config.model << ["transformer"]])
combined = sweep1.chain(sweep2)

Advanced Features

Custom Processing

Apply custom functions to each sweep iteration:

def run_experiment(config):
    print(f"Running with lr={config.learning_rate}")
    # Your training code here
    return {"accuracy": 0.95, "loss": 0.1}

results = []
sweep = Sweep(Config).product([
    Config.learning_rate << [0.001, 0.01, 0.1]
]).each(run_experiment)

for result in sweep:
    results.append(result)

Sweep Slicing

Access specific parts of a sweep:

sweep = Sweep(Config).product([
    Config.learning_rate << [0.001, 0.01, 0.1, 0.5],
    Config.batch_size << [16, 32]
])

# Get first 3 combinations
first_three = sweep[:3]

# Get every other combination
every_other = sweep[::2]

# Get specific combination
specific = sweep[5]

Saving and Loading

Persist sweep configurations:

# Save sweep to JSONL file
sweep.save("hyperparameter_sweep.jsonl")

# Each line contains one parameter combination
# {"Config.learning_rate": 0.001, "Config.batch_size": 32}
# {"Config.learning_rate": 0.01, "Config.batch_size": 32}
# ...

Hyperparameter Search Patterns

Grid Search

from params_proto.v2.hyper import Sweep

class ModelConfig(ParamsProto):
    learning_rate = Proto(default=0.001)
    weight_decay = Proto(default=1e-4)
    dropout = Proto(default=0.1)

# Grid search over hyperparameters
sweep = Sweep(ModelConfig).product([
    ModelConfig.learning_rate << [1e-4, 1e-3, 1e-2],
    ModelConfig.weight_decay << [1e-5, 1e-4, 1e-3],
    ModelConfig.dropout << [0.0, 0.1, 0.2]
])

print(f"Total combinations: {len(sweep)}")  # 27 combinations

# Run experiments
best_accuracy = 0
best_config = None

for i, config in enumerate(sweep):
    print(f"Experiment {i+1}/{len(sweep)}")
    
    # Your training code here
    accuracy = train_model(config)
    
    if accuracy > best_accuracy:
        best_accuracy = accuracy
        best_config = config.__dict__.copy()

print(f"Best accuracy: {best_accuracy}")
print(f"Best config: {best_config}")

Random Search

import random

# Generate random parameter combinations
def random_search(n_trials=50):
    learning_rates = [1e-4, 5e-4, 1e-3, 5e-3, 1e-2]
    weight_decays = [1e-5, 5e-5, 1e-4, 5e-4, 1e-3]
    dropouts = [0.0, 0.1, 0.2, 0.3, 0.4]
    
    configs = []
    for _ in range(n_trials):
        config = {
            'learning_rate': random.choice(learning_rates),
            'weight_decay': random.choice(weight_decays),
            'dropout': random.choice(dropouts)
        }
        configs.append(config)
    
    return configs

# Use with Sweep
random_configs = random_search(20)
for config in random_configs:
    ModelConfig._update(config)
    # Run experiment
    train_model()

Integration with ML Frameworks

PyTorch Integration

import torch
import torch.nn as nn
from params_proto.v2.hyper import Sweep

class TrainingConfig(ParamsProto):
    learning_rate = Proto(default=0.001)
    batch_size = Proto(default=32)
    optimizer = Proto(default="adam")

def train_pytorch_model():
    # Create model
    model = nn.Linear(10, 1)
    
    # Create optimizer based on config
    if TrainingConfig.optimizer == "adam":
        optimizer = torch.optim.Adam(model.parameters(), lr=TrainingConfig.learning_rate)
    elif TrainingConfig.optimizer == "sgd":
        optimizer = torch.optim.SGD(model.parameters(), lr=TrainingConfig.learning_rate)
    
    # Training loop
    for epoch in range(10):
        # Your training code here
        pass
    
    return model

# Hyperparameter sweep
sweep = Sweep(TrainingConfig).product([
    TrainingConfig.learning_rate << [0.001, 0.01, 0.1],
    TrainingConfig.optimizer << ["adam", "sgd"]
])

for config in sweep:
    model = train_pytorch_model()
    # Evaluate and save results

Complete Module Reference

class params_proto.v2.hyper.TypeVar

Bases: object

Type variable.

The preferred way to construct a type variable is via the dedicated syntax for generic functions, classes, and type aliases:

class Sequence[T]:  # T is a TypeVar
    ...

This syntax can also be used to create bound and constrained type variables:

# S is a TypeVar bound to str
class StrSequence[S: str]:
    ...

# A is a TypeVar constrained to str or bytes
class StrOrBytesSequence[A: (str, bytes)]:
    ...

Type variables can also have defaults:

class IntDefault[T = int]:

…

However, if desired, reusable type variables can also be constructed manually, like so:

T = TypeVar('T')  # Can be anything
S = TypeVar('S', bound=str)  # Can be any subtype of str
A = TypeVar('A', str, bytes)  # Must be exactly str or bytes
D = TypeVar('D', default=int)  # Defaults to int

Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function and type alias definitions.

The variance of type variables is inferred by type checkers when they are created through the type parameter syntax and when infer_variance=True is passed. Manually created type variables may be explicitly marked covariant or contravariant by passing covariant=True or contravariant=True. By default, manually created type variables are invariant. See PEP 484 and PEP 695 for more details.

classmethod __new__(*args, **kwargs)

has_default()

params_proto.v2.hyper.dot_join(*keys)

remove Nones from the keys, but not ‘’,

class params_proto.v2.hyper.Item

Bases: tuple

Item(key, value)

static __new__(_cls, key, value)

Create new instance of Item(key, value)

key

Alias for field number 0

value

Alias for field number 1

params_proto.v2.hyper.key_items(d, prefix=None)

Takes in tuples of [key, value], or [None, [[key, value], …]] returns wtf

params_proto.v2.hyper.flatten_items(row) → Iterable[Item]

Return type:

Iterable[Item]