SmashConfig

SmashConfig is an essential tool in pruna for configuring parameters to optimize your models. The SmashConfig contains configuration for pruna. You can see the configuration for different components in the Customize components section.

Usage examples SmashConfig

This manual explains how to define and use a SmashConfig.

from pruna import SmashConfig
smash_config = SmashConfig()

After creating an empty SmashConfig, you can set activate a algorithm by adding it to the SmashConfig:

smash_config['quantizer'] = 'hqq'

Additionally, you can overwrite the defaults of the Algorithms you have added by setting the hyperparameters in the SmashConfig:

smash_config['hqq_weight_bits'] = 4

You’re done! You created your SmashConfig and can now pass it to the smash function.

Class API SmashConfig

class SmashConfig(configuration=None, batch_size=1, device=None, cache_dir_prefix=PosixPath('/home/docs/.cache/pruna'))

Wrapper class to hold a ConfigSpace Configuration object as a Smash configuration.

Parameters:

  • configuration (list[str] | Dict[str, Any] | Configuration | None, optional) – The configuration to be used for smashing. If None, a default configuration will be created.

  • batch_size (int, optional) – The number of batches to process at once. Default is 1.

  • device (str | torch.device | None, optional) – The device to be used for smashing, options are “cpu”, “cuda”, “mps”, “accelerate”. Default is None. If None, the best available device will be used.

  • cache_dir_prefix (str, optional) – The prefix for the cache directory. If None, a default cache directory will be created.

add(request)

Add an algorithm or specify the hyperparameters of an algorithm to the SmashConfig.

Parameters:

request (str | list[str] | dict[str, Any]) – The value to add to the SmashConfig.

Return type:

None

Examples

>>> config = SmashConfig()
>>> config = SmashConfig()
>>> config.add("fastercache")
>>> config.add("diffusers_int8")
>>> config
SmashConfig(
 'fastercache': True,
 'diffusers_int8': True,
)
add_data(arg)
add_data(dataset_name, *args, **kwargs)
add_data(datasets, collate_fn, *args, **kwargs)
add_data(datasets, collate_fn, *args, **kwargs)
add_data(datamodule)

Add data to the SmashConfig.

Parameters:

arg (Any) – The argument to be used.

add_processor(processor)

Add a processor to the SmashConfig.

Parameters:

processor (str | transformers.AutoProcessor) – The processor to be added to the SmashConfig.

Return type:

None

add_tokenizer(tokenizer)

Add a tokenizer to the SmashConfig.

Parameters:

tokenizer (str | transformers.AutoTokenizer) – The tokenizer to be added to the SmashConfig.

Return type:

None

flush_configuration()

Remove all algorithm hyperparameters from the SmashConfig.

Examples

>>> config = SmashConfig(["fastercache", "diffusers_int8"])
>>> config.flush_configuration()
>>> config
SmashConfig()
Return type:

None

load_from_json(path)

Load a SmashConfig from a JSON file.

Parameters:

path (str| Path) – The file path to the JSON file containing the configuration.

Return type:

None

save_to_json(path)

Save the SmashConfig to a JSON file, including additional keys.

Parameters:

path (str| Path]) – The file path where the JSON file will be saved.

Return type:

None