`gemclus.sparse`.SparseLinearModel¶

class gemclus.sparse.SparseLinearModel(n_clusters=3, gemini='mmd_ova', groups=None, max_iter=1000, learning_rate=0.001, alpha=0.01, batch_size=None, dynamic=False, solver='adam', verbose=False, random_state=None)[source]¶

This is the SparseLinearModel clustering model. When deriving, the only methods to adapt is the _compute_gemini methods which should be able to return the gradient with respect to the conditional distribution p(y|x).

On top of the vanilla Linear GEMINI model, this variation brings a group-lasso penalty constraint to ensure feature selection via a proximal gradient during training.

Parameters:

n_clustersint, default=3

The maximum number of clusters to form as well as the number of output neurons in the neural network.

gemini: str, GEMINI instance or None, default=”mmd_ova”

GEMINI objective used to train this discriminative model. Can be “mmd_ova”, “mmd_ovo”, “wasserstein_ova”, “wasserstein_ovo”, “mi” or other GEMINI available in gemclus.gemini.AVAILABLE_GEMINI. Default GEMINIs involve the Euclidean metric or linear kernel. To incorporate custom metrics, a GEMINI can also be passed as an instance. If set to None, the GEMINI will be MMD OvA with linear kernel.

groups: list of arrays of various shapes, default=None

If groups is set, it must describe a partition of the indices of variables. This will be used for performing variable selection with groups of features considered to represent one variable. This option can typically be used for one-hot-encoded variables. Variable indices that are not entered will be considered alone. For example, with 3 features, accepted values can be [[0],[1],[2]], [[0,1],[2]] or [[0,1]].

max_iter: int, default=1000

Maximum number of epochs to perform gradient descent in a single run.

learning_rate: float, default=1e-3

Initial learning rate used. It controls the step-size in updating the weights.

dynamic: bool, default=False

Whether to run the path in dynamic mode or not. The dynamic mode consists of affinities computed using only the subset of selected variables instead of all variables.

solver: {‘sgd’,’adam’}, default=’adam’

The solver for weight optimisation.

‘sgd’ refers to stochastic gradient descent.
‘adam’ refers to a stochastic gradient-based optimiser proposed by Kingma, Diederik and Jimmy Ba.

alpha: float, default=1e-2

The weight of the group-lasso penalty in the optimisation scheme.

batch_size: int, default=None

The size of batches during gradient descent training. If set to None, the whole data will be considered.

verbose: bool, default=False

Whether to print progress messages to stdout

random_state: int, RandomState instance, default=None

Determines random number generation for weights and bias initialisation. Pass an int for reproducible results across multiple function calls.

Attributes:

W_: ndarray of shape (n_features, n_clusters): The linear weights of model
b_: ndarray of shape (1, n_clusters): The biases of the model
optimiser_: `AdamOptimizer` or `SGDOptimizer`: The optimisation algorithm used for training depending on the chosen solver parameter.
labels_: ndarray of shape (n_samples): The labels that were assigned to the samples passed to the fit() method.
n_iter_: int: The number of iterations that the model took for converging.
groups_: list of lists of int or None: The explicit partition of the variables formed by the groups parameter if it was not None.

Examples using `gemclus.sparse.SparseLinearModel`¶

Feature selection using the Sparse MMD OvO (Logistic regression)

Feature selection using the Sparse Linear MI (Logistic regression)

Grouped Feature selection with a linear model

gemclus.sparse.SparseLinearModel¶

Examples using gemclus.sparse.SparseLinearModel¶

`gemclus.sparse`.SparseLinearModel¶

Examples using `gemclus.sparse.SparseLinearModel`¶