Evaluation

Evaluation#

tn4ml.eval.plot_loss(history, validation=True, figsize=(5, 5), save_path=None, legend_args={})[source][source]#

Plot the loss of the model during training and validation.

Parameters:

  • history (dict) – History object from the model training.

  • validation (bool) – Whether to plot the validation loss.

  • figsize (tuple) – Size of the figure.

  • save_path (str) – Path to save the plot.

  • legend_args (dict)

Return type:

Displays the plot.

tn4ml.eval.plot_accuracy(history, figsize=(5, 5), save_path=None, legend_args={})[source][source]#

Plot the accuracy of the model during training and validation.

Parameters:

  • history (dict) – History object from the model training.

  • validation (bool) – Whether to plot the validation accuracy.

  • figsize (tuple) – Size of the figure.

  • save_path (str) – Path to save the plot.

  • legend_args (dict) – Arguments for the legend.

Return type:

Displays or saves the plot.

tn4ml.eval.get_roc_curve_data(y_true, y_scores, anomaly_det=False)[source][source]#

Calculate the ROC curve data from normal and anomaly scores. Use it when both y_true and y_scores are not binary.

Parameters:

  • y_true (numpy.ndarray) – True or normal scores.

  • y_scores (numpy.ndarray) – Predicted scores or anomaly scores.

  • anomaly (bool) – Whether the scores are anomaly scores or

  • anomaly_det (bool)

Returns:

  • fpr_loss (numpy.ndarray) – False positive rate values.

  • tpr_loss (numpy.ndarray) – True positive rate values.

tn4ml.eval.get_precision_recall_curve_data(y_true, y_scores, anomaly_det=False)[source][source]#

Calculate the ROC curve data from normal and anomaly scores. Use it when both y_true and y_scores are not binary.

Parameters:

  • y_true (numpy.ndarray) – True or normal scores.

  • y_scores (numpy.ndarray) – Predicted scores or anomaly scores.

  • anomaly_det (bool)

Returns:

  • fpr_loss (numpy.ndarray) – False positive rate values.

  • tpr_loss (numpy.ndarray) – True positive rate values.

tn4ml.eval.get_FPR_for_fixed_TPR(tpr_window, fpr, tpr, tolerance)[source][source]#

Calculate the FPR for a fixed TPR value.

Parameters:

  • tpr_window (float) – Fixed TPR value.

  • fpr (numpy.ndarray) – False positive rate values.

  • tpr (numpy.ndarray) – True positive rate values.

  • tolerance (float) – Tolerance value for the fixed TPR value.

Returns:

fpr – FPR value for the fixed TPR value.

Return type:

float

tn4ml.eval.get_TPR_for_fixed_FPR(fpr_window, fpr, tpr, tolerance)[source][source]#

Calculate the TPR for a fixed FPR value.

Parameters:

  • fpr_window (float) – Fixed FPR value.

  • fpr (numpy.ndarray) – False positive rate values.

  • tpr (numpy.ndarray) – True positive rate values.

  • tolerance (float) – Tolerance value for the fixed FPR value.

Returns:

tpr – TPR value for the fixed FPR value.

Return type:

float

tn4ml.eval.get_mean_and_error(data)[source][source]#

Calculate the mean and standard deviation of the input data.

Parameters:

data (numpy.ndarray) – Input data to calculate the mean and standard deviation.

Returns:

  • mean (numpy.ndarray) – Mean of the input data.

  • std (numpy.ndarray) – Standard deviation of the input data.

tn4ml.eval.plot_ROC_curve_from_metrics(y_true, y_scores, title='ROC Curve', save_path=None)[source][source]#

Calculates TPR and FPR from input metrics and plots the ROC curve.

Parameters:

  • y_true (numpy.ndarray) – List or array of true binary labels (0 or 1).

  • y_scores (numpy.ndarray) – List or array of predicted scores or probabilities.

  • title (str (Optional)) – Title for the plot. Defaults to “ROC Curve”.

  • save_path (str (Optional)) – Path and name to save the plot.

Return type:

Displays or saves the plot.

tn4ml.eval.plot_ROC_curve_from_data(fpr, tpr, title='ROC Curve', save_path=None)[source][source]#

Plots the ROC curve from input FPR and TPR values.

Parameters:

  • fpr_loss (numpy.ndarray) – False positive rate values.

  • tpr_loss (numpy.ndarray) – True positive rate values.

  • title (str (Optional)) – Title for the plot. Defaults to “ROC Curve”.

  • save_path (str (Optional)) – Path and name to save the plot. Example: ./ROC_curve.pdf

  • fpr (ndarray)

  • tpr (ndarray)

Return type:

Displays or saves the plot.

tn4ml.eval.plot_PR_curve(y_true, y_scores, title='Precision-Recall Curve', save_path=None)[source][source]#

Calculates precision and recall from input metrics and plots the Precision-Recall curve.

Parameters:

  • y_true (numpy.ndarray) – List or array of true binary labels (0 or 1).

  • y_scores (numpy.ndarray) – List or array of predicted scores or probabilities.

  • title (str (Optional)) – Title for the plot. Defaults to “Precision-Recall Curve”.

  • save_path (str (Optional)) – Path and name to save the plot.

Return type:

Displays or saves the plot.

tn4ml.eval.compare_AUC(save_dir='.', bond_dims=None, spacings=None, initializers=None, embedding='trigonometric', nruns=0, fig_size=(6, 5), labels=None, anomaly_det=False)[source][source]#

Example of code to compare the TPR values for fixed FPR for different values of hyperparameters, when spacing parameter is fixed. - code for generating plots from the paper “tn4ml: Tensor Network Training and Customization for Machine Learning”

This works with the results saved in the directory structure as follows:

root_dir/initializer_string/bond_' + str(bond_dim) + '/spacing_' + str(spacing) + '/' + embedding_string+'/run_' + str(nrun)

Example:

root_dir/randn_1e-1/bond_10/spacing_2/trigonometric/run_1
Parameters:

  • save_dir (str) – Directory where the results are saved.

  • bond_dims (list[int]) – List of bond dimensions.

  • spacings (list[int]) – List of spacing values. If model is tn4ml.models.smpo.SpacedMatrixProductOperator, then spacing is required.

  • initializers (list[str]) – List of initializers.

  • embedding (str) – List of embeddings.

  • nruns (int) – Number of runs for each model. Assumes that the model is run at least 2 times.

  • fig_size (tuple) – Size of the figure.

  • labels (dict) –

    Dictionary containing the labels for the bond dimensions Example:

    LABELS = {'5': (r'bond = 5', 'o', '#016c59'),
        '10': (r'bond = 10','X', '#7a5195'),
        '30': (r'bond = 30', 'v', '#67a9cf'),
        '50': (r'bond = 50', 'd', '#ffa600')}

  • anomaly_det (bool)

Return type:

Displays or saves the plot.

tn4ml.eval.compare_TPR_per_FPR(save_dir='.', FPR_fixed=0.1, bond_dims=None, spacings=None, initializers=None, embedding='trigonometric', nruns=0, fig_size=(6, 5), labels=None, anomaly_det=False)[source][source]#

Example of code to compare the TPR values for fixed FPR for different values of hyperparameters, when spacing parameter is fixed.

This works with the results saved in the directory structure as follows:

root_dir/initializer_string/bond_<bond_dim>/spacing_<spacing>/<embedding_string>/run_<nrun>

Example:

root_dir/randn_1e-1/bond_10/spacing_2/trigonometric/run_1
Parameters:

  • save_dir (str) – Directory where the results are saved.

  • FPR_fixed (float) – Fixed FPR value.

  • bond_dims (list[int]) – List of bond dimensions.

  • spacings (list[int]) – List of spacing values. If model is tn4ml.models.smpo.SpacedMatrixProductOperator, then spacing is required.

  • initializers (list[str]) – List of initializers.

  • embedding (str) – Embedding method used.

  • nruns (int) – Number of runs for each model. Assumes that the model is run at least 2 times.

  • fig_size (tuple) – Size of the figure.

  • labels (dict) –

    Dictionary containing the labels for the bond dimensions.

    Example:

    LABELS = {'5': (r'bond = 5', 'o', '#016c59'),
        '10': (r'bond = 10', 'X', '#7a5195'),
        '30': (r'bond = 30', 'v', '#67a9cf'),
        '50': (r'bond = 50', 'd', '#ffa600')}

  • anomaly_det (bool)

Returns:

Displays or saves the plot.

Return type:

None

tn4ml.eval.compare_FPR_per_TPR(save_dir='.', TPR_fixed=0.95, bond_dims=None, spacings=None, initializers=None, embedding='trigonometric', nruns=0, fig_size=(6, 5), labels=None, anomaly_det=False)[source][source]#

Example of code to compare the FPR values for fixed TPR for different values of hyperparameters, when spacing parameter is fixed. - code for generating plots from the paper “tn4ml: Tensor Network Training and Customization for Machine Learning”

This works with the results saved in the directory structure as follows:

root_dir/initializer_string/bond_' + str(bond_dim) + '/spacing_' + str(spacing) + '/' + embedding_string+'/run_' + str(nrun)

Example:

root_dir/randn_1e-1/bond_10/spacing_2/trigonometric/run_1
Parameters:

  • save_dir (str) – Directory where the results are saved.

  • TPR_fixed (float) – Fixed TPR value.

  • bond_dims (list[int]) – List of bond dimensions.

  • spacings (list[int]) – List of spacing values. If model is tn4ml.models.smpo.SpacedMatrixProductOperator, then spacing is required.

  • initializers (list[str]) – List of initializers.

  • embedding (str) – List of embeddings.

  • nruns (int) – Number of runs for each model. Assumes that the model is run at least 2 times.

  • fig_size (tuple) – Size of the figure.

  • labels (dict) –

    Dictionary containing the labels for the bond dimensions Example:

    LABELS = {'5': (r'bond = 5', 'o', '#016c59'),
        '10': (r'bond = 10','X', '#7a5195'),
        '30': (r'bond = 30', 'v', '#67a9cf'),
        '50': (r'bond = 50', 'd', '#ffa600')}

  • anomaly_det (bool)

Return type:

Displays or saves the plot.

Contents