pixyz.losses (Loss API)¶

Loss¶

class pixyz.losses.losses.Loss(p1, p2=None, input_var=None)[source]¶

Bases: object

input_var¶

loss_text¶

abs()[source]¶

mean()[source]¶

sum()[source]¶

estimate(x={}, **kwargs)[source]¶

train(x={}, **kwargs)[source]¶: Train the implicit (adversarial) loss function.

test(x={}, **kwargs)[source]¶: Test the implicit (adversarial) loss function.

Negative expected value of log-likelihood (entropy)¶

CrossEntropy¶

class pixyz.losses.CrossEntropy(p1, p2, input_var=None)[source]¶

Bases: pixyz.losses.losses.Loss

Cross entropy, a.k.a., the negative expected value of log-likelihood (Monte Carlo approximation).

$-\mathbb{E}_{q(x)}[\log p(x)] \approx -\frac{1}{L}\sum_{l=1}^L \log p(x_l),$

where $x_l \sim q(x)$ .

loss_text¶

estimate(x={})[source]¶

Entropy¶

class pixyz.losses.Entropy(p1, input_var=None)[source]¶

Bases: pixyz.losses.losses.Loss

Entropy (Monte Carlo approximation).

$-\mathbb{E}_{p(x)}[\log p(x)] \approx -\frac{1}{L}\sum_{l=1}^L \log p(x_l),$

where $x_l \sim p(x)$ .

Note:: This class is a special case of the CrossEntropy class. You can get the same result with CrossEntropy.

loss_text¶

estimate(x={})[source]¶

StochasticReconstructionLoss¶

class pixyz.losses.StochasticReconstructionLoss(encoder, decoder, input_var=None)[source]¶

Bases: pixyz.losses.losses.Loss

Reconstruction Loss (Monte Carlo approximation).

$-\mathbb{E}_{q(z|x)}[\log p(x|z)] \approx -\frac{1}{L}\sum_{l=1}^L \log p(x|z_l),$

where $z_l \sim q(z|x)$ .

Note:: This class is a special case of the CrossEntropy class. You can get the same result with CrossEntropy.

loss_text¶

estimate(x={})[source]¶

Negative log-likelihood¶

NLL¶

class pixyz.losses.NLL(p, input_var=None)[source]¶

Bases: pixyz.losses.losses.Loss

Negative log-likelihood.

$-\log p(x)$

loss_text¶

estimate(x={})[source]¶

Lower bound¶

ELBO¶

class pixyz.losses.ELBO(p, approximate_dist, input_var=None)[source]¶

Bases: pixyz.losses.losses.Loss

The evidence lower bound (Monte Carlo approximation).

$\mathbb{E}_{q(z|x)}[\log \frac{p(x,z)}{q(z|x)}] \approx \frac{1}{L}\sum_{l=1}^L \log p(x, z_l),$

where $z_l \sim q(z|x)$ .

loss_text¶

estimate(x={}, batch_size=None)[source]¶

Divergence¶

KullbackLeibler¶

class pixyz.losses.KullbackLeibler(p1, p2, input_var=None)[source]¶

Bases: pixyz.losses.losses.Loss

Kullback-Leibler divergence (analytical).

$D_{KL}[p||q] = \mathbb{E}_{p(x)}[\log \frac{p(x)}{q(x)}]$

loss_text¶

estimate(x, **kwargs)[source]¶

Similarity¶

SimilarityLoss¶

class pixyz.losses.SimilarityLoss(p1, p2, input_var=None, var=['z'], margin=0)[source]¶

Bases: pixyz.losses.losses.Loss

Learning Modality-Invariant Representations for Speech and Images (Leidai et. al.)

estimate(x)[source]¶

MultiModalContrastivenessLoss¶

class pixyz.losses.MultiModalContrastivenessLoss(p1, p2, input_var=None, margin=0.5)[source]¶

Bases: pixyz.losses.losses.Loss

Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data

estimate(x)[source]¶

Adversarial loss (GAN loss)¶

AdversarialJensenShannon¶

class pixyz.losses.AdversarialJensenShannon(p, q, discriminator, input_var=None, optimizer=<class 'torch.optim.adam.Adam'>, optimizer_params={}, inverse_g_loss=True)[source]¶

Bases: pixyz.losses.adversarial_loss.AdversarialLoss

Jensen-Shannon divergence (adversarial training).

$D_{JS}[p(x)||q(x)] \leq 2 \cdot D_{JS}[p(x)||q(x)] + 2 \log 2 = \mathbb{E}_{p(x)}[\log d^*(x)] + \mathbb{E}_{q(x)}[\log (1-d^*(x))],$

where $d^*(x) = \arg\max_{d} \mathbb{E}_{p(x)}[\log d(x)] + \mathbb{E}_{q(x)}[\log (1-d(x))]$ .

loss_text¶

estimate(x={}, discriminator=False)[source]¶

d_loss(y1, y2, batch_size)[source]¶

g_loss(y1, y2, batch_size)[source]¶

AdversarialKullbackLeibler¶

class pixyz.losses.AdversarialKullbackLeibler(q, p, discriminator, **kwargs)[source]¶

Bases: pixyz.losses.adversarial_loss.AdversarialLoss

Kullback-Leibler divergence (adversarial training).

$D_{KL}[q(x)||p(x)] = \mathbb{E}_{q(x)}[\log \frac{q(x)}{p(x)}] = \mathbb{E}_{q(x)}[\log \frac{d^*(x)}{1-d^*(x)}],$

where $d^*(x) = \arg\max_{d} \mathbb{E}_{p(x)}[\log d(x)] + \mathbb{E}_{q(x)}[\log (1-d(x))]$ .

Note that this divergence is minimized to close q to p.

loss_text¶

estimate(x={}, discriminator=False)[source]¶

g_loss(y1, batch_size)[source]¶

d_loss(y1, y2, batch_size)[source]¶

AdversarialWassersteinDistance¶

class pixyz.losses.AdversarialWassersteinDistance(p, q, discriminator, clip_value=0.01, **kwargs)[source]¶

Bases: pixyz.losses.adversarial_loss.AdversarialJensenShannon

Wasserstein distance (adversarial training).

$W(p, q) = \sup_{||d||_{L} \leq 1} \mathbb{E}_{p(x)}[d(x)] - \mathbb{E}_{q(x)}[d(x)]$

loss_text¶

d_loss(y1, y2, *args, **kwargs)[source]¶

g_loss(y1, y2, *args, **kwargs)[source]¶

train(train_x, **kwargs)[source]¶: Train the implicit (adversarial) loss function.

Auto-regressive loss¶

ARLoss¶

class pixyz.losses.ARLoss(step_loss, last_loss=None, step_fn=<function ARLoss.<lambda>>, max_iter=1, return_params=False, input_var=None, series_var=None, update_value=None)[source]¶