phlearn.phnns

The phnns subpackage implements neural networks and functionality for learning dynamic systems and pseudo-Hamiltonian systems.

Classes present in phlearn.phnns

Functions present in phlearn.phnns

load_phnn_model()

store_phnn_model()

load_cdnn_model()

store_cdnn_model()

load_baseline_model()

store_baseline_model()

generate_dataset()

train()

compute_validation_loss()

batch_data()

train_one_epoch()

l1_loss_pHnn()

npoints_to_ntrajectories_tsample()

load_dynamic_system_model()

store_dynamic_system_model()

class phlearn.phnns.A_estimator(kernel_size=3, ttype=torch.float32)

Bases: Module

Creates an estimator of a symmetric convolution operator to apply to the left-hand side of the PDE system or the integral V.

Parameters
kernel_sizeint
ttypetorch type, default torch.float32

Methods

forward([x])

Returns
forward(x=None)
Returns
(N, N) tensor

Damping matrix

training: bool
class phlearn.phnns.BaseNN(nstates, noutputs, hidden_dim, timedependent, statedependent, ttype=torch.float32)

Bases: Module

Neural network with three hidden layers, where the first has Tanh-activation, the second has ReLU-activation and the third has linear activation. The network can take either system states or time or both as input. If it is expected to take neither states nor time as input, the network is replaced by trainable parameters. Independently of whether the network uses state and/or time or neither, it can be called with both state and time:

pred = network(x=x, t=t)
Parameters
nstatesint

Number of states in a potential state input.

noutputsint

Number of outputs from the last linear layer.

hidden_dimint

Dimension of hidden layers.

timedependentbool

If True, time input is expected.

statedependentbool

If True, state input is expected.

training: bool
class phlearn.phnns.BaselineNN(nstates, hidden_dim, timedependent=True, statedependent=True)

Bases: BaseNN

Neural network for estimating the right hand side of a set of dynamic system equations with three hidden layers, where the first has Tanh-activation, the second has ReLU-activation and the third has linear activation. The network can take either system states or time or both as input. If it is expected to take neither states nor time as input, the network is replaced by trainable parameters. Independently of whether the network uses state and/or time or neither, it can be called with both state and time:

pred = network(x=x, t=t)

The output dimension of the network is always nstates.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint

Dimension of hidden layers.

timedependentbool

If True, time input is expected.

statedependentbool

If True, state input is expected.

training: bool
class phlearn.phnns.BaselineSplitNN(nstates, hidden_dim, noutputs_x=None, noutputs_t=None, external_forces_filter_x=None, external_forces_filter_t=None, ttype=torch.float32)

Bases: Module

Composition of two neural networks for estimating the right hand side of a set of dynamic system equations. The networks have three hidden layers, where the first has Tanh-activation, the second has ReLU-activation and the third has linear activation. One network takes system states and the other takes time as input. The output of the composition is the sum of the outputs of the two networks:

pred = network(x, t) = network_x(x) + network_t(t)

Both networks are instantiated from the ExternalForcesNN class, allowing adjustment of the number and location of non-zero contributions from each network. The output dimension of a BaselineSplitNN is always nstates.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint

Dimension of hidden layers. Equal for both networks.

noutputs_xint or None, default None

Number of non-zero outputs to estimate with network_x(x)

noutputs_tint or None, default None

Number of non-zero outputs to estimate with network_t(t)

external_forces_filter_xlistlike of ints or None, default None

If provided, this decides to which states the output of network_x is contributing. See ExternalForcesNN for fruther description.

external_forces_filter_tlistlike of ints or None, default None

If provided, this decides to which states the output of network_t is contributing. See ExternalForcesNN for fruther description.

ttypetorch type, default torch.float32

Methods

forward(x, t)

Defines the computation performed at every call.
forward(x, t)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class phlearn.phnns.CentralPadding(d)

Bases: Module

Module that performs periodic even padding on the last dimension of the input tensor. The tensor is padded by adding the first d elements to the end and the last d elements before the beginning of the tensor.

Parameters
dint

The number of elements to pad on each side of the tensor.

Methods

forward(x)

Defines the computation performed at every call.
forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class phlearn.phnns.DynamicSystemNN(nstates, rhs_model=None, init_sampler=None, controller=None, ttype=torch.float32)

Bases: Module

Base class for implementing neural networks estimating the right hand side of equations of the form:

dx/dt = f(x, t) + u

where x is the system state, t is time and u is optional control inputs.

Parameters
nstatesint

Number of system states.

rhs_modelcallable, default None

Model estimating the right hand side of the above equation. Should take inputs x and t, where x is a tensor of shape (nsamples, nstates) and t is a tensor of shape (nsamples, 1), and return a tensor of shape (nsamples, nstates), estimating the time derivative of each state in each sample.

controllercallable, default None

Additional external ports set by a controller. Callable taking a tensor x of shape (nstates,) and a scalar t as input and returning a tensor of shape (nstates,). Note that this function should not take batch inputs, and that when calling PseudoHamiltonianNN.sample_trajectory when a controller is provided, the Runge-Kutta-4 method will be used for integration in favor of SciPy’s solve_ivp.

init_samplercallable, default None

Function for sampling initial conditions. Callable taking a number specifying the number of inital conditions to sample, M, as input and returning a tensor of shape (M, nstates) with inital conditions for the system. This sampler is used when calling simulate_trajectory() and simulate_trajectories() if no initial condition is provided. If not provided, initial conditions are uniformly sampled from (0, 1).

ttypetorch type, default torch.float32

Methods

seed(seed)

Set the torch seed.

set_controller(controller)

Set controller.

simulate_trajectories(ntrajectories, ...[, ...])

Calls simulate_trajectory() ntrajectories times.

simulate_trajectory(integrator, t_sample[, ...])

Simulate a trajectory using the rhs_model and sample at times t_sample.

time_derivative(integrator, *args, **kwargs)

See :py:meth:~`utils.derivatives.time_derivative`

lhs
lhs(dxdt)
seed(seed)

Set the torch seed.

Parameters
seedint
set_controller(controller)

Set controller.

simulate_trajectories(ntrajectories, integrator, t_sample, x0=None, noise_std=0, references=None)

Calls simulate_trajectory() ntrajectories times.

Parameters
integratorstr or False

Specifies which solver to use during simulation. If False, the problem is left to scipy’s solve_ivp. If ‘euler’, the system is simulated with the forward Euler method. If ‘midpoint’, ‘rk4’ or ‘srk4’ the system is simulated with the classic Runge-Kutta-4 method.

t_sample(T, 1) or (ntrajectories, T, 1) tensor or ndarray

Times at which the trajectory is sampled.

x0(ntrajectories, N) tensor or ndarray, default None

Initial condition. If None, an initial condition is sampled with the internal sampler.

noise_stdnumber, default 0.

Standard deviation of Gaussian white noise added to the samples of the trajectory.

referenceslist of phlearn.control.Reference, default
None

If the system has a controller a list of ntrajectories reference objects may be passed.

Returns
xs(ntrajectories, T, N) tensor
t_sample(ntrajectories, T, 1) tensor
us(ntrajectories, T, N) tensor or None
simulate_trajectory(integrator, t_sample, x0=None, xspatial=None, noise_std=0.0, reference=None)

Simulate a trajectory using the rhs_model and sample at times t_sample.

Parameters
integratorstr or False

Specifies which solver to use during simulation. If False, the problem is left to scipy’s solve_ivp. If ‘euler’, ‘midpoint’, ‘rk4’ or ‘srk4’ the system is simulated with the forward euler method, the implicit midpoint method, the explicit Runge-Kutta 4 method or a symmetric fourth order Runge-Kutta method, respectively.

t_sample(T, 1) tensor or ndarray

Times at which the trajectory is sampled.

x0(N,) tensor or ndarray, default None

Initial condition. If None, an initial condition is sampled with the internal sampler.

noise_stdnumber, default 0.

Standard deviation of Gaussian white noise added to the samples of the trajectory.

referencephlearn.control.Reference, default None

If the system has a controller a reference object may be passed.

Returns
xs(T, N) tensor
us(T, N) tensor
time_derivative(integrator, *args, **kwargs)

See :py:meth:~`utils.derivatives.time_derivative`

training: bool
class phlearn.phnns.EarlyStopping(patience=None, min_delta=0.0)

Bases: object

Keeps track of whether the validation loss has decreased by at least min_delta for the last patience calls.

Parameters
patienceint, default None

If None, patience is infinite

min_deltanumber, default 0.

Methods

__call__(val_loss)

Checks if val_loss is at least self.min_delta smaller than the so far observed lowest validation loss.
class phlearn.phnns.ExternalForcesNN(nstates, noutputs, hidden_dim, timedependent, statedependent, external_forces_filter=None, ttype=torch.float32)

Bases: BaseNN

Neural network for estimating external forces of a pseudo-Hamiltonian system with three hidden layers, where the first has Tanh-activation, the second has ReLU-activation and the third has linear activation. The network can take either system states or time or both as input. Independently of whether the network uses state and/or time, it can be called with both state and time:

pred = network(x=x, t=t)

If neither time or state input is to be expected, the neural network is replaced by trainable parameters.

Parameters
nstatesint

Number of states in a potential state input.

noutputsint

Number of external forces to estimate.

timedependentbool

If True, time input is expected.

statedependentbool

If True, state input is expected.

external_forces_filterlistlike of ints or None, default None

If None, noutputs == nstates must be true. In this case, one external force is estimated for each state. If noutputs != nstates, external_forces_filter must decribe which states external forces should be estimated for. Either, external_forces_filter must be a 1d liststructure of length nstates filled with 0 and 1, where 1 indicates that an external force should be estimated for state corresponding to that index. Alternatively, external_forces_filter can be an array of shape (nstates, noutputs) of 0s and 1s, such that when it is multiplied with the network outout of shape (noutputs,), the right output is applied to the correct state.

ttypetorch type, default torch.float32
training: bool
class phlearn.phnns.ForwardPadding(d)

Bases: Module

Module that performs periodic forward padding on the last dimension of the input tensor. The tensor is padded by adding the first d elements to the end of the tensor.

Parameters
dint

The number of elements to pad at the end of the tensor.

Methods

forward(x)

Defines the computation performed at every call.
forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class phlearn.phnns.HamiltonianNN(nstates, hidden_dim=100)

Bases: BaseNN

Neural network for estimating a scalar function H(x), with three hidden layers, where the first has Tanh-activation, the second has ReLU-activation and the third has linear activation. The network takes system states as input, but can be called with both state and time:

pred = network(x=x, t=t)

The output dimension of the network is always 1.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint

Dimension of hidden layers.

training: bool
class phlearn.phnns.PDEBaseNN(nstates, noutputs, hidden_dim, timedependent, statedependent, spacedependent=False, ttype=torch.float32)

Bases: Module

Base neural network module for solving partial differential equations (PDEs). The network can handle various combinations of time, state, and spatial inputs. It consists of multiple convolutional layers, linear layers, and activation functions. The specific architecture depends on the input dependencies specified during initialization.

Parameters
nstatesint

Number of states in a potential state input.

noutputsint

Number of outputs from the last linear layer.

hidden_dimint

Dimension of hidden layers.

timedependentbool

If True, time input is expected.

statedependentbool

If True, state input is expected.

spacedependentbool, optional

If True, spatial input is expected. Default is False.

ttypetorch.dtype, optional

Data type for the time input tensor. Default is torch.float32.

training: bool
class phlearn.phnns.PDEBaselineNN(nstates, hidden_dim=100, timedependent=False, statedependent=True, spacedependent=False, period=20, number_of_intermediate_outputs=4)

Bases: PDEBaseNN

Neural network for estimating the right-hand side of spatially discretized PDEs. It is based on the PDEBaseNN architecture and includes additional layers and parameters specific to the baseline model.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint, optional

Dimension of hidden layers. Default is 100.

timedependentbool, optional

If True, time input is expected. Default is False.

statedependentbool, optional

If True, state input is expected. Default is True.

spacedependentbool, optional

If True, spatial input is expected. Default is False.

periodint, optional

Period value used in the model. Default is 20.

number_of_intermediate_outputsint, optional

Number of intermediate outputs. Default is 4.

training: bool
class phlearn.phnns.PDEBaselineSplitNN(nstates, hidden_dim=100, timedependent=False, statedependent=True, spacedependent=False, period=20, number_of_intermediate_outputs=4)

Bases: Module

A model composed of a PDEBaselineNN model only depending on the state variables and an PDEExternalForcesNN model that can depend on space and/or time variables.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint, optional

Dimension of hidden layers. Default is 100.

timedependentbool, optional

If True, time input is expected. Default is False.

statedependentbool, optional

If True, state input is expected. Default is True.

spacedependentbool, optional

If True, spatial input is expected. Default is False.

periodint, optional

Period value used in the PDEExternalForcesNN model. Default is 20.

number_of_intermediate_outputsint, optional

Number of intermediate outputs in the PDEBaselineNN model. Default is 4.

Methods

forward(x, t, xspatial)

Defines the computation performed at every call.
forward(x, t, xspatial)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
class phlearn.phnns.PDEExternalForcesNN(nstates, hidden_dim=100, timedependent=False, spacedependent=True, statedependent=False, period=20, ttype=torch.float32)

Bases: PDEBaseNN

Neural network for estimating the external forces in a pseudo-Hamiltonian PDE. It is based on the PDEBaseNN architecture but sets up a network where the kernel size is 1 for all convolutional layers, including the first one, so that it cannot learn approximations of spatial derivatives. The network can take any combination of state, space and time as input.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint, optional

Dimension of hidden layers. Default is 100.

timedependentbool, optional

If True, time input is expected. Default is False.

spacedependentbool, optional

If True, spatial input is expected. Default is True.

statedependentbool, optional

If True, state input is expected. Default is False.

periodint, optional

Period value used in the model. Default is 20.

ttypetorch.dtype, optional

Data type for the time input tensor. Default is torch.float32.

training: bool
class phlearn.phnns.PDEIntegralNN(nstates, hidden_dim=100)

Bases: PDEBaseNN

Neural network for estimating the quadrature approximation of an integral over the spatial discretization points where the states of a PDE is evaluated. It is based on the PDEBaseNN architecture and takes state variables as input and outputs a scalar function.

Parameters
nstatesint

Number of states in the input.

hidden_dimint, optional
Dimension of hidden layers. Default is 100.
training: bool
class phlearn.phnns.PseudoHamiltonianNN(nstates, skewsymmetric_matrix=None, hamiltonian_true=None, grad_hamiltonian_true=None, dissipation_true=None, external_forces_true=None, hamiltonian_est=None, dissipation_est=None, external_forces_est=None, **kwargs)

Bases: DynamicSystemNN

Implements a pseudo-Hamiltonian neural network abiding to the pseudo-Hamiltonian formulation:

dx/dt = (S(x) - R(x))*grad[H(x)] + F(x, t)

where x is the system state, S is the skew-symmetric interconnection matrix, R is the positive semi-definite dissipation matrix, H is the Hamiltonian of the system and F is the external ports.

It is possible to provide function estimators like neural networks to model R, H and F. All estimators must subclass torch.nn.Module, such that gradients can be recorded with PyTorch.

If R, H or F are known, they can be provided and used in favor of estimators. Note that R, H and F must be functions both taking as input and returning tensors, and that the gradients of H(x) must be available through autograd unless the true gradient is provided.

Parameters
nstatesint

Number of system states.

skewsymmetric_matrix(N, N) tensor

Corresponds to the S matrix. Must either be a (nstates, nstates) tensor, or callable taking a tensor input of shape (nsamples, nstates) and returning an tensor of shape (nsamples, nstates, nstates).

hamiltonian_truecallable, default None

The known Hamiltonian H of the system. Callable taking a torch tensor input of shape (nsamples, nstates) and returning a torch tensor of shape (nsamples, 1). If the gradient of the Hamiltonian is not provided, the gradient of this function will be computed by torch and used instead. If hamiltonian_true is provided, hamiltonian_est will be ignored.

grad_hamiltonian_truecallable, default None

The known gradient of the Hamiltonian. Callable taking a tensor input of shape (nsamples, nstates) and returning a tensor of shape (nsamples, nstates).

dissipation_true(N, N) tensor or callable, default None

The known R matrix. Must either be an (nstates, nstates) tensor, or callable taking a tensor input of shape (nsamples, nstates) and returning a tensor of shape (nsamples, nstates, nstates). If dissipation_true is provided, dissipation_est will be ignored.

external_forces_truecallable, default None

The external ports affecting the system. Callable taking two tensors as input, x and t, of shape (nsamples, nstates), (nsamples, 1) respectively, and returning a tensor of shape (nsamples, nstates). If external_forces_true is provided, external_forces_est will be ignored.

hamiltonian_estcallable, default None

Estimator for the Hamiltonian. Takes a tensor of shape (nsamples, nstates) as input, returning a tensor of shape (nsamples, 1).

dissipation_estcallable, default None

Estimator for the R matrix. Takes a tensor of shape (nsamples, nstates) as input, returning a tensor either of shape (nstates, nstates) or of shape (nsamples, nstates, nstates).

external_forces_estcallable

Estimator for the external ports. Takes a tensor of shape (nsamples, nstates) as input, returning a tensor of shape (nsamples, nstates).

training: bool
class phlearn.phnns.PseudoHamiltonianPDENN(nstates, kernel_sizes=[1, 3, 1, 0], skewsymmetric_matrix=None, dissipation_matrix=None, lhs_matrix=None, hamiltonian_true=None, grad_hamiltonian_true=None, dissintegral_true=None, grad_dissintegral_true=None, external_forces_true=None, hamiltonian_est=None, dissintegral_est=None, external_forces_est=None, **kwargs)

Bases: DynamicSystemNN

Implements a pseudo-Hamiltonian neural network abiding to the spatially discretized pseudo-Hamiltonian PDE formulation:

A*dx/dt = S*grad[H(x)] - R*grad[V(x)] + F(x, t, xspatial)

where x is the system state, A is a symmetric matrix, S is a skew-symmetric matrix, R is the symmetric dissipation matrix, H and V are discretized integrals of the system, F is the external force depending on state, time and space.

What is x here is usually u in the literature, and xspatial is x. We use x for the state to be consistent with the ODE case.

It is possible to provide function estimators like neural networks to model H, V and F, as well as A, S and R. All estimators must subclass torch.nn.Module, such that gradients can be recorded with PyTorch.

If either of A, R, S, H, V or F are known, they can be provided and used in favor of estimators. Note that R, H and F must be functions both taking as input and returning tensors, and that the gradients of H(x) must be available through autograd unless the true gradient is provided.

Parameters
nstatesint

Number of system states M after discretization, i.e. the number of spatial discretization points, including only one of the boundaries of the periodic domain.

kernel_sizesList or ndarray of four integers, default [1, 3, 1, 0]

All the integers should be 0 or odd, the first three should be maximum M (nstates), and the last should be 0 or 1.

skewsymmetric_matrix(1, 1, kernel_size[1]) ndarray or callable,

default None Corresponds to band of width kernel_size[1] of the S matrix. If None, S is assumed to be given by S_estimator with kernel size kernel_size[1], i.e. a trainable skew-symmetric Toeplitz matrix with a diagonal band of kernel_size[1], and periodicity imposed.

dissipation_matrix(1, 1, kernel_size[2]) ndarray or callable,

default None Corresponds to band of width kernel_size[2] of the R matrix. If None, R is assumed to be given by A_estimator with kernel size kernel_size[2], i.e. a trainable symmetric Toeplitz matrix with a diagonal band of kernel_size[2], and periodicity imposed.

lhs_matrix(1, 1, kernel_size[0]) ndarray or callable,

default None Corresponds to band of width kernel_size[0] of the A matrix. If None, A is assumed to be given by A_estimator with kernel size kernel_size[0], i.e. a trainable symmetric Toeplitz matrix with a diagonal band of kernel_size[0], and periodicity imposed.

hamiltonian_truecallable, default None

The known Hamiltonian H of the system. Callable taking a torch tensor input of shape (nsamples, nstates) and returning a torch tensor of shape (nsamples, 1). If the gradient of the Hamiltonian is not provided, the gradient of this function will be computed by torch and used instead. If hamiltonian_true is provided, hamiltonian_est will be ignored.

grad_hamiltonian_truecallable, default None

The known gradient of the Hamiltonian. Callable taking a tensor input of shape (nsamples, nstates) and returning a tensor of shape (nsamples, nstates).

dissintegral_truecallable, default None

The known dissiaption integral V of the system. Callable taking a torch tensor input of shape (nsamples, nstates) and returning a torch tensor of shape (nsamples, 1). If the gradient of the integral is not provided, the gradient of this function will be computed by torch and used instead. If dissintegral_true is provided, dissintegral_est will be ignored.

grad_dissintegral_truecallable, default None

The known gradient of the dissipation integral. Callable taking a tensor input of shape (nsamples, nstates) and returning a tensor of shape (nsamples, nstates).

external_forces_truecallable, default None

The external ports affecting the system. Callable taking two or three tensors as input, x, t and optionally xspatial, of shape (nsamples, nstates), (nsamples, 1), (nsamples, nstates), resp. Returning a tensor of shape (nsamples, nstates). If external_forces_true is provided, external_forces_est will be ignored.

hamiltonian_estcallable, default None

Estimator for the Hamiltonian. Takes a tensor of shape (nsamples, nstates) as input, returning a tensor of shape (nsamples, 1).

dissintegral_estcallable, default None

Estimator for the dissipation integral. Takes a tensor of shape (nsamples, nstates) as input, returning a tensor of shape (nsamples, 1).

external_forces_estcallable, default None

Estimator for the external ports. Takes as input from zero to three tensors of shape (nsamples, nstates), (nsamples, 1), (nsamples, nstates), resp. Returns a tensor of shape (nsamples, nstates).

Methods

dV_correction()

Amends the grad of the dissipation integral so that it is zero for the zero solution.

external_forces_correction()

Amends the external force term to correpond with the dissipation term being zero for the zero solution but the full PHNN model is unchanged.

lhs(dxdt)

Applies the A matrix to the left-hand side of the PHNN formulation.

simulate_trajectory(integrator, t_sample[, ...])

Simulate a trajectory using the rhs_model and sample at times t_sample.
dV_correction()

Amends the grad of the dissipation integral so that it is zero for the zero solution.

external_forces_correction()

Amends the external force term to correpond with the dissipation term being zero for the zero solution but the full PHNN model is unchanged.

lhs(dxdt)

Applies the A matrix to the left-hand side of the PHNN formulation.

simulate_trajectory(integrator, t_sample, x0=None, xspatial=None)

Simulate a trajectory using the rhs_model and sample at times t_sample.

Parameters
integratorstr or False

Specifies which solver to use during simulation. If False, the problem is left to scipy’s solve_ivp. If ‘euler’, ‘midpoint’, ‘rk4’ or ‘srk4’ the system is simulated with the forward euler method, the implicit midpoint method, the explicit Runge-Kutta 4 method or a symmetric fourth order Runge-Kutta method, respectively.

t_sample(T,) tensor or ndarray

Times at which the trajectory is sampled.

x0(M,) tensor or ndarray, default None

Initial condition. If None, an initial condition is sampled with the internal sampler.

xspatial(M,) tensor or ndarray, default None

The spatial points. Required if rhs_model depends on these.

Returns
xs(T, M) tensor

The solutions at the time steps given by t_sample

usNone

Placeholder for control output

training: bool
class phlearn.phnns.R_NN(nstates, hidden_dim, diagonal=False)

Bases: BaseNN

Neural network for estimating the parameters of a damping matrix. with three hidden layers, where the first has Tanh-activation, the second has ReLU-activation and the third has linear activation. The network takes system states as input.

When called with a batch input, the network returns a batch of matrices of size (nstates, nstates). All damping parameters are assumed to be positive.

Parameters
nstatesint

Number of states in a potential state input.

hidden_dimint

Dimension of hidden layers.

diagonalbool

If True, only damping coefficients on the diagonal are estimated. If False, all nstates**2 entries in the R matrix are estimated.

training: bool
class phlearn.phnns.R_estimator(state_is_damped, ttype=torch.float32)

Bases: Module

Creates an estimator of a diagonal damping matrix of shape (nstates, nstates), where only a chosen set of states are damped.

Parameters
state_is_dampedlistlike of bools

Listlike of boolean values of length nstates. If state_is_damped[i] is True, a learnable damping parameter is created for state i. If not, the damping of state i is set to zero.

ttypetorch type, default torch.float32

Methods

forward([x])

Returns

get_parameters()

Returns
forward(x=None)
Returns
(N, N) tensor

Damping matrix

get_parameters()
Returns
ndarray

Damping parameters

training: bool
class phlearn.phnns.S_estimator(kernel_size=3, ttype=torch.float32)

Bases: Module

Creates an estimator of a skew-symmetric convolution operator to apply to the integral H.

Parameters
kernel_sizeint
ttypetorch type, default torch.float32

Methods

forward([x])

Returns
forward(x=None)
Returns
(N, N) tensor

Damping matrix

training: bool
class phlearn.phnns.Summation

Bases: Module

Module that performs summation along all dimensions except the batch dimension. It computes the sum of elements across each dimension and keeps the dimensionality intact by using the keepdims=True argument.

Methods

forward(x)

Defines the computation performed at every call.
forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool
phlearn.phnns.batch_data(data, batch_size, shuffle)

Partition data into batches of batch_size.

Parameters
datatuple

((x_start, x_end, t_start, t_end, dt, u), dxdt)

batch_sizeint
shufflebool
Returns
batcheslist of tuples
phlearn.phnns.compute_validation_loss(model, integrator, valdata=None, valdata_batched=None, loss_fn=MSELoss())

Compute the validation loss of model.

Parameters
modelDynamicSystemNN
integratorstr or False

Specifies how to evaluate the right-hand side of the differential equation. If False or ‘euler’ is is evaluated at (x_start, t_start). If ‘midpoint’, ‘rk4’ or ‘srk4’ the right hand side is evaluated such that the corresponding integration scheme is the implicit midpoint method, the classic Runge-Kutta-4 method or a symmetric fourth-order Runge-Kutta method, respectively.

valdatatuple, default None

((x_start, x_end, t_start, t_end, dt, u), dxdt) Either valdata or valdata_batched must be provided.

valdata_batchedlist of tuples, default None

Batched validation data. A list of tuples containing validation data batches. Either valdata or valdata_batched must be provided.

loss_fncallable, default torch.nn.MSELoss()
Returns
vlossfloat
phlearn.phnns.generate_dataset(pH_system, ntrajectories, t_sample, true_derivatives=False, nsamples=None, seed=None, noise_std=0.0, references=None, xspatial=None, ttype=torch.float32)

Generate a dataset consisting of simulated data.

Parameters
pH_systemphlearn.phsystems.PseudoHamiltonianSystem
ntrajectoriesint

Number of trajectories in dataset

t_sample(T, 1) ndarray

Times to sample the trajectories at.

true_derivativesbool, default False

If True, let the returned derivatives dxdt be the true value of the derivatives. If False, estimate derivatives with the finite differences.

nsamplesint, default None

Number of samples to keep. If smaller than ntrajectories*T, the last overflowing samples will be dropped. If None, all samples are kept.

noise_stdnumber, default 0.

Standard deviation of Gaussian white noise added to the samples of the trajectory.

referenceslist of phlearn.control.Reference, default
None

If the pH_system has a controller, a list of ntrajectories reference objects may be passed.

xspatial(1, M) ndarray

If pH_system is a discretized PDE system, this should be the spatial points at which the system is discretized.

ttypetorch type, default torch.float32
Returns
If the system is an ODE:

(x_start, x_end, t_start, t_end, dt, u) : tuple

If the system is a PDE:

(x_start, x_end, t_start, t_end, dt, u, xspatial) : tuple

with

x_start : (ntrajectories * (T-1), N) or (nsamples, N) tensor x_end : (ntrajectories * (T-1), N) or (nsamples, N) tensor t_start : (ntrajectories * (T-1), 1) or (nsamples, 1) tensor t_end : (ntrajectories * (T-1), 1) or (nsamples, 1) tensor dt : (ntrajectories * (T-1), 1) or (nsamples, 1) tensor u : (ntrajectories * (T-1), N) or (nsamples, N) tensor xspatial : (1, M) ndarray, same as input

dxdt(ntrajectories * (T-1), N) or (nsamples, N) tensor
phlearn.phnns.l1_loss_pHnn(pHnn_model, l1_param_forces, l1_param_dissipation, x, t=None, xspatial=None)

Compute L1 penalty loss of external force and dissipation terms:

L1 = l1_param_forces * | f(x, t) | + l1_param_dissipation * | R(x) |
Parameters
pHnn_modelPseudoHamiltonianNN
l1_param_forcesnumber
l1_param_dissipationnumber
x(nsamples, N) tensor
t(nsamples, 1) tensor, default None
Returns
penaltynumber
phlearn.phnns.load_baseline_model(modelpath)

Loads a BaslineNN or a BaselineSplitNN that has been stored using the store_baseline_model().

Parameters
modelpathstr
Returns
modelBaslineNN, BaselineSplitNN
optimizertorch.optim.Adam
metadictdict

Contains information about the model and training details.

phlearn.phnns.load_dynamic_system_model(modelpath)

Loads a DynamicSystemNN from disk. See :py:meth:~`pseudo_hamiltonian_neural_network.load_phnn_model` and :py:meth:~`model.load_baseline_model`.

phlearn.phnns.load_phnn_model(modelpath)

Loads a PseudoHamiltonianNN that has been stored using the store_phnn_model(). Assumes that the Hamiltonian function has either been provided or has been modeled with a HamiltonianNN, that the external force has either been provided or modelled with a ExternalForcesNN, and that the dissipation has either been provided or been modelled with a R_estimator or a R_NN.

Parameters
modelpathstr
Returns
modelPseudoHamiltonianNN
optimizertorch.optim.Adam
metadictdict

Contains information about the model and training details.

phlearn.phnns.npoints_to_ntrajectories_tsample(npoints, tmax, dt)

Compute number of trajectories and the necessary sample time to achieve npoints samples from trajectories lasting tmax time with a dt sample rate.

Parameters
npointsint
tmaxnumber
dtnumber
Returns
int
(npoints, ) ndarray
phlearn.phnns.store_baseline_model(storepath, model, optimizer, **kwargs)

Stores a BaslineNN or a BaselineSplitNN with additional information to disc. The stored model can be read into memory again with load_baseline_model().

Parameters
storepathstr
modelBaselineNN, BaselineSplitNN, PDEBaselineNN
optimizertorch optimizer
* * kwargsdict

Contains additional information about for instance training hyperparameters and loss values.

phlearn.phnns.store_dynamic_system_model(storepath, model, optimizer, **kwargs)

Stores a DynamicSystemNN to disk. See :py:meth:~`port_hamiltonian_neural_network.store_phnn_model` and :py:meth:~`model.store_baseline_model`.

phlearn.phnns.store_phnn_model(storepath, model, optimizer, **kwargs)

Stores a PseudoHamiltonianNN with additional information to disc. The stored model can be read into memory again with load_phnn_model().

Parameters
storepathstr
modelPseudoHamiltonianNN
optimizertorch optimizer
* * kwargsdict

Contains additional information about for instance training hyperparameters and loss values.

phlearn.phnns.train(model, integrator, traindata, optimizer=None, valdata=None, epochs=1, batch_size=1, shuffle=True, l1_param_forces=0.0, l1_param_dissipation=0.0, loss_fn=MSELoss(), batch_size_val=None, verbose=True, early_stopping_patience=None, early_stopping_delta=0.0, return_best=False, store_best=False, store_best_dir=None, modelname=None, trainingdetails={})

Train a :py:meth:~`DynamicSystemNN`.

Parameters
modelDynamicSystemNN
integratorstr or False

Specifies how to evaluate the right-hand side of the differential equation. If False or ‘euler’ is is evaluated at (x_start, t_start). If ‘midpoint’, ‘rk4’ or ‘srk4’ the right hand side is evaluated such that the corresponding integration scheme is the implicit midpoint method, the classic Runge-Kutta-4 method or a symmetric fourth-order Runge-Kutta method, respectively.

traindatatuple

((x_start, x_end, t_start, t_end, dt, u), dxdt)

optimizertorch optimizer
valdatatuple

((x_start, x_end, t_start, t_end, dt, u), dxdt) Validation loss is computed at the end of every epoch. Validation data is required to use early stopping.

epochsint, default 1
batch_sizeint, default 1
shufflebool, delfault False
l1_param_forcesnumber, default 0.

L1 penalty parameter punishing the size of the external force estimates.

l1_param_dissipationnumber, default 0.

L1 penalty parameter punishing the size of the dissipation estimates.

loss_fncallable, default torch.nn.MSELoss()
batch_size_valint, default None

If not provided, all of the validation data is in one batch.

verbosebool, default False

If True, print information about training progress.

early_stopping_patienceint, default None

Patience for early stopping. If None, patience is infinite.

early_stopping_deltanumber, default 0.
return_bestbool, default False

If True, return a copy of the model achieving the lowest validation loss. If False, return the completely trained model.

store_bestbool, default False

If True, store the model every time a new lowest validation loss is achieved. If validation data is not provided, the model is saved after every epoch. If False, no model is saved.

store_best_dirstr, default None

Directory for storing the best model. If None, the models/<timestamp> directory is created and used.

modelnamestr, default None

Directory for storing the best model. If None, the model name is set to <timestamp>.model. If a name is provided, the model will be overwritten.

trainingdetailsdict, default {}

Dictionary of information to store together with the model. The number of epochs trainged, loss and validation loss is added to this dict before saving a model.

phlearn.phnns.train_one_epoch(model, traindata_batched, loss_fn, optimizer, integrator, l1_param_forces, l1_param_dissipation)

Train model for one epoch.

Parameters
modelDynamicSystemNN
traindata_batchedlist of tuples
loss_fncallable
optimizertorch optimizer
integratorstr or False

Specifies how to evaluate the right-hand side of the differential equation. If False or ‘euler’ is is evaluated at (x_start, t_start). If ‘midpoint’, ‘rk4’ or ‘srk4’ the right hand side is evaluated such that the corresponding integration scheme is the implicit midpoint method, the classic Runge-Kutta-4 method or a symmetric fourth-order Runge-Kutta method, respectively.

l1_param_forcesnumber
l1_param_dissipationnumber
Returns
training_lossnumber