Variational formulation of physics-informed neural networks

Chinmay Katke, C. Nadir Kaplan

Abstract

Physics-informed neural networks (PINN) solve differential equations by minimizing a phenomenological loss function derived from these equations. However, higher-order derivatives in the differential equations describing many physical systems lead to higher computational costs. Additionally, solving coupled differential equations with PINN is complex due to manually or algorithmically determined ad hoc weight factors appearing in the loss function. We propose a variational PINN (vfPINN) algorithm that optimizes the functionals in integral form (e.g., Lagrangian, Hamiltonian, or Rayleighian) to address these issues. vfPINN naturally uses lower-order derivatives and replaces ad hoc weight factors with rigorous physical scales. Our simulations using vfPINN show promising results for benchmark systems like steady state Sine-Gordon equation and other ordinary differential equations (ODEs). We also explore the solution stability using the notion of conjugate points which are defined by a lack of positive definiteness in the second variation of the functional around the desired solution. Our ongoing research extends to locating conjugate points numerically in the domain and finding the corresponding unstable solutions using saddle point search algorithm that we developed for linear and non-linear ODEs.