These papers by M. Flannery discuss the issues that arise if we try to couple nonholonomic constraints to the Lagrangian. Flannery shows that it is not possible to define a variational principle in such cases, but d’Alembert’s principle (or generalizations of it) still holds.
M. Flannery, The enigma of nonholonomic constraints, American Journal of Physics 73, 265 (2005)
M. Flannery, The elusive d’Alembert-Lagrange dynamics of nonholonomic systems, American Journal of Physics 79, 932 (2011)
M. Flannery, D’Alembert–Lagrange analytical dynamics for nonholonomic systems, Journal of Mathematical Physics 52, 032705 (2011)
J. Douglas, Solution of the inverse problem in the calculus of variations, Trans. Amer. Math. Soc. 50 (1941), 71-128)
These papers discuss variational principles that are able to deal with nonconservative forces:
C. Galley, Classical Mechanics of Nonconservative Systems, Physical Review Letters 110, 174301 (2013)
C. Galley, D. Tsang and L. C. Stein, The Principle of Stationary Nonconservative Action for Classical Mechanics and Field Theories, arXiv:1412.3084 (2014)
D. H. Kobe, Derivation of Maxwell’s equations from the gauge invariance of classical mechanics, American Journal of Physics 48, 348 (1980)
Here are presentations and research papers discussing the falling cat problem (by increasing complexity):
R. Mehta, Mathematics of the Falling Cat
T. R. Kane and M. P. Scher, A Dynamical Explanation of the Falling Cat Phenomenon, Int. J. Solid Structures 5, 663 (1969)
H. Essén and A. Nordmark, A Simple Model for the Falling Cat Problem, Eur. J. Phys. 39, 035004 (2018)
R. Montgomery, Gauge Theory of the Falling Cat, Fields Communications 1, 193 (1993)