A neurodynamic model for solving nonsmooth constrained optimization problems with affine and bound constraints

Document Type : Original Article

Authors

1 Department of Mathematics, Chabahar Maritime University. Chabahar, Iran

2 Department of Mathematics, Chabahar Maritime University, Chabahar, Iran.

3 Chabahar Maritime University

Abstract

In this paper, a non-penalty differential inclusion-based model is proposed for solving nonsmooth convex optimization problems subject to linear equality and bound constraints. We prove the convergence of the trajectories to the equality feasible region in finite time. Also, the equivalence of the equilibrium point of the proposed neural network and the optimal solution of the original optimization problem is shown. In addition, we prove the stability of the proposed neural network in the sense of Lyapunov and the global convergence to an exact optimal solution to the original problem. In comparison with some existing models for solving nonsmooth convex optimization problems, there does not exist any penalty parameter or penalty function in the model's structure and the implementation of the proposed model is easier. In the end, as an application, the proposed neural network is reduced to a model for solving nonsmooth convex optimization problems subject to linear equality and $x\geq0$ constraints. Also, some illustrative examples are given to show the effectiveness of the proposed neural networks.

Keywords


[1] Ahmad I, Kaur A, Sharma MK, Robust duality for generalized convex nonsmooth vector programs with uncertain data in constraints, RAIRO-Oper. Res. 55(4), 2181–2188 (2021). doi:10.1051/ro/2021095.
 
[2] Antoniou A, Lu WS, Practical optimization, Springer, New York, USA (2007).
 
[3] Aubin J, Cellina A, Differential inclusions, Springer-Verlag, Berlin, Germany, (1984).
 
[4] Bazaraa MS, Sherali HD, Shetty CM, Nonlinear programming: theory and algorithms, Wiley, New York, USA (1993).
 
[5] Beliakov G, Bagirov A, Non-smooth optimization methods for computation of the conditional value-at-risk and portfolio optimization, Optimization, 55(5-6), 459–479 (2006).
 
[6] Bertsekas D, Constrained optimization and Lagrange multiplier methods, Academic, New York, USA (1982).
 
[7] Bian W, Xue X, Neural network for solving constrained convex optimization problems with global attractivity, IEEE Transactions on Circuits and Systems I: Regular Papers, 60(3), 710–723 (2013).
 
[8] Chen D, Cao X, Li S, A multi-constrained zeroing neural network for timedependent nonlinear optimization with application to mobile robot tracking control, Neurocomputing, 460, 331–344 (2021). doi:10.1016/j.neucom.2021.06.089
 
[9] Cheng L, Hou ZG, Homma N, Tan M, Gupta MM, Solving convex optimization problems using recurrent neural networks in finite time, in: International Joint Conference on Neural Networks, IJCNN2009, Atlanta, Georgia, USA, 538–543 (2009).
 
[10] Cheng L, Hou ZG, Lin Y, Tan M, Zhang WC, Wu FX, Recurrent neural network for non-smooth convex optimization problems with application to the identification of genetic regulatory networks, IEEE Transactions on Neural Networks, 22(5), 714–726 (2011).
 
[11] Chua L, Lin GN, Nonlinear programming without computation, IEEE Transactions on Circuits and Systems, 31(2), 182–188 (1984).
 
[12] Clarke F, Optimization and nonsmooth analysis, Wiley, New York, USA, (1969).
 
[13] Dai J, Yang X, Xiao L, Jia L, Li Y, ZNN with Fuzzy Adaptive Activation Functions and Its Application to Time-Varying Linear Matrix Equation, IEEE Transactions on Industrial Informatics, (2022) doi: 10.1109/TII.2021.3099819.
 
[14] Ebadi M. J., Hosseini A., Hosseini M. M., A projection type steepest descent neural network for solving a class of nonsmooth optimization problems, Neurocomputing, 235, (2017), 164–181.
 
[15] Ebadi M. J., Hosseini M. M., Karbassi S. M., An efficient one-layer recurrent neural network for solving a class of nonsmooth pseudoconvex optimization problems, Journal of Theoretical and Applied Information Technology, 96(7) (2018) 1999-2014.
 
[16] Ebadi M. J., Hosseini A., Jafari H., An efficient one-layer recurrent neural network for solving a class of nonsmooth optimization problems, Journal of New Researches in Mathematics, 6(24), 97–110
(2020).http://journals.srbiau.ac.ir/article_15615_f34599f523793828ae53dca49834f495.pdf
 
[17] Ebadi M.J., Jafari H., (2021) Solving a Class of Optimal Control Problems by Using Chebyshev Polynomials and Recurrent Neural Networks. In: Allahviranloo T., Salahshour S., Arica N. (eds) Progress in Intelligent Decision Science. IDS 2020. Advances in Intelligent Systems and Computing, vol 1301. Springer, Cham. https://doi.org/10.1007/978-3-030-66501-2_15
 
[18] Effati S, Baymani M, A new nonlinear neural network for solving convex nonlinear programming problems, Applied Mathematics and Computation, 168(2), 1370–1379 (2005).
 
[19] Effati S, Mansoori A, Eshaghnezhad M, An efficient projection neural network for solving bilinear programming problems, Neurocomputing, 168, 1188-1197 (2015).
 
[20] Eshaghnezhad M, Effati S, Mansoori A, A neurodynamic model to solve nonlinear pseudo-monotone projection equation and its applications, IEEE transactions on cybernetics, 47(10), 3050–3062 (2017).
 
[21] Forti M, Nistri P, Quincampoix M, Generalized neural network for nonsmooth non-linear programming problems, IEEE Transactions on Circuits and Systems I: Regular Papers, 51(9),1741–1754 (2004).
 
[22] Gao X, A novel neural network for nonlinear convex programming, IEEE Transactions on Neural Networks, 15(3), 613–621 (2004).
 
[23] Guo Z, Liu Q, Wang J, A one-layer recurrent neural network for pseudoconvex optimization subject to linear equality constraints, IEEE Trans. Neural Networks, 22, 1892–1900 (2011).
 
[24] Heydarpoor, F, Abbasi, E., Ebadi, M. J., Karbassi, M. M. Solving multi-objective functions for cancer treatment by using Metaheuristic Algorithms, International Journal of Combinatorial Optimization Problems and Informatics, 11(3), 61-75.
 
[25] Heydarpoor, F., Abbasi, E., Ebadi, M. J., Karbassi, M. M. Solving optimal control problem of cancer treatment by artificial neural networks, International Journal of Interactive Multimedia and Artificial Intelligence, 6(4) (2020): 18-25. doi: 10.9781/ijimai.2020.11.011.
 
[26] Hosseini A, Hosseini SM, Soleimani-damaneh M, A differential inclusion-based approach for solving nonsmooth convex optimization problems, Optimization, 62(9), 1203–1226 (2013).
 
[27] Hosseini A, A non-penalty recurrent neural network for solving a class of constrained optimization problems, Neural Networks, 73, 10–25 (2016).
 
[28] Hu X, Zhang B, A new recurrent neural network for solving convex quadratic programming problems with an application to the k-winners-take-all problem, IEEE Transactions on Neural Networks, 20(4), 654–664 (2009).
 
[29] Jamali N., Sadegheih A., Lotfi M. M., Lincoln C. Wood, Ebadi M. J., Estimating the Depth of Anesthesia During the Induction by a Novel Adaptive Neuro-Fuzzy Inference System: A Case Study, Neural Processing Letters 53, 131–175 (2021). https://doi.org/10.1007/s11063-020-10369-7.
 
[30] Jiang X, Qin S, Xue X, Liu X, A second-order accelerated neurodynamic approach for distributed convex optimization, Neural Networks, 146, 161–173 (2022). doi:10.1016/j.neunet.2021.11.013.
 
[31] Ju X, Hu D, Li C, He X, Feng G, A Novel Fixed-Time Converging Neurodynamic Approach to Mixed Variational Inequalities and Applications, IEEE Transactions on Cybernetics, (2021). doi:10.1109/TCYB.2021.3093076.
 
[32] Kennedy M, Chua L, Unifying the Tank and Hopfield linear programming Circuit and the Canonical Nonlinear Programming circuit of Chua and Lin, IEEE Transactions on Circuits and Systems, 34(2), 210–214 (1987).
 
[33] Kennedy M, Chua L, Neural networks for nonlinear programming, IEEE Transactions on Circuits and Systems, 35(5), 554–562 (1988).
 
[34] Li W, Bian W, Smoothing neural network for regularized optimization problem with general convex constraints, Neural Networks, 143, 678–689 (2021). doi:10.1016/j.neunet.2021.08.001.
 
[35] Li G, Song S, Wu C, Du Z, A neural network model for nonsmooth optimization over a compact convex subset, In: Lecture Notes in Computer Science, Vol. 3971, Springer-Verlag, Berlin, Germany, 344–349 (2006).
 
[36] Liang Xb, Wang J, A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints, IEEE Transactions on Neural Networks, 11(6), 1251–1262 (2000).
 
[37] Liu N, Wang J, Qin S, A one-layer recurrent neural network for nonsmooth pseudoconvex optimization with quasiconvex inequality and affine equality constraints, Neural Networks, 147, 1–9 (2022). doi:10.1016/j.neunet.2021.12.001.
 
[38] Liu SB, Wang J, A simplified dual neural network for quadratic programming with its KWTA application, IEEE Transactions on Neural Networks, 17(6), 1500–1510 (2006).
 
[39] Liu Q, Wang J, A recurrent neural network for nonsmooth convex programming subject to linear equality and bound constraints, ICONIP 2006 LNCS, Vol. 4223, Springer, Heidelberg, Germany, 1004–1013 (2006).
 
[40] Liu Q, Wang J, A one-layer recurrent neural network with a discontinuous hardlimiting activation function for quadratic programming, IEEE Transactions on Neural Networks, 19(4), 558–570 (2008).
 
[41] Liu Q, Wang J, A one-layer recurrent neural network for non-smooth convex optimization subject to linear equality constraints, In: Lecture Notes in Computer Science, Vol. 5507, Springer-Verlag, Berlin, Germany, 1003–1010 (2009).
 
[42] Liu Q, Wang J, Finite-time convergent recurrent neural network with a hardlimiting activation function for constrained optimization with piecewise-linear objective functions, IEEE Transactions on Neural Networks, 22(4), 601–613 (2011).
 
[43] Liu Q, Wang J, A one-layer recurrent neural network for constrained nonsmooth optimization, IEEE Trans. Systems, Man, and Cybernetics Part B, 41, 1323–1333 (2011).
 
[44] Liu Q, Guo Z, Wang J, A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for dynamic portfolio optimization, Neural Networks, 26, 99–109 (2012).
 
[45] Liu Q, Wang J, A one-layer projection neural network for nonsmooth optimization subject to linear equalities and bound constraints, IEEE Transactions on Neural Networks and Learning Systems, 24(5), 812–824 (2013).
 
[46] Liu Q, Yang S, Wang J, A collective neurodynamic approach to distributed constrained optimization, IEEE Transactions on Neural Networks and Learning Systems, 28(8), 1747–1758 (2017.)
 
[47] Mordukhovich BS, Variational analysis and generalized differentiation, I: basic theory, Springer, New York, USA, (2006a).
 
[48] Mordukhovich BS, Variational analysis and generalized differentiation, II: applications, Springer, New York, USA, (2006b).
 
[49] Nazemi AR, A dynamic system model for solving convex nonlinear optimization problems, Communications in Nonlinear Science and Numerical Simulation, 17(4), 1696–1705 (2012).
 
[50] Qin S, Xue X, A two-layer recurrent neural network for nonsmooth convex optimization problems, IEEE transactions on neural networks and learning systems, 26(6), 1149–1160 (2015).
 
[51] Soleimani-damaneh M, Nonsmooth optimization using Mordukhovich’s subdifferential, SIAM Journal on Control and Optimization, 48(5), 3403–3432 (2010).
 
[52] Tank D, Hopfield J, Simple neural optimization networks: An a/d converter, signal decision circuit, and a linear programming circuit, IEEE Transactions on Circuits and Systems, 33(5), 533–554 (1986).
 
[53] Wang J, A deterministic annealing neural network for convex programming, Neural Networks, 7(4), 629–641 (1994).
 
[54] Wang J, Hu Q, Jiang D, A Lagrangian network for kinematic control of redundant robot manipulators, IEEE Transactions on Neural Networks, 10(5), 1123–1132 (1999).
 
[55] Wei F, Chen G, Wang W, Finite-time stabilization of memristor-based inertial neural networks with time-varying delays combined with interval matrix method, Knowledge-Based Systems, 230, 107395 (2021). doi:10.1016/j.knosys.2021.107395.
 
[56] Xia Y, A new neural network for solving linear and quadratic programming problems, IEEE Transactions on Neural Networks, 7(6), 1544–1548 (1996).
 
[57] Xia Y, Wang J, On the stability of globally projected dynamical systems, Journal of Optimization Theory and Applications, 106(1), 129–150 (2000).
 
[58] Xia Y, Leung H, Wang J, A projection neural network and its application to constrained optimization problems, IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 49(4), 447–458 (2002).
 
[59] Xia Y, Wang J, A recurrent neural network for nonlinear convex optimization subject to nonlinear inequality constraints, IEEE Transactions on Circuits and Systems I: Regular Papers, 51(7), 1385–1394 (2004).
 
[60] Xia Y, Wang J, A recurrent neural network for solving nonlinear convex programs subject to linear constraints, IEEE Transactions on Neural Networks, 16(2), 379–386 (2005).
 
[61] Yang Y, Cao Y, A new neural network for solving nonlinear convex programs with linear constraints, Neurocomputing, 74(17), 3079–3083 (2011).
 
[62] Yashtini M, Malek A, A discrete-time neural network for solving nonlinear convex problems with hybrid constraints, Applied Mathematics and Computation, 195(2), 576–584 (2008).
 
[63] Zhang Z, Xiao T, Qin X, Fly visual evolutionary neural network solving largescale global optimization, International Journal of Intelligent Systems. 36, 6680–6712 (2021). doi:10.1002/int.22564.
 
[64] Zhang S, Constantinides AG, Lagrange programming neural networks, IEEE Transactions on Circuits and Systems-II: Analog and Digital Signal Processing, 39(7), 441–452 (1992).