In this paper, with the aim of estimation of internal dynamic matrix of a discrete linear system, the discrete-time Hamilton-Jacobi-Bellman (HJB) equation for optimal control has been extracted, Heuristic Dynamic Programming algorithm (HDP) for solving it, has been presented and then a neural network approximation for cost function and control input has been extracted to simplify solving of HJB. Design process of the optimal controller shows that, we don't need to know the system matrix. This important issue and convergence of the HDP algorithm to the optimal control policy makes possible the estimation of the internal dynamic matrix.
