Detecting a panic behavior within a human crowd is of a high importance as it allows to prevent disasters. Online analysisof video streams, real-time processing and accurate detection are required to ensure effective surveillance of the crowdedplaces. However, these requirements are not simultaneously fulfilled by the existing techniques. Rapid advances in artificialintelligence are investing the power for automatic public surveillance and timely detection of a possible abnormal behavior.Thus, the aim of the present work is to propose an online, real-time and effective technique for panic behavior detection. Itrelies on a handcrafted feature that accounts for the characteristics of the crowd tounderstandpeople behaviors and a longshort-term memory neural network to predict future feature values. Experiments are performed on well-known datasets ofpanic situations to evaluate the performance and accuracy of the proposed algorithm. Results show the system yields excellentperformances in terms of accuracy and processing time with respect to the state of the art techniques.