A Comparative Analysis of Static Feature Extraction Techniques for PE File Malware Detection using LightGBM and Gradient Boosting Variants

Abstract

Efficiency of traditional signature-based detection has been severely diminished due to the exponential growth of sophisticated malware. For this reason, the cybersecurity industry has pivoted toward Machine Learning (ML) solutions. This paper presents
a comprehensive comparative analysis of machine learning models for static malware detection using Windows Portable Executable (PE) files. By evaluating the efficiency of feature extraction methodologies, specifically targeting structural metadata, byte histograms, and import tables, we demonstrate the limitations of deep learning architectures, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), which often suffer from high computational overhead and the curse of dimensionality. In contrast, Gradient Boosted Decision Trees (GBDTs) such as XGBoost, CatBoost, and LightGBM establish a definitive performance upper bound for tabular PE metadata. Our findings emphasize that, when combined with optimized feature engineering, LightGBM achieves a superior speed to-accuracy ratio, making it an ideal candidate for real-time, resource
constrained endpoint deployment.