Abstract: 　　The excavation load of top pipe machines is a crucial parameter in construction projects and has a significant impact on construction safety. This study aims to improve the accuracy of load prediction by constructing an enhanced Transformer model based on the temporal features of load data and domain knowledge. The proposed model incorporates geomechanical constraints and power balance constraints into the loss function of the Transformer model, enabling better learning of internal data features. A dataset was constructed using field cutting test data to build the prediction model. Multiple experiments were conducted to validate the model’s predictive performance. The results demonstrate that the proposed model achieves smaller loss values on both training and validation datasets, with predicted values closely matching actual measurements. Comparative experiments and ablation studies show that compared to three classical machine learning models and two submodels, the proposed model yields the smallest Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), indicating higher prediction accuracy and better ability to capture load data variation patterns. The method described in this paper enables advance prediction of loads during top pipe construction, playing a crucial role in ensuring high-quality tunnel construction.