Predicting building energy consumption needs to be anticipated to save building energy and effectively control the predictions. This study depicted the target building as a physical model to improve the learning performance in a data-scarce environment and proposed a model that uses simulation results as the input for a data-driven model. Case studies were conducted with different quantities of data. The proposed hybrid method proposed in this study showed a higher prediction accuracy showing a cvRMSE of 22.8% and an MAE of 6.1% than using the conventional data-driven method and satisfying the tolerance criteria of ASHRAE Guideline 14 in all the test cases.