After some more testing it appears the issue was with the chiseled blocks on the far left and far right exterior front corners. I copied and pasted those 2 corner blocks in a creative world to inspect them and see no visual holes or gaps. I am thinking I used the tools from the mod "QP's Chisel tools" on those 2 blocks in particular and in combination with "chiseled blocked retention" somehow broke or corrupted these 2 blocks, it was also in a multiplayer world with high ping so that could have played a role.