So I had a customer that had internet drops. They were sharing an internet connection bridged off another fortigate. They had traffic shaping enabled and tldr the max bandwidth setting was causing dropped bytes.
To resolve this I had to set the inbandwidth and outbandwidth of the WAN interface to the correct value.
conf system interface
set inbandwidth 10240
set outbandwidth 10240
After seeing that and putting the max traffic shaper bandwidth to a higher value e.g. 20480, the dropped bytes issue stopped happening.