Trains using branch lines for no reason

Ok, I have had this problem a few times and I am not sure why it occurs.

For all of my stations that are near the main, they point towards the tracks, so a train going to the station goes down the mainline, turns off the main line onto the short branch, stops at the station, goes back down the branch line, and turns back onto the main.

For some reason, trains passing by on the mainline (not headed to that station) turn off the main line, go through the station without stopping, then get back on the main line and keep heading the way they were going before.

Sometimes this happens, sometimes it does not.

I hope this helps!