When you use the keyboard shortcuts to toggle move/delete, those states are not exclusive. You can enter build mode, toggle move on, and toggle delete on. The game gets confused (not to mention me the user) in this state.
When you use the mouse to toggle move/delete it works as expected and turns off the active build/move/delete state.
Also right-clicking the mouse does not end the move or delete state if you toggled it on with the keyboard shortcut.