Hey Alric! This is a well-known behavior, although the cause cna be manifold. For me, the issue was an incorrectly made connector on the encoder cable (pin positions in the connector did not match pins positions on the board).
I recommend you read through this forum post and see if pin-position checking is right for you 