Looks like i’ve got it. Now the tools still don’t fit to the picture, but the positioning from bot coordinates to the picture coordinates seem to match. The reason that the tools are not matching is, that they are on a different height level than the soil, so you could either calibrate the camera on the tool grid or the soil bed. Calibrating with the soil bed makes obviously more sense 
Next thing is, that the camera reacts really sensitive to sunlight, or light generally. As you can see, plants are mostly just white and its really hard to configure the weed detector so can detect with any weather.
So I tried to change some settings in OpenCV using a modified version of the Take-Photo farmware.
The difference is not that great, but the weed detector works better for me now. Those pictures were taken only a few minues after the upper ones in this post.
I’m still not completely fine with the results, so I will mount a LED strip to the gantry and try to make photos at night. Sadly I’ve already changed my electronics to 24V but a stepdown module will probably save the day 
I’m just writing this, to keep my experience with cheap USB cams documented 
.