Trying to improve the weed Detection function(Using Deep Learing)

Hello, we are a small team of farmbot builders sitting in China.
We are trying to combine image processing and deep learning technology with fasrmbot, and we intend to try to improve the original weed detecition function first, and then proceed to develop crop recognition, growth condition, pest and disease functions one after another after successful testing.
During this period, we encountered some problems:
We are trying to combine image processing and deep learning technology with fasrmbot, and we intend to try to improve the original weed recognition function first, and then proceed to develop crop recognition, growth condition, pest and disease functions one after another after successful testing.
During this period, we encountered some problems.
Since we have never used Elixir, we want to use other languages (e.g. python) to implement communication between web front-end and devices (e.g. cameras), but we are not sure how Farmbot os interacts with web front-end, can you explain the communication flow between them roughly or provide a flowchart? Thanks.
Another question is that currently the photo result of the web page must be refreshed to see, can we get the photo taken in real time and display it on the page, is it related to the rerender mechanism of the React component? Or can we get the video stream from the endoscope camera directly, which will help us a lot to get the training data and do image processing.
Hope to get your replies to help our development work.
Best Regards. :smile:

2 Likes

Hi @SK.LIN ! Software Documentation for this is at Web App | FarmBot Software Development Documentation

My summary : Web App API ( REST-ful JSON ) manages bot Resources for the Account and MQTT channels manage Device Commands, Responses and logs etc.

A Python 3 interface published by the FarmBot Developers also has documentation at FarmBot Software Development | FarmBot Software Development Documentation.

[edit]
The FarmBot Inc.-supplied and supported code for take_photo is this Python 3 code here.

2 Likes

@SK.LIN That sounds like an interesting project. I would take a look at the documentation mentioned in @jsimmonds message above.

The best way to do this is to use a wrapper library like FarmBot.js or FarmBot.Py. Please see the developer docs for more information.

On Chrome and firefox, the photo will eventually show up. It takes some time for post-processing (~30 seconds).

This is currently not possible.

We just spent the Spring Festival holiday, sorry for the late reply.
First of all, thanks all for your replies, which gave us some new ideas. We will try something new next and ask for advice again if we have any questions, thanks again.

1 Like

@SK.LIN Not possible via any FarmBot-supported API currently existing. But is possible if you work closer to Buildroot Linux ( that FBOS runs on ) and the Raspberry Pi hardware :slight_smile: