Events stop after message "Detecting available UARTs["ttyACM0","ttyAMA0"] and “Success: Firmware flashed. MD5: F118B218F87”

Running events stop with these logs:

Success: Firmware flashed. MD5: F118B218F87 SEP 6, 3:00PM
Detecting available UARTs[“ttyACM0”,“ttyAMA0”]

This happens every time an event is running (taking pictures, usually).

We are running our Farmbot Genes 1.5 XL in a private network. We just recently updated both FBOS and
WebApp.

Any idea?

Thanks

@ladiazgarcia Does the bot also go off-line to the Web App at these moments ?
With the info you’ve given, either

  • the Genesis bot is re-starting FBOS ( the Raspberry Pi is re-booting )

or

  • the USB connection between the Raspberry Pi and the Farmduino PCB is not secure

Can you check these ?

In case of FBOS re-starting, it’s recommended to get a known good SHDC card and flash the latest (v14.3.2) FBOS image onto it and then re-configure.

Thanks for your quick response @jsimmonds, the problem I’m referring to happens randomly within an event so it is hard to know if the bot goes off-line at these specific moments. I’ll schedul a few more events to try to see what’s going on. About the options you are given:

  1. the bot is restarting because of an issue with the SHDC card. Re: I’m using the card that came with the robot. I also tried a new one, and the problem persists.
  2. About the connection between the Raspberry and the Farmduino. It might be… How do I check that?

Additional info about my setup: I’m running the bot in a local network; the router I’m using has an ethernet cable connected to my server running the WebApp locally. The bot is connected via wifi to the router. We are using the router also to provide internet to other devices. Could this be affecting the bot? Today I’m going to try disconnecting the internet cable from the router, and see what happens. Worst case scenario would be to buy another router just for the robot.

Thanks again,

Luis

We just ran the following test:

  1. disconnected the internet cable from the router, so we have an “isolated” set up.
  2. run a sequence with 400 plants
  3. at some point, connectivity appears lost. in the connection diagram, a red line appears between the message broker and the farmbot, and grey line between the raspberry pi and the farmduino. See image below for reference.
  4. bot stops working (motors stop, lights off, etc.)
  5. log message: “Farmbot disconnected”
  6. connectiivity is recovered
  7. log message: “Detecting available UARTs[“ttyACM0”,“ttyAMA0”]”
  8. log message: “firmaware flashed…” and connectivity between farmduino and raspberry pi appears in red
  9. everything goes to normal, however, the sequence was not completed.

Our router could be faulty, that’s a possbility. Any ideas before investing in a new one?

Thanks,

L

@ladiazgarcia From that picture it is likely the issue relates to Raspberry Pi power. Notice the Yellow disk next to Voltaje : ok ?
This indicates a low voltage event has occurred. ( see pop-up text when you press ‘(?)’ )

More experienced forum members may have better suggestions, but meantime, my advice is to find a person familiar with electronics, cables and power supplies who could diagnose the issue with the 5VDC supply for the Raspberry Pi model 3B which comes over USB from the Farmduino v1.5 PCB ( all powered by the 24VDC 150W power adapter ).

edit

The Farmbot Software package components ( Web App and FBOS on the bot ) have no capability to continue or re-start any Farm Event that was interrupted ( unfinished ). Such event is simply abandoned.
Events due but not started15s before the Farmbot OS re-starts and all future Events should be executed normally though.

Thanks @jsimmonds, it seems that the problem might be the power. We have been running short sequences (taken pictures on 100 plants) and monitoring the voltage indicator. In general, there are a lot of power fluctuations here so we have been using a voltate regulator. In one of the sequences we ran, we saw that right before the farmbot went offline, the voltage disc changed from green to yellow, then yellow to red. I guess this could cause the raspberry to restart and therfore interrupt the sequence. However, we have seen other sequences in which the voltage disk is yellow or red without causing bot restarting.

In my mind, the fact that these reboots happen only when running something makes sense given that e.g. motors requiere “additional” power when excecuting a sequence. Under low voltage, this would cause a reboot.

more updates:

We ran a sequence to photograph 100 plants, and attached a voltmeter to the robot. Voltage goes between 4.7 and 4.8. Sometimes it can go below; however, these low voltages did not cause the problem I’m refering above. Almost at the end of the sequence, when voltage was aparently average (~4.7-4.8), the bot disconnected and the logs commented above appeared.

Power adapter is 24V, as expected.

Any idea?

@ladiazgarcia

Next actions to take might be

  • Replace the USB A to Micro B (power) cable between the Farmduino and the Raspberry Pi 3 with a good quality charge cable.

  • Replace the Farmduino v1.5 PCB

edit

To get better detail and understanding of the quality of the 5VDC ( over USB ) supplied to the RPi3B, I own and can recommend the Joulescope instrument.

@ladiazgarcia Unfortunately I can’t provide support on this issue if your device is on a private network.

If you are familiar with SSH, you can:

  • Re-flash FBOS
  • Paste your SSH public key into the id_rsa.pub field of the “Advanced” network settings in configurator.
  • SSH into the device.
  • Run RingLogger.attach() to see device logs.
  • Attempt to reproduce the problem.
  • Determine if the device produces any useful logs within SSH to see what happened.

Again, we only provide technical support for users on the main public server (I am able to remotely inspect those devices) and all other support is provided on a “best effort” basis only.

1 Like

I just re flashed the ssd card and configured the bot to connect to my.farm.bot. You should be able to see it now. We are running a test taking pictures of 600 plants. Let’s see if the problem appears. It is taking more time tho, but I guess it is normal because our internet connection is not very good. One interesting we noticed is that last time I reflashed the ssd card with the newest OS, and under a private set up, the motors were not making any noise… After this last reflash, noise came back.

What could we do in the meantime? Just wait for the error to appear?

Best,

L

1 Like

Great, I am remotely connected to your device now. Thanks.

Are you a researcher / industrial user? We may have a new feature of interest if you need to take that many photos. Please let me know.

I am watching your logs now.

The settings don’t carry over between servers. Most likely, you just need to enable SILENT MODE in the device settings page.

Yes. Additionally, some bugs will be reported to our backend error reporter, which I will be able to view remotely.

If will continue taking a look at the device but am not doing anything at the moment since the device looks like it is already trying to reproduce the issue (no errors yet).

If you have any ideas on ways to make the problem appear again, please let me know. I will let you know if I see anything.

1 Like

Excellent.

I’m a researcher, I work in the intersection of genomics and phenomics. The idea is to connect a plant phenotype (e.g. plant growth rate) to DNA variation. I usually use plant collections n>300 and multiple reps, so we are talking about 1000-2000 plants. I’m interested in things like leaf number, biomass, plant growth rate, etc. The Farmbot is an excellent tool for this purpose, I might just need a couple of tweaks. The private setup is ideal for me since we don’t have a reliable internet connection here, plus we can have almost unlimited storage space. Also, it gives me the possibility to modify the 1000 plants limit. We are not using all the Farmbot capabilities, so far we are using just the camera, lightning, and watering system.

1 Like

That’s very interesting @ladiazgarcia . I have more information to share with you privately, since it is beyond the scope of this bug report.

2 Likes

After the first sequence without error, I scheduled a new one last night, in which I was able to see the messages described above… At 9:57pm (CT), I got the log: “Detecting available UARTs: [“ttyACM0”, “ttyAMA0”]”, and almost immediately after that, there was this message: “Success: Firmware flashed. MD5: F118B218F87”. A minute later, the lights, as well as other peripherals (which are not in use), went off. Strangely, the bot continued taking a few more pictures after that, around 10 or so. Out of the 600 plants scheduled to be photographed, only 477 were completed.

When this happened in our private set up, no more pictures were taken after the “Detecting available UARTs…” message.