Blinking Light around E-Stop Button

Troubleshooting
1

Currently our Farmbot is in a state where the ring around the E-Stop button is flashing

[Uploading: Photo Jun 12, 10 16 09 AM.jpg…]

I cannot find anything in the documentation that describes this state. See:

Our Farmbot is essentially unresponsive. It will not display the Configurator. It does not display the Farmbot WiFi. It does not respond to the Web application.

This condition persists even after power cycling.

What is this state? How do I clear / reset from this state? Why is it not documented?

-B

2

Hi @bcatanzaro

I will take a look at your device now. For future reference, if you purchased your kit from FarmBot, Inc. you can also receive support by contacting contact@farmbot.io (please have your order number ready).

I will have more details available in a moment.

3

@bcatanzaro

Your SD card is most likely corrupted. The only remedy is to re-flash your SD card from scratch using Etcher and re-run the WiFi configurator. All of your data will be backed up on our servers and there is no risk of data loss.

Once you have re-configurated WiFi and FarmBot is at least able to connect to WiFi, please let me know and I can continue investigating the problem.

Thanks,

Rick

4

Thanks Rick. FYI - we updated to the latest FarmBot/OS. So it was re-flashed recently. We’ll do that again.

We were having trouble with what appeared to be a flaky WiFi signal. It also takes the system a very long time to boot up. We’ll keep track of the boot-up time and conditions after re-flashing the SD card and let you know.

Can you point me to a URL that will give me an introduction to lower level diagnostics. I have some skill in this area, but am not familiar with the FarmBot/OS. I’d like to get better informed so I can perform a better diagnosis.

Thanks!

5

@bcatanzaro I am very interested to hear about your setup, as this will help us assist customers with similar problems.

What version were you on previously? There was a bug in versions prior to 10.1.0 that would sometimes corrupt SD cards when upgrading. If your SD card is corrupted again, please let me know as this would be a sign that the fix is not working.

This is probably correct. I looked at the historic data for your account and your WiFi levels were < 50% for most of the day. It is unlikely that a device can operate with that level of connectivity. I would recommend switching to Ethernet or buying a WiFi range extender before attempting to fix any other problems. The WiFi module on a Raspberry Pi is not as good as those used in laptops. Furthermore, the underlying protocols used by FarmBot require a steady connection.

How long did it take? Typically an Express bot can boot up in about 4 minutes and a Genesis bot in under 2 minutes. Hopefully once the WiFi signal improves you will not need to reboot so often.

To answer your question, the blinking light means “Firmware not installed” or “Firmware not detected”. As you have mentioned, we need to update our docs to reflect that. If you know the IP address of your device and are on the same LAN as the FarmBot, you can visit http://1.2.3.4/logger in a web browser to see low level logs (replace 1.2.3.4 with the IP address of your device). On top of the other documents you linked to in the original post, you may wish to take a look at the diagnostic codes in the connectivity panel which will be crucial for fixing the weak WiFi signal.

Please let me know if there’s anything else we can do to help with your setup. Have a great weekend!

6

We flashed: farmbot-rpi3-10.1.0

We tried this a few weeks ago and ran into quite a bit of trouble dealing with the Configurator.

We use an AirTies mesh extender to extend the WiFi to boost the signal. It is physically between the Farmbot and the router. It is about 20 ft from the router and 10 ft from the Farmbot (in a straight line). Candidly, the AirTies is a bit flaky at times. So, it is now on a timer to reboot (x1) per night at 4:30 AM. Will that cause the Farmbot to get upset?

I timed the boot sequence. It was quite fast now:

  • 1 min to boot from power-up to get the farmbot WiFi to Configurator
  • 1.5 min to re-boot and connect with a PC to the web application

There are a few diagnostics I’d like to track

  1. How do I get the IP address (with x4 kids + x2 adults there are a lot of IP addresses on our LAN)?
  2. Will the RPi respond to a ping?

I’m trying to figure out if I need to run CAT-5 to the device (or to a WAP rather than a mesh extender). So I want to see logs that will help me understand the connectivity of the Farmbot to my network.

Is there a network management tool you use for monitoring traffic?

Thanks!

-B

P.S. The image below shows connectivity. Sync is coming in/out every few seconds (Synced and Sync Unknown) At what frequency (seconds, minutes, …) will connectivity paralyze the Farmbot? For example, if it drops (x1)/hour, I’d expect it to recover; (x1)/min it may not.

image

P.P.S. Got lucky… Fing found the Farmbot. http://IPADDR/Logger is empty

7

Here’s our current status from a web browser (Firefox) on a PC.

image
image1173×667 59.5 KB

My wife has an iPad logged into the same URL and her’s shows Debug events and mine does not. Are these settings in cookies on the browser? How else would her interface show something different than mine?

  • OS was flashed at 1:15 PM
  • Don’t know why we’re seeing an E-stopped event at 1:22 PM
  • My wife unlocked it at 1:29 PM
  • Don’t know the source of the errors at 1:30 PM

Controls are inactive now

image
image664×522 19.4 KB

It is responding to pings

image

Red button is solid on and Green light is blinking
image

Logger does have lots of messages…

-B

8

@bcatanzaro

This is the second-to-latest version. We recommend always running the latest version unless there is a good reason. Furthermore, we don’t backport bug fixes to old versions- only the newest version receives bug fixes.

Occasional network resets are fine. Problems start when the network connectivity drops below 65% sporadically (more than once per minute).

We provide this information in the device settings area. You can open the menu by clicking the FBOS version:
image

It seems like you’ve found the answer to this question already, but for those finding this thread via web search: “yes” FarmBot will respond to pings.

After looking at your account, I think you do need to switch to Ethernet. I am skeptical that you will be able to get any amount of gardening done with the current WiFi network. We have an admin backend that tracks WiFi strength over time and your signal seems to dip very low, sometimes as low as 20%- this is not adequate to support FBOS.

I would not get too bogged down in those details. Right now it is quite obvious that the problem is the WiFi. If you run a firewall on your LAN, we do provide a document for IT professionals to assist with unblocking of ports. That does not seem to be the issue with your account, however.

This problem will go away when you get your device onto a stronger network connection, assuming there is no firewall interfering (does not seem to be the case).

It’s a bit complicated to explain, but essentially debug level logs are ephemeral. They are not stored on the API. If you do not have a good websocket connection, you will not be able to see them. We use cookies to keep users logged in to the app, but not for log storage. All of that happens at the API layer.

Probably her device has a better connection than yours, or your device is not able to keep the websocket connection open.

This is probably because FarmBot’s network cut out again. I see this happen a lot on weak connections, such as WiFi APs that have heavy interference or installations which have obstruction from the AP.

I imagine you are seeing a lot of :nxdomain and :econnrefused errors. This occurs when WiFi is not strong enough to support a real-time connection to the message broker.

I hope this helps! Again, I would highly urge you to address your network connectivity issues before trying anything else. We deal with a lot of customer support requests on a daily basis and unfortunately, low connectivity is the most common issue faced. I am happy to help you with setup, but I do not think we will have success until we can get the WiFi signal strength to stay above 65% at all times.

1 Like
9

Thanks for all the answers…they all make sense to me.

It may takes us a bit of effort to run an outdoor CAT-5. In the interim:

If we configure sequences while connected by CAT-5 temporarily, will the Farmbot faithfully execute these after we disconnect the CAT-5, while the WiFi is flaky?

-B

10

Connected CAT-5 to RPi port. It appears good…

image
image800×230 27.4 KB

However, I just lost all pings to the Farmbot. The AirTies responds to a ping, so the network reaches the AirTies mesh extender, but now the Farmbot is not responding anymore.

  • Red button has a solid red light on
  • Green LED next to the Red button is flashing very quickly

I will download the latest firmware tomorrow and re-flash the SD-Card.

image

image

-B

11

Definitely this is the Best thing to do next :slightly_smiling_face:

What’s at the other end of the Cat 5 wire ?

12

The AirTies mesh extender (like many WiFi extenders) has an RJ-45 connection as well as extending the radio.

So the connection is from the Farmbot -> CAT-5 -> AirTies -> WiFi -> Router -> Internet

Since I can ping the AirTies, I know the connection from AirTies -> WiFi -> Router works.

13

Yes, but :slight_smile:
this image of your logs suggests that you’re still on wireless !

https://forum.farmbot.org/uploads/default/original/2X/6/6531965407a3d5a552883c3d90d9fca3db304e87.png

Do a Factory Reset and choose wired :slight_smile:

1 Like
14

It is now completely wired and the OS has been re-flashed to 10.1.2. Connectivity remains an issue.

  • Rebooted fine
  • Connected to wired network using Configurator
  • Red light around red button is lit, solid
  • Green light is solid
  • Controlled watering and moving from the my.farm.bot web interface
  • After fewer than 10 min, lost connectivity (see attached image)
  • Red light around red button is lit, solid
  • Green light is flashing
  • AirTies device is functioning. Tested by connecting both Farmbot and another device (laptop) into RJ-45 ports. Lights on RJ-45 connector indicate traffic to/from Farmbot, yet web application is unable to Sync with Farmbot. The grey cable is connected to the Farmbot. The black cable is connected to the laptop

Currently, we are unable to get the Configurator to come back up. We have re-flashed the O/S and re-booted. We are unable to connect or change the network configuration.

  • Red light around red button is lit, solid
  • Green light is off

image
image782×569 49.9 KB

image
image740×744 180 KB

15

Try a new SDHC 32GB (class 10 will work) . . do write-verification when you flash FBOS onto the SD
I’m a bit old-school so I also do a sha256sum check on farmbot-rpi3-10.1.2.img before flashing.

16

@bcatanzaro I am back in the office and can take a deeper look. It looks like your bot is offline, though.

  • If the device was intentionally unplugged, please plug it back in and notify me when I am able to resume troubleshooting. I cannot troubleshoot until the device is back online.
  • If the device went offline for unknown reasons, please send me a screenshot of the /logger endpoint. I would also advise that you perform a hard reset afterwards (flash the SD card from Etcher) if this is the case.

I will pause troubleshooting until the device is back online.

17

We’re in a pretty bad state. Not sure how to get the device back on-line.

  • Keep losing connection even w/hardwired CAT-5
  • Cycled the power in hopes of triggering the Configurator to go to hotspot to reconfigure back to radio / WiFi. Configurator did not come up.
  • Re-flashed with the latest OS (10.1.2) and that did not cause the Configurator to engage.

We don’t know what to do at this point. We ordered a new SD card with the (vain?) hope that it was corrupted beyond repair with re-flashing.

  1. How do we trigger the Configurator?
  2. How do we reset the state of the RPi (and Farmduino) to factory?

I know Linux, C, C++, C#, G Code, Python, … I build sophisticated instruments for a living (see my LinkedIn invite I sent to you). I’m not afraid of a deeper dive for diagnostics / reformat / reset if that is what is required.

image
image1040×225 23.1 KB

18

@bcatanzaro Do not purchase or replace any equipment yet. This could very well be a simple fix that does not require any money to solve. We will figure this out but it is going to take time to isolate. The device is exhibiting signs of a network issue that still needs to be isolated. It looks similar to customers with network firewall issues, or a device that is simply too far away from the access point.

If the network is still unstable, can you try unscrewing the Pi from the electronics box and bringing it inside? You should be able to re-configurate the device without electronics attached (although you will get a Code 30 error, since there is no firmware to talk to). Let’s try to get the WiFi extender out of the picture entirely to make sure there are no other problems.

Is it safe for me to assume you do not run any network security software such as firewalls or content filtering?

19

Thanks for the encouragement!

Nothing too fancy on our network, but more than the average household. No special firewall, port blocking or port forwarding, or devices outside the DMZ. Mostly standard configured, password protected router & radios (WiFi):

  • NetGear consumer router running DHCP and (x2) radios (2.4 GHz / 5.0 GHz)
  • Switches (x4) all over the house to feed many wired devices
  • NetGear consumer router configured as WAP (x1) with a separate WiFi with (x2) radios (2.4 GHz / 5.0 GHz)
  • AirTies Mesh Extender to boost the radio signal near the Farmbot.

Options:

  1. Disconnect RPi as you suggest. Bring it to the switch closest to the router and ???
  2. Run cable from RPi in Farmbot to an ad hoc switch. Run that switch to the router.

If we go with (1) as you suggest, what do you want us to do once we connect the RPi to the switch?

-B

1 Like
20

I’d like you to leave it plugged in until I have time to take a look (I can remote in to your device which gives me a very detailed picture of what is going on).

This might be a good way to get more things done in one swoop, but I worry that the switch could complicate the network setup. If it is easy to run a cable that far, we can try it. The benefit would be the ability to debug firmware issues if any are found. Plugging the Pi directly into your main router will be good enough for now and won’t add an extra point of failure. The problems you have are entirely network related, based on the logs I am seeing.