Our local group has been experiencing a failed to authorize token error for several weeks now (almost daily). Screenshot attached. Any advice would be appreciated.
When you receive that message, does the bot not come back online? when Farmbot fails to reauthorize with the web services, it tries again every 15 seconds. In your case the TCP socket is being
closed. This could be from a number of things but not limited to a network error, a network disconnect, latency to your wifi Accesspoint if you are using wifi etc. I don’t have a way of reproducing that error, but if you just wait it out, farmbot should come back online when it can reach the server again.
Your comment suggesting that this issue is of no interest to my team is unwelcome and untrue.
This issue, along with all support requests are important to us because we want our customers and the DIY community to succeed in using FarmBot.
We respond to most forum posts and emails within 2 business days. This one we responded to in about 5 hours.
However, sometimes we need more time to investigate an issue before responding. Sometimes we don’t have an immediate answer because we need additional information, in which case we’ll ask for it. Other times we sit on the sidelines as other community members discuss and brainstorm on their own, and we chime in as needed or upon request. In some cases the original post doesn’t ask a question and/or it doesn’t appear to be directed at us, but rather the community at-large, in which case we may not be compelled to respond.
In any case, we’re doing our best to help people resolve their issues and we’re learning from bug reports and feature suggestions to make FarmBot better. We’re also constantly balancing our time between providing support and improving everything.
Moving forward I will appreciate you keeping forum posts on-topic and your personal commentary to yourself.
Hey Connor! So really it depends on the day. Sometimes the software will come back in 5 min or less. Other times it won’t come back until a day later. We are using a NETGEAR EX6100-100NAS AC750 WiFi Range Extender with Gigabit Ethernet to increase the wifi signal that the farmot is receiving. We checked this piece of equipment to make sure all was working correctly and it is. There is cat-5 running from the extender to the raspberry pi and it has tested good as well. The token refresh issue has been really annoying since we can’t always depend on it to come back immediately. Our group of volunteers who have helped build / configure the device only gets to meet 1 hour a week after work so this problem has slowed down our progress. Do you think it may be network related on our end or an issue with the farmbot server?
Thanks for the response!
Just a thought to rule out all that may be WiFi related:
Can you run a hard cable directly to the FB? Not as a permanent fix, but rather as a temporary debug step. If you can, and after doing so you no longer see the disconnects, you can dig deeper into WiFi-related causes.
Unfortunately, we don’t have a jack outside in the location where the farmbit is located so that will not be possible…even for testing. I’ve already considered the option and was wanting to use that as a troubleshooting method as you stated.
Ok, then you might try a few things to affect the WiFi connection, just to see if you get better or worse results (to help determine if it is WiFi related).
- just move the WiFi hardware around. Sometimes even a few feet or even just turning the WiFi router can have impact on the quality of the signal.
- try creating a WiFi hotspot with some other device - this might rule out quality of service from your WiFi source/provider.
- run a WiFi monitoring app like Netstumbler (http://www.netstumbler.com/downloads) on a laptop to give you some idea of the WiFi signal strength in the area.
- a really drastic step would be to set up a local MQTT host that you could hardwire right to the FB, but I think that is a lot of effort (but great learning!)
And maybe try another brand or model of WiFi stuff…
And just a silly thought - you said that some days are good and some are bad…
Frequency disruption? (Is the microwave running on the bad days? , or maybe bandwidth being used at a greater rate by other devices (phones, tablets, etc… on the same subnet)
Too much ether in the net?
(The last two are just jokes…)
So this is in a business location. The closest AP is ~75ft away and is traveling through a sheet rock wall and a concrete wall. My laptop shows a strong signal but that doesn’t mean the wireless receiver on the raspberry pi or the extender is receiving a strong signal. I’m filing a request to put in another AP but that may take some time.
If anyone else has seen this issue and has any other ideas, please share
If it’s something on my end with the network then we’ll continue investigating. Does anyone think this could be related to the software hosted on the farmbot servers? Just throwing anything out there.
It’s really hard to know for sure what is causing the issue. If you are interested you take a look at this helpful page about tcp sockets.
The TLDR: it is unlikely to be a client <-> server issue, and likely a network issue, although if you can pinpoint what kind of network issue, we may be able to tweak or expose some settings in FarmbotOS to help mitigate the issue.
For what it’s worth, Range extenders have caused me personally endless problems similar to what you are experiencing and i avoid them at all costs unless completely necessary.
One potentially easy way to trouble shoot is to remove the Raspberry Pi from the electronics enclosure and connect it directly to you router or whatever supplies the Range extender with internet via a cat5 or cat6 ethernet cable. If the problem doesn’t persist you have found that something between the source and pi is causing the issue. If the problem does persist, you’ve eliminated that as an issue and you can look into other more complex things.
@linglelite I have personal experience with seeing TCP sockets become “hung” when wireless is flaky. The physical layer drops out and comes back intermittently (roaming or signal strength) which is undesirable. This causes TCP to attempt management of the exchange by trying to ask for retransmission from the client side or the server retransmitting because it did not receive an acknowledgement. It turns into a train wreck quickly. In my experience, what you’re probably seeing is that the upper level of software let go of the socket but it is still in use. Hard to say specifics. Can you adjust any socket linger options or something similar like this document (link) describes? On Windows it’s a TCP stack adjustment in the registry of I remember but I don’t know other platforms. I used UDP in the past as my go-to because of wireless and had much greater success. I realize that is not an option here. Hopefully this helps a little. I would need to study the Farmbot architecture more so I understand where the endpoints of the socket are. I’m just getting started on this platform!! I hope this helps explain a bit more - tried to add to Connor’s answer a little. If there’s a way to have the socket open and close gracefully and keep the comms short, I’ve found this is the best way to increase robustness on wireless with TCP.
Here’s a link that talks about graceful handling…
This is a very helpful response. For what it’s worth, this particular TCP request is using HTTP, and i’m not sure of the global configs available via the libraries FarmbotOS uses. I will put it on my list to take a look through the documentation. I already set the TCP timeout pretty high to mitigate similar (but not the same) issues relating to TCP timeouts.
Thanks for the responses! I’ll continue to dig further into the issue on my end but I really do appreciate everyone’s help!
Here is another issue we are seeing with the Farmbot. I’m attaching a screenshot of the farmbot log that was seen this morning.
What version is your bot on? I believe i fixed that issue a while back.
Our farmbot actually hasn’t connected since June 16 . Our group plans on meeting tomorrow to investigate more but I can’t pull the farmbot version until it connects. I do know that we pulled the latest version when we made the SD card back in April if that helps.