Direct, local control via the Raspberry Pi

Hi,
I’ve just completed initial construction of a new Genesis 1.5, and we are starting to dig into the software side of things for our project. (We are mounting sensors on the FarmBot to do automated plant monitoring.) I was hoping we could throw a python script onto the raspberry pi on the bot to implement our own logic, and connect with other software. Or run it on another machine locally and hit an API on the Raspberry Pi. Maybe something like:

bot = farmbot.connect(, credentials)
bot.move_to(1400, 200, 100)
bot.lights_on()
img = bot.take_photo()
bot.home()

For our application we need to write an automation script that will control the FarmBot as well as other hardware and software in sync. (For example, we might scan around taking imagery, analyze those images to determine a strategy, and then move around an arbitrary path.) I’d like to just run a python script on another machine on the same network and have it send messages to the pi on the FarmBot. That, or run that script directly on the farmbot raspberry pi. We don’t expect to have internet access for our device, and we need to run this script on a clock to gather periodic data.

I see that FarmWare is deprecated, so I’m wondering how best to control the FarmBot? Can we just connect directly to the Raspberry Pi and send it messages? If not, we are happy to implement this functionality and contribute anything helpful we create. Would this be of interest? Initially it can be a very basic control interface that doesn’t use any of the concepts from the FarmBot web application like plants or groups. (This is an API to enable custom use cases and novel inventions based on the FarmBot platform, not a replacement or change to the web app, which has its own target use cases.) Just a basic REST API to move to coordinates and toggle controls for lights, vacuum, etc. This basic level of control from any script on the local network seems to satisfy many of the needs I see people post about who want to write their own software, and it might actually save a lot of headaches of trying to shoehorn in these capabilities from the web app’s perspective.

Would people be open to this? If so, can you point me to the code that currently receives messages off the cloud message bus to execute commands? Then I can identify the key functions to call, and figure out how best to wrap them in a tiny REST API. Or maybe there is already a REST API sitting on the PI that I should add to? Or maybe none of this is necessary, and there is a better way? Please, share your thoughts!

Thanks for your help, and thanks for creating such a great platform!

Hi,

Thanks for reaching out! We want to see more developers building things for the FarmBot platform, and software development questions are always welcome. In case you haven’t seen it yet, we offer developer documentation that covers the basics of software development for FarmBot devices. The documentation was written as a guide (not a reference), so it is best to read it from start-to-finish. If something still does not make sense, please let us know- I am happy to update the docs and provide explanations.

The good news is that we will be publishing an update to the sequence editor so that users can run arbitrary Lua code on the device, even if it is offline (assuming they run the code via a timed farm event).

The initial version will have all of the basics, such as movement and pin control. We can add more available functions to the Lua API if you need them- please make a feature request after the initial release if that is the case. We anticipate the release will go out soon, and it is my top priority at the moment. The release is less than a few weeks away, possibly even sooner.

If you find that Lua snippets are too limited for your use case, and you absolutely cannot keep your device on the internet, the next best option would be to self-host a FarmBot Web App instance within your LAN, ideally behind a firewall. You could then run your custom controller software inside the LAN, possibly on the same machine that runs the Web App.

Instructions on how to run a self-hosted server can be found here.

We don’t expect to have internet access for our device

Although FarmBot can tolerate extended internet outages, it’s important to note that FarmBot was not designed to run entirely off-grid. Numerous design choices were made under this assumption. We don’t know how FBOS will behave if the WiFi is turned off for more than a day. Again, it is possible to run a FarmBot on intermittent internet connectivity. Still, we have not built any features to support fully offline setups, nor do we test against that scenario when releasing new features. I’m not necessarily saying that it is impossible to run a FarmBot entirely offline, but it is not a design goal for FarmBot OS in 2020. Building a syncing, offline-first architecture, unfortunately, requires more developer resources than our 2 person development team can handle at the moment.

we need to run this script on a clock to gather periodic data.

A FarmEvent that triggers a sequence that executes Lua code may fit your needs. Please let us know if there are specific features you need within the Lua sandbox. Feature requests are welcome.

I see that FarmWare is deprecated, so I’m wondering how best to control the FarmBot?

There are a few options:

  • The next release of FBOS will offer a Lua command block that will allow arbitrary code execution. This will be the easiest path forward.
  • We built the Web App interface using FarmBotJS. If a task can be done in the Web App, it is possible to do it in FarmBotJS.
  • Once we finish releasing the Lua feature, I will begin work on a Python wrapper. It will offer the same feature set as FarmBotJS but in a Python package. Paired with a local Web App instance, this might meet your needs.

We are happy to implement this functionality and contribute anything helpful we create. Would this be of interest? Initially, it can be a fundamental control interface that doesn’t use any of the FarmBot web application concepts like plants or groups.

can you point me to the code currently receiving messages off the cloud message bus to execute commands?

All RPC commands enter FarmBot OS (via MQTT / AMQP) here. The codebase is primarily Elixir. FarmBot OS is a mature codebase with 7 years of history, so it will take time to learn the system. I am happy to answer specific questions if you have any.

Thanks for your help, and thanks for creating such a great platform!

Thanks, and I hope that helps! Please let me know if you have further questions.

Hi Rick,
Thanks for your reply! I guess it feels to me like this is a much heavier weight approach than we really need. At some layer in the system the the raspberry pi is just sending hardware control commands to the arduino to move the gantry, or control other devices. Is that basic functionality exposed somewhere? We don’t really need or want any of the other web app functionality, sequences, alerts, or anything. We just want a control interface for the hardware so that we can then run with it and pursue our own goals. Is there a core interface layer that we could use, and then just turn off the rest?

Thanks,
Jeff

@rosejn

If you just want to directly control the Farmduino, you could absolutely unplug the Farmduino from the RPi and send GCode over the wire. The GCode spec is fairly simple and I am happy to help you along the way. It sounds like FarmBot’s hardware meets the needs of your project but the software was not designed for the environment you are using it in. If this is a correct assumption, perhaps you would be better served by replacing FBOS entirely, but keeping the Farmduino firmware? This is easy to do. Are you trying to use FarmBot in a non-agricultural setting, Ie. industrial automation?

The hardware can definitely be controlled directly with GCode using custom software of your own creation. This might be the best option, since FarmBot OS was not designed to handle the use case you need.

Conversly, if you can find a way to add a mobile hot spot, run an ethernet line, purchase a WiFi extender, etc… there are more options available. The amount of effort required to get offline-only operation to work in FBOS will be orders of magnitude more than the level of effort required to find a reliable internet connection.

Here are some examples that might meet your needs (assuming you do not want to just direct control the Farmduino via serial cable):

Would any of these work? Please let me know.

OK, it seems like a big jump to go from the standard software setup to creating a new gcode based library and custom OS image. This is application software that autoboots running on a typical Raspberry Pi linux distro, isn’t it? Or is Farmbot OS really a customized OS because it isn’t a typical RPi? Before digging in to do that, I’m curious if there might be a middle ground that enables this use case for everyone without needing a different codebase? Many people seem to just want to control their farmbot with their own code, and this would dramatically simplify things for those use cases.

It seems like this would require two things: improved offline logic for the existing software (e.g. retry backoff logic), and a locally hosted json api for basic control of the farmbot. Could we not leverage the existing gcode interface library and just wrap with some http endpoints to expose on the local network? Does that seem reasonable/feasible to you? Why the emphasis on cloud access?

UPDATE: The Lua functionality shipped today, 2 FEB 2021, as part of FBOS 12.3.4

@rosejn

This is application software that autoboots running on a typical Raspberry Pi Linux distro, isn’t it?

No, it is not. It is a fully-fledged embedded Linux system, similar to what you might find in the OS of a TV set-top box or an automobile control system. This is not a desktop application.

Many hobbyists that like to use the Raspberry Pi will build applications on top of Raspberry Pi OS and then configure Systemd or Init to boot their app at startup time. This is not what FarmBot OS does, nor is it typical of commercial embedded Linux systems.

is Farmbot OS really a customized OS because it isn’t a typical RPi?

It is a custom Linux kernel, compiled from source via buildroot, with a custom init system, a different bootloader and a partition layout that is completely different than Raspberry Pi OS. Many of the things you would find in a desktop Linux distro are not there. Simply put, FarmBot OS is not a flavor of Raspberry Pi OS, and it may not resemble what many Raspberry Pi hobbyists would see on their own projects. You can learn more by exploring the SD card layout and the source code for FarmBot OS.

Many people seem to want to control their FarmBot with their own code, which would dramatically simplify things for those use cases.

You can control your device, though, and I’ve shown you how with the examples in previous posts. The device has an MQTT RPC system, a fully built Javascript wrapper library, and soon, a Lua editor that can be interacted with from the browser without the need to install software or host servers (scheduled for release in a few days- switch your device to the “alpha” update channel if you want to take a sneak peek). We also have numerous example snippets on how to accomplish remote control of the device. The device needs to be online, though. None of the alternatives will be less work than improving your internet connectivity. We have decided to follow a client/server model rather than a peer-to-peer model for particular reasons.

It seems like this would require two things: improved offline logic for the existing software (e.g., retry backoff logic) and a locally hosted JSON API for basic control of the FarmBot.

This question would take more time than I have to respond to, but the simple answer is that is “the edge cases are deceptively complicated.” This becomes even more apparent when faced with the reality that most users are _ not_ software developers.

I would encourage you to take a look at the codebase and ask questions as you go. As you learn the codebase more, things will make more sense. This is the advantage of Open Source software. I’m happy to answer questions as you ask them.

2 Likes

OK, thanks Rick! Clearly I have lots to learn. I’m curious why it is necessary to use a custom linux distro and kernel build for FBOS? Is this because of real-time constraints? If we need to put together our own solution because we can’t rely on internet then will we also need to build off of this custom linux system? (This is a hard requirement for us because we have specific target installations that we don’t control where no internet access is available, and many others where internet access is spotty while we need very reliable, periodic sampling over the plant bed)

I’m really wondering why the resistance to a local API in addition to the message queue interface? I would imagine that in Elixir we could expose the low-level operations of the farmbot via a JSON API in a single file that just takes in args and calls the same functions you use now to emit gcode to the farmduino. Would that not be an easier way to enable this kind of thing rather than requiring a custom OS build? I guess I was hoping that we could contribute something of use to the community, rather than just go build our own playground.

I totally appreciate that you want to have this be as user friendly as possible for most users who are not devs, and that is a great goal. It just seems like it would be ideal if there was also the hacker access point so that we could easily connect and control the bot from our own random scripts without requiring cloud access.

If we just needed to run some basic commands the Lua script mechanism might help, but we need to gather tons of data using a depth sensing camera and a spectrometer mounted on the farmbot, stitch imagery and spectra, analyze with neural nets, maybe take some more samples, adjust as needed. This will require running on a separate machine (that will be sitting next to the farmbot), which can do the heavy lifting and store many gigs of data, as well as have a GPU to use for the deep learning.

Thanks a lot for your help and explanations.

why it is necessary to use a custom linux distro and kernel build for FBOS? Is this because of real-time constraints?

FarmBot does not have real-time constraints. In our case, we need a build system that is easily customized and can be reproducibly built on any machine, and reliably generate build images that will work for customers without intervention. Back when FarmBot was in its “garage prototype” phase, we did in fact build on top of Raspbian like you are suggesting. The build process was lengthy and hard to reproduce. The performance was dismal (FarmBot Express did not exist yet, but I imagine that would have made things even worse). It was uncommon for a device to operate for more than a few weeks in real-world conditions, for a variety of reasons. We switched to Nerves, which uses BuildRoot under the hood. Memory usage went down, production image sizes went down, SD card corruption became far less common, and the amount of time we needed to waste hunting down build issues went down to nearly 0. It was very different than desktop Linux, but these are the tradeoffs that must be made for embedded systems.

The vast majority of commercial IoT projects are using either Buildroot or Yocto. There is a reason for this and I am unaware of any commercial products that ship software updates to their customers running desktop Linux distros.

If we need to put together our own solution because we can’t rely on the internet then will we also need to build off of this custom Linux system?

If you don’t care about using the web UI at all, and you really just want to use Raspberry Pi OS, the answer is “no”, you do not need to use FarmBot OS at all. You can flash Raspberry Pi OS onto an SD card and then directly talk to the Farmduino via /dev/ttyACM0. It is relatively simple to write a GCode parser- mostly just string manipulation. Any programming language that can split a string into tokens will work. You can even find the old serial port adapter I wrote in Ruby if you are curious how such a script might be written. Some of the GCodes have changed, but they haven’t changed that much. Controlling a FarmBot is just a matter of sending and receiving GCode over a serial line.

FarmBot, from a hardware perspective, is just a raspberry pi plugged into an Arduino-like microcontroller. You could flash Raspberry Pi OS onto your SD Card and then write a script to handle the test from /dev/ttyUSB0. It’s all ASCII based- there is no magic binary protocol to learn. You could even run a cable from the Farmduino to your laptop and control the device from a terminal emulator if you wanted to.

I would imagine that in Elixir we could expose the low-level operations of the FarmBot via a JSON API in a single file that just takes in args and calls the same functions you use now to emit GCode to the Farmduino.

There are a lot of things that seem “easy enough” to do. Unfortunately, adding new developer APIs implies that we will support them for a very long time. If we make the decision to create a new set of RPC APIs, we have essentially doubled the cost of ownership for our RPC system (again, we only have one FBOS maintainer- me). People will begin to write critical systems on top of the new APIs, and we are then in a position where we must support these users or jeopardize system stability. A good example is the Farmware system- we specifically said that it was beta only and could be changed without notice. When the system had scalability problems, or we needed to make changes to the system because of unrelated environmental issues, people would become very disappointed with that, regardless of the fact that we provided numerous disclaimers about the stability.

The API you are suggesting is not impossible, but it does not fit into our long-term roadmap, and creating new developer APIs is not a tax you pay once at the time of development. It requires an extreme amount of forethought and constant maintenance as the system evolves. We learned this the hard way with the Farmware subsystem and won’t repeat that mistake. We simply don’t have the resources to say “yes” to every feature request that is suggested. I hope you can understand that.

If you still wanted to implement the API yourself, the file can be found here. This is the HTTP server that the configurator exposes and it would be a good starting point if you are truly interested in contributing code to FBOS. By all means, do feel free to fork FarmBot OS. That is the beauty of Open Source. I can’t teach you how to write Elixir code (great resource here if you are new to the language), but I’d be happy to answer questions you have about environment setup.

Would that not be an easier way to enable this kind of thing rather than requiring a custom OS build? I guess I was hoping that we could contribute something of use to the community, rather than just go build our own playground.

The best solution would be to add additions to the Lua sandbox. Lua has long term support.

It just seems like it would be ideal if there was also the hacker access point so that we could easily connect and control the bot from our own random scripts without requiring cloud access.

Again, your best bet would be to either fork FBOS to meet your needs or write a script that runs on Raspberry Pi OS and which writes directly to the Farmduino’s serial bus.

Thanks a lot for your help and explanations.

No problem! Happy to answer any questions you have, regardless of which solution you chose moving forward. Please let me know if you need help along the way, either here or via PM. This helps us find places that the developer docs need updating.

2 Likes