From Simulation to Autonomous Mustering

The Challenge

This was not a routine feature expansion. It was genuine R&D: the team had to take livestock-herding algorithms developed in a simulation environment and determine whether they could work in real paddocks, with real animals, real terrain, and real flight hardware. The goal was not a staged demo in a perfectly controlled environment. It was to test whether autonomous livestock movement was technically achievable under practical field conditions.

The starting point was an existing drone training simulation with a cattle model. The next step required expanding that simulated world into a physical, autonomous cattle muster using consumer-grade, off-the-shelf DJI hardware. A simulation can control its terrain, animal behaviour, sensors, and timing. A paddock cannot. Cattle respond to pressure and movement in variable ways, terrain changes constantly, and environmental conditions affect what the drone can see and how it can fly.

The system also had to operate inside constraints imposed by a large third-party technology stack. DJI hardware, flight controls, sensors, the DJI Cloud API, communications, and field operations all had to work together. The autonomy pipeline needed to interpret the world, plan movement, control the drone, and recover useful data from each trial without assuming that any external component would behave exactly like the simulation.

Livestock welfare and stockmanship added another constraint. Effective mustering is not simply moving animals from one location to another. A drone applies pressure from behind to encourage movement, then reduces pressure or moves ahead to slow and settle a mob. Animal behaviour, terrain, environmental conditions, timing, and human judgement all influence the result. Automation had to support calm, useful livestock movement rather than treat cattle as interchangeable objects in a pathfinding problem.

The wider operating environment introduced further external constraints. Trials had to be meaningful across south-east and north-west Queensland, while autonomous flight and agricultural drone operations remained subject to practical safety and regulatory requirements. The challenge was to make progress without overstating what the system could do: prove a useful capability under defined conditions, learn from the limits, and build a foundation for further development.

The Solution

Actuator Digital adapted the existing simulation models to match observed DJI drone behaviour, then built the supporting autonomy systems around that real-world reference:

• Livestock perception: Trained a custom YOLO detection system to identify cattle in the drone’s camera view, providing the perception layer needed to reason about a moving herd.
• Terrain-aware planning: Implemented RRT pathfinding that could account for the terrain around the livestock instead of relying on the simplified assumptions of a simulation.
• Third-party integration: Connected the autonomy systems to DJI hardware and the DJI Cloud API, working within the capabilities and constraints of an off-the-shelf platform rather than a bespoke research vehicle.
• Autonomous control: Integrated AI virtual-stick control so the system could direct the drone during autonomous mustering while responding to its observed flight behaviour.
• System communication: Used MQTT to connect the relevant services and move information through the autonomy pipeline.
• Flight replay and data tools: Built tools to capture, replay, and inspect flights so the team could investigate behaviour, compare trials, and iterate on the models and control logic.

This created a loop between simulation, perception, path planning, flight control, and field data. Each trial provided evidence about where the model matched reality, where third-party systems imposed limits, and what needed to change before the next test.

The work was deliberately iterative. Rather than treating the simulation as a finished answer, Actuator used it as a starting point for testing assumptions against observed DJI behaviour, real livestock movement, and the complexity of operating in paddocks.

The Outcome

Watch Sky Kelpie’s autonomous mustering demonstration:

The trials demonstrated a major step forward: livestock-herding algorithms developed in simulation were transferred into practical field testing. Across trials in south-east and north-west Queensland, Sky Kelpie demonstrated autonomous livestock movement in real paddock conditions, showing that the capability was technically achievable under certain conditions. The broader trials showed the platform could locate a mob, drive it in the required direction, and settle it in the paddock.

The result was not a claim that automation replaces experienced stockpeople, nor that every paddock can be automated in the same way. It was stronger and more useful than that: a working proof that simulation, computer vision, path planning, third-party drone systems, and real-world stockmanship principles could be brought together in an autonomous livestock-mustering system.

The work gave Sky Kelpie a foundation for broader repeatable on-farm tasks, including livestock headcounts, pasture and biomass analysis, infrastructure and water monitoring, fence-line checks, and pest identification. Those are future opportunities rather than outcomes claimed as delivered by this project, but they show why the R&D matters beyond a single muster.