Shaping Agriculture’s Future: Reproducing Expert Skills with Automated and Unmanned Combine Harvesters

While global demand for food is rising, the number of people working in agriculture continues to decline each year, and the workforce is aging. The amount of farmland managed per person is growing, increasing the workload for farmers. In order to increase food supply, there is a growing need to streamline and reduce labor needed for agricultural tasks.

“Agriculture must continue.”
With this conviction in mind, Kubota has developed a range of products and services that address the key issues of concern for the future, including global food shortages.

Another of Kubota’s aims is sustainable agriculture that delivers improvements in both crop yield and quality without additional burden on people or the planet. One way to achieve this is agricultural machinery that enables completely unmanned operation for farm tasks. Kubota is working on developing agricultural machinery with a vision toward the future of agriculture.

Tractors, rice transplanters, and combine harvesters are the machines responsible for the basic tasks of plowing, planting, and harvesting. Kubota has already developed tractors and rice transplanters equipped with autonomous self-driving functions. However, there were major challenges in automating combine harvesters, the last step in this process.

Compared to tractors that till soil and rice transplanters that plant seedlings, combine harvesters presented countless challenges to overcome. The first was that the machines would have to distinguish between crops and other obstacles. Second, they would need capabilities to harvest various types of crops. And third, the crops are not always in the same condition – they could be bent or falling over. Making this work unmanned and automated required the incorporation of expert skills into the combine harvester’s movements.

Development continued under the concept of “an unmanned and automated combine harvester that does the same work as expert farmers.” In 2024, the DRH1200A, the world’s first combine harvester to enable automated operation under monitoring by people, was released. With this, Kubota completed its lineup of unmanned and automated tractors, rice transplanters, and combine harvesters, providing an integrated smart agriculture system. How did Kubota achieve the difficult task of unmanned and autonomous operation of combine harvesters? Let us go behind the scenes for some stories about how it was developed.

One of the major challenges in making combine harvesters automatic was coming up with the technology to distinguish between people, crops, and obstacles. When expert farm workers operate machinery, they rely on their past experience to accurately determine the situation in the field. This is why the automated machinery required functions that allow it to make the same judgements as expert farmers.

Distinguishing between crops, people, and obstacles is a particular challenge for combine harvesters. For example, tractors generally work in empty fields, so they can be equipped to distinguish between people and obstacles using sonar and laser sensors that detect positions of objects. On the other hand, when a combine harvester performs harvesting work, it is assumed that there will be crops in the field. They may have millimeter wave radar and laser sensors, but these will also react to crops, making it impossible to accurately detect people and obstacles.

“Is that a crop, a person, or an obstacle?” For combine harvesters to be able to distinguish between objects, the development team realized that conventional sensors would need to be enhanced with detection technologies using AI cameras that can capture images and identify people using image recognition. The goal of this project is to automate the decision-making of expert farm workers.

To identify people in images taken by cameras, it is necessary to train the AI with photo data. However, this also presents unique challenges with combine harvesters. Such learning would require photos that have people standing in fields, but there are very few such samples available. In addition, combine harvesters, unlike tractors and rice transplanters, are operated during harvesting season. Photos of people in fields of fully-grown crops were required.

What is more, for determining details such as the color and type of clothing a person is wearing, “people in a field” data requires an incomparably larger volume of samples than “identifying people walking on the road”. To address this, Kubota went out to fields in various regions and took its own photos with various patterns of people in fields. It would eventually collect a sample size of several million photos over a period of three to four years.

Developers went to fields themselves and gathered data that they applied to their work. This hands-on method is a major part of Kubota’s unique approach to development, with which the company has steadily taken on difficult challenges. And it has now enabled combine harvesters to identify people in fields.

• 
• 

  Sensors are attached to multiple parts of the unit. The unique properties of the AI camera, millimeter-wave radar, and laser sensor are utilized, working together to distinguish between people, crops, and obstacles.

Another challenge for achieving expert-level work efficiency is the work of harvesting. Doing these tasks on par with expert farmers required moving about the field efficiently to harvest, and automatically controlling the machine according to the height of leaning or fallen crops.

Kubota already has a technology called Takumigari (“skilled harvesting”). It allows the combine harvester to design the optimal route for harvesting simply by cutting two or three laps around the perimeter of the field.

The objective of this project was to determine how much of the initial perimeter harvesting, which is usually performed manually, could be automated and unmanned. The major challenge here was what is called “corner cutting”. This is the process of harvesting at an angle in the corners of the field before automated operation begins to create space for the combine harvester to turn.

The process was analyzed in detail to consider the extent to which corner harvesting can be unmanned and automated. The development team made frequent visits to the fields, carefully observing the work of the expert farm workers. They discovered that work efficiency could be improved by observing the height and state of the so-called “ridges” along the fields and having the unit pass over the ridges to perform the corner harvesting.

• 

  An expert farm worker cuts the corners while passing the machine just above the top of the ridge.

Automating this expert work requires accurate detection of the height and state of the ridges. The developers incorporated a map generating feature that employs point cloud analysis using laser sensors to accurately detect perimeter conditions. By expanding the range in which the combine harvester can move when it does its harvesting while it automatically operates as it avoids obstacles and ridges, they succeeded in automating corner harvesting at a level equivalent to seasoned experts. This dramatically improves the automated driving range compared to conventional combine harvesters and makes significant progress in reducing the number of workers and amount of labor required.

• 

  The laser sensor detects the state of the ridges and obstacles. It separates objects by density, allowing it to distinguish between ridges and obstacles.

Another skill that seasoned experts uniquely possess is the ability to harvest according to crop conditions. Crops vary in height depending on type, and if they lean or fall over, their heights may not be consistent. Expert farm workers combine their experience and senses as they adjust their machinery to perform harvesting work according to the state of the crops. Enabling a machine to do this automatically required quantification of crop height detection and machine control according to height, along with exploring patterns of parameters that could cover all possible situations.

• 

  This chart gives an overview of parameter adjustment. Automation requires fine-tuned settings according to the height and condition of crops.

This is why Kubota traveled each year from the south to the north of the Japanese archipelago, tracking harvest seasons while visiting fields all over the country and watching how seasoned experts do their harvesting. To find methods that experts use to make subtle changes that suit conditions such as crop volume, density, and inclination, they collected data on factors including crop height, machine speed, reel positions (forward, reverse, up, down), and rotations. They made numerous hypotheses, assumptions, and verifications to derive parameters that could cover as many scenarios as possible, resulting in automated harvesting for crops with angles of up to 60 degrees.

We believe that this development project fully utilized Kubota’s field-centered approach, in which the developers visit fields to see the challenges for themselves, as well as a working environment in which team members from specialized fields can freely exchange ideas and take on new challenges. In particular, this was Kubota’s first attempt at introducing AI into its agricultural machinery, and our internal knowledge on the subject was not sufficient. However, the open development system that allowed team members who specialize in AI-building and sensors to freely share their views and effortlessly consult with one another helped to realize new initiatives, including image diagnosis using AI.

Kubota is committed to this hands-on, field-based approach to developing products, through which those in charge of R&D for agricultural machinery go to the farms and thoroughly consider the needs the people there have. What is more, Kubota has a culture of determined trial-and-error and of letting people take on uncharted areas, which has enabled it to deliver the world’s first unmanned and automated combine harvester.

With this product, Kubota has achieved unmanned and automated operation of tractors, rice transplanters, and combine harvesters. However, in terms of realizing Kubota’s vision of fully unmanned and, beyond that, sustainable agriculture, it is only halfway there. There are still many obstacles to overcome. From here onward, with its field-centered approach and free and open development system, Kubota will continue to contribute to the future of agriculture.