Development and adaptation of computer vision and machine learning methods for solving precision agriculture problems using unmanned aerial systems (AP14869972)

The project is aimed at solving monitoring problems to support precision agriculture technologies. During the project implementation, computer vision and machine learning methods will be developed to solve the problems of classifying and identifying images, including multispectral ones, obtained using UAVs. Tasks to be solved: classification of weeds, identification of stresses of useful plants. During the implementation of the project, the factors hindering the development of useful plants will be recreated, large sets of labeled images will be created and models will be trained, including deep neural networks, with the help of which the above tasks will be solved.

Objective of the project

To develop and adapt computer vision and machine learning methods for solving precision agriculture problems by processing data and images obtained using unmanned aerial systems.

Project tasks

  1. Development of a multifunctional software and hardware system for the collection and preprocessing of images and data in order to solve the problems of precision agriculture;
  2. Formation of an experimental site for the reproduction of negative factors affecting the processes of precision agriculture;
  3. Formation of data sets for solving problems of precision agriculture using machine learning and UAVs;
  4. Development of methods for identifying and classifying negative factors in order to estimate their influence on the development of useful plants;
  5. Experimental evaluation of the developed methods and reporting.

The main objects of research and experiments are shown in the figure.

 

Results

  1. A prototype of a multifunctional software and hardware system for collecting and preprocessing images and data has been developed to solve precision farming problems. The prototype includes an unmanned aerial platform with gimbal elements, including a precision positioning system with a payload capacity of up to 1 kg and software for processing images obtained using this platform. The software includes, in varying degrees of readiness, functions for reading data (loading images of different spectral ranges), functions for preprocessing images and calculating spectral indices, functions for training machine learning models, and map generation functions. Software was developed to create ultra-high resolution maps of agricultural fields (4K) and software that creates the shortest route for a group of drones suitable for automatic flight over agricultural fields.
  • The video shows photo combining for:
  • 100 photos from one soybean field flyover. Combining parameters: volume – 860Mb, execution time – 15 seconds, peak RAM usage – 3.36Gb
  • 395 photos from the second flyover of the soybean field. Combining parameters: volume – 3.8Gb, execution time – 75.88 seconds, peak RAM usage – 7.11Gb.
4. Five experimental sites with a total area of ​​1260 m2 (each 252 m2)  implementing various negative factors affecting the processes of precision farming were created as a result of completing the task.
Sugar beets of Abulkhair hybrid variety are planted on the sites.
   The first site provides the best conditions for crop growth;
   The second area contains a deficiency of fertilizers;
   The third plot implements no weed control;
   The fourth section is experiencing moisture deficiency (irrigation);
   Fifth section. contains all three mentioned negative factors.

5. More than 500 photographs of sugar beet and soybean fields were marked as a result of the task, . Weeds and beneficial crops (sugar beets, soybeans) are individually highlighted in each photo.
There are 10 species of Weeds – Amaránthus retrofléxus, Convolvulus arvensis, Setaria glauca, Xanthium strumarium, Cirsium arvense, Echinochloa crusgalli, Hibiscus trionum, Phragmites australis, Abutilon theophrasti, Cuscuta, and two useful crops – Beet (Beta vulgaris), Soybean (Glycine max) ).

6. A prototype method for identifying several types of weeds has been developed. Work is underway to improve the accuracy of the developed methods and their experimental testing.
A method for recognizing beneficial and weed plants in soybean and sugar beet fields was developed as a part of solving problem of identifying and classifying negative factors affecting the development of beneficial plants. Yolo v7 neural network was trained.
Recognition accuracy for soybean field: Glycine max (soybean) – 87%, Amaranthus retroflexus (breadweed) – 77%, Convolvulus arvensis (convolvulus) – 89%, Setaria glauca (bristleweed) – 88%, Xanthium strumarium (Cocklebur)  – 88%, Cirsium arvense (sow thistle) ) – 94%, Hibiscus trionum (hibiscus) – 86%, Abutilon theophrasti (ropetail) – 82%. Recognition accuracy for sugar beet field: (Beta vulgaris (sugar beet) – 90%, Convolvulus arvensis (convolvulus) – 75%, Hibiscus trionum (hibiscus) – 74%, Phragmites australis (reed) – 77%, Abutilon theophrasti (cableweed) – 85%, Cuscuta (dodder) – 71%.

Publications

  1. Mukhamediev, R.; Amirgaliyev, Y.; Kuchin, Y.; Aubakirov, M.; Terekhov, A.; Merembayev, T.; Yelis, M.; Zaitceva, E.; Levashenko, V.; Popova, Y.; Symagulov, A.; Tabynbayeva, L. Operational Mapping of Salinization Areas in Agricultural Fields Using Machine Learning Models Based on Low-Altitude Multispectral Images. Drones 2023, 7, 357. https://doi.org/10.3390/drones7060357(Scopus: Q1, 75%, WoS: Q2, IF:4.8)
  2. Zaitseva, E., Levashenko, V., Mukhamediev, R., Brinzei, N., Kovalenko, A., & Symagulov, A. (2023). Review of Reliability Assessment Methods of Drone Swarm (Fleet) and a New Importance Evaluation Based Method of Drone Swarm Structure Analysis. Mathematics, 11(11), 2551. https://www.mdpi.com/2227-7390/11/11/2551
  3. Mukhamediev R. I. Yakunin, K., Aubakirov, M., Assanov, I., Kuchin, Y., Symagulov, A., Levashenko V., Zatceva E., Sokolov D., Amirgaliyev, Y. . Coverage path planning optimization of heterogeneous UAVs group for precision agriculture //IEEE Access. – 2023. – Т. 11. – №. 15. – С. 5789-5803, doi: 10.1109/ACCESS.2023.3235207, https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10011226 (Scopus Quartile: Q1, 90%, JCR Category Quartile: Q2, WoS IF=3.476)
  4. Zaitseva, E., Levashenko, V., Brinzei, N., Kovalenko, A., Yelis, M., Gopejenko, V., & Mukhamediev, R. (2023). Reliability Assessment of UAV Fleets. In Emerging Networking in the Digital Transformation Age: Approaches, Protocols, Platforms, Best Practices, and Energy Efficiency(pp. 335-357). Cham: Springer Nature Switzerland. https://www.researchgate.net/profile/Elena-Zaitseva-2/publication/369409989_Reliability_Assessment_of_UAV_Fleets/links/641d5db6a1b72772e42293f8/Reliability-Assessment-of-UAV-Fleets.pdf
  5. Mukhamediev Ravil, Merembayev Timur, Symagulov Adilkhan, Kuchin Yan, Jan Rabcan. Determination of soil salinity using a UAV// The 21st INTERNATIONAL CONFERENCE INFORMATION TECHNOLOGIES AND MANAGEMENT 2023, April 20-21, 2023, ISMA University of Applied Sciences, Riga, Latvia
  6. Symagulov Adilkhan, Kuchin Yan, Jan Rabcan, Nadezhda Nikitina, Ravil Mukhamedyev, Laila Tabynbaeva. Unmanned aerial platform prototype with a multifunctional hardware and software system for acquiring and processing images and data for precision agriculture// The 21st INTERNATIONAL CONFERENCE INFORMATION TECHNOLOGIES AND MANAGEMENT 2023, April 20-21, 2023, ISMA University of Applied Sciences, Riga, Latvia
  7. Symagulov Adilkhan, Kuchin Yan, Jan Rabcan, Laila Tabynbaeva. Using UAVs and machine learning to generate plotted data sets for precision farming// The 21st INTERNATIONAL CONFERENCE INFORMATION TECHNOLOGIES AND MANAGEMENT 2023, April 20-21, 2023, ISMA University of Applied Sciences, Riga, Latvia
  8. Mukhamediev, R.I.; Merembayev, T.; Kuchin, Y.; Malakhov, D.; Zaitseva, E.; Levashenko, V.; Popova, Y.; Symagulov, A.; Sagatdinova, G.; Amirgaliyev, Y. Soil Salinity Estimation for South Kazakhstan Based on SAR Sentinel-1 and Landsat-8,9 OLI Data with Machine Learning Models. Remote Sens. 2023, 15, 4269. https://doi.org/10.3390/rs15174269