Some of the world’s most unique and innovative plant imaging technology has been developed here at the Institute for Advanced Learning and Research (IALR).
The Spatially and Mechanically Accurate Robotic Table (SMART) Platform, allows researchers to automatically take thousands of photographs over a plant’s life cycle, collecting data from beginning to end. With a mix of computer and human analysis of these photos, researchers can examine plant growth at every stage of development, allowing for a better understanding of the impact of different variables on plants.
What is a SMART Platform?
The SMART Platforms consist of an aluminum frame with a tray holding up to 86 individually potted plants. A color-sensitive webcam mounted to a mechanical arm, called a gantry, then takes a picture of each plant, providing a measurement of plant canopy size as well as other morphological traits of the plant, including leaf shape, discoloration and more. These functions are all controlled using Python code, which tells the gantry and camera when to move across the table and take photos.
IALR Vice President of Applied Research Scott Lowman (left) and Virginia Tech Research Specialist Mitchell Doss (right) look at the potted plants inside the SMART Platform.
“It’s like having a fixed camera above every plant on the table.” – Dr. Scott Lowman, Vice President of Applied Research, IALR
In most cases, the gantry and camera are programmed to photograph each plant every 15-30 minutes. By capturing images of plants during their life cycle, researchers can visually fill in the blanks that traditional measurement techniques leave behind.
Initially conceptualized by Jerzy Nowak and Alfred Wicks of Virginia Tech, and based on off-the-shelf equipment used in traditional manufacturing, the SMART Platform concept was later picked up by IALR’s Vice President of Applied Research, Dr. Scott Lowman, during his post-doctoral research in imaging. Dr. Lowman revitalized the idea, custom designing and building new functional tables, control systems and software that are now fully operational. Five SMART Platforms are currently in use, four hosted on IALR’s campus and another at Hargrave Military Academy.
Accuracy and Analysis
These tables are accurate to within a thousandth of an inch. This means that wherever the camera is programmed to go, it will return to that exact spot every single time. This accuracy allows researchers to capture timelapse photography of plant life, visualizing plant growth and death as if a stationary camera had been placed above each plant.
Researchers can utilize this technology to analyze the growth curves of plants during an experiment. Using the fully automatic system to take measurements every 15 minutes, they can see the impact of different variables over time.
“I can start an experiment, and as long as I can take care of the plants, it can run continuously until it’s completed or the plants die.” – Mitchell Doss, Virginia Tech Research Specialist
A color-sensitive webcam mounted to a mechanical arm, called a gantry, takes a picture of each plant, providing a measurement of plant canopy size as well as other morphological traits of the plant, including leaf shape, discoloration and more. These functions are all controlled using Python code, which tells the gantry and camera when to move across the table and take photos.
These tables are unique because they turn each plant into its own experimental unit. In traditional plant research, scientists take the fresh and dry weight of the harvested plant as a data point. However, this method leaves a gap within the plant’s early development and growth stages. Researchers can gain hyper-detailed data on plant growth by taking measurements of a plant throughout its life cycle.
Analyzing the data gathered from the SMART Platform is a complicated process. After conducting an experiment, researchers have folders on the computer containing thousands of images of each plant. From here, complex lines of code conduct image analysis that detects the amount of greenness (which equals the size of the plant) within an image. By calculating the difference between the number of green pixels from one image to another, researchers gain data on how much and how quickly a plant has grown.
Continued Improvement
Summer interns Kendall Moore (left) and Hunter Pruitt (right) both spent time improving niche aspects of the SMART Platform. Moore designed a hydroponics system to implement into the tables while Pruitt developed a program that automates the tables’ coordinate-finding system.
To continue the development of the SMART Platform, the Applied Research division utilizes IALR’s Summer Internship program to provide young researchers with the opportunity to work with – and improve – this technology. This summer, the team of interns helped build two brand-new SMART Platforms.
“We need people with diverse skills and backgrounds to move these tables forward.” – Dr. Scott Lowman
The interns also carry out individual projects to improve niche aspects of the SMART Platforms. Hunter Pruitt, a rising senior at North Carolina State University, spent his summer working to automate the tables’ coordinate-finding system. Eliminating the need to manually input the coordinates for a plant on the table makes the technology more efficient and easier to use.
Another intern this summer, Kendall Moore, a rising senior at the University of Virginia, designed a hydroponics system to implement into the tables. This will allow for new research on hydroponic growth techniques in addition to potted plants.
The current coding system and graphic user interface used to run the tables was developed by Samuel Hedrick, an intern during the summer of 2021. This allows the table to be user-friendly to those who don’t have a specific background in computer science.
Mitchell Doss, a Virginia Tech Research Specialist currently earning a master’s degree in horticulture, has worked with the SMART Platforms since 2014, when he was a part of the original team of interns who helped with image analysis. Now, the roles have reversed, as Doss has taken on supervising the Coding and Robotics interns.
“IALR is the place for opportunities.” – Mitchell Doss
A jack of all trades in terms of research, Doss has been a part of IALR’s Applied Research Division since 2021. He has conducted various experiments via the SMART Platforms and is carrying out research for his master’s program on potassium levels in fertilizers.
Lowman, Doss and Virginia Tech faculty member Dr. Kaylee South are drafting a research paper introducing the SMART Platform into the present literature. Once this is published, it will allow future research done via the tables to reference it, as opposed to including tedious details within the methods and materials sections of the paper.
What’s next?
Plants shake as they grow. That fact is common knowledge in the scientific community, but it is unclear why they shake or what that movement says about a plant’s overall health. The meaning of plant movement is one area IALR researchers are beginning to explore using the SMART Platform’s complete imaging capabilities.
“We’ll be practically the only people in the world that can look at plant movement as an indicator of plant health scientifically.” – Dr. Scott Lowman
One inconvenience in developing code to detect plant movement is determining the difference between movement and growth. As it stands, plant growth is categorized as the total amount of green pixels in an image, whereas plant movement is the amount of change in green pixels found in an image. In short, if green pixels are found where there were none before, or there are no green pixels where there were before, that is considered plant movement.
Recently, IALR obtained a 3D-scanning device that allows the user to create a three-dimensional model of any object. Implementing this technology within the SMART Platform could allow for further exploration into plant growth and movement.
The capability to accurately measure plant movement could offer a new dependent variable for experiments. This possibility can also be paired with IALR’s extensive endophyte library to provide early detections of bacterial impact.
“It’s not what you expect to find; it’s what you don’t expect to find that makes it interesting.” – Dr. Scott Lowman
Moving forward, the objective is to integrate new types of cameras onto the table, such as multispectral, hyperspectral and lidar cameras that could detect changes in wavelength reflection that human vision cannot detect. Different aspects of plant movement could also be studied, such as leaf tip curling and other changes in the structure of plants.
Another development is the use of a hydroponics system to further automate the process of experimentation via the SMART Platform. In the future, this technology could be merged with the Controlled Environment Agriculture Center at IALR.
The eventual goal is to share the SMART Platform technology and research capabilities with companies and institutions commercially. After many alterations and tweaks throughout the years, an efficient and user-friendly product is nearly ready for market.
