AScience/AutonomyStack
AScience/AutonomyStack is a autonomous ground vehicle navigation stack that allows an autonomous vehicle to perceive its environment and its current state to control its steering direction and speed to reach a desired target location, without human intervention. Specfically, AScience/AutonomyStack converts the exteroceptive sensor data (camera, LiDAR and Radar) and the proprioceptive vehicle sensor data to the desired vehicle speed and steering direction commands to reach the desired target location.

AScience/AutonomyStack consists of 6 integrated modules:
-
Vehicle Image Segmenter. This module uses ML/AI techniques, namely, U-Net is used as up-sampler (decoder) and ResNet34 is used for feature extraction. Each vehicle mounted camera frame is segmented into the following categories: sky, background, smooth trail, rough trail, puddle, high vegetation (trees), medium vegetation (e.g. shrubs, small trees), low vegetation (e.g. grass, small shrubs), obstacle, and unknown. This module was trained using the Yamaha CMU off-road data set (~1000 images, 4 locations, 4 seasons) augmented with about 500 additional images.
-
Aerial Image Segmenter. This module is like the vehicle image segmenter except that aerial/drone images which are obtained using a drone or are available a priori are used and segmented to identify drivable and non-drivable areas.
-
Object detector. This module processes the vehicle mounted camera frames using the YOLO8 algorithm to determine objects position and type (vehicle type, human, animal, building, etc.). The position of an identified object can be set as a dynamic waypoint. YOLO is a single-shot object detection technique containing 24 convolutional layers followed by 2 fully connected layers. It is currently the fastest object detection algorithm which is trained with 2.8M objects across 350 classes. To increase the object detector accuracy for off-road military applications, we augmented the dataset with about 400 annotated images of military vehicles, off-road road signs, and 4-legged animals, then we retrained the model using transfer learning.
-
Global Planner. This module takes as input the waypoints, point clouds (from Lidars/stereo cameras), and optionally directional speed-made-good maps calculated using an NG-NRMM software such as IVRESS/NG-NRMM. Then it uses an incremental A* algorithm to map a minimal cost (min. time & energy) global path to the next waypoint.
-
Local Planner. The input to this module includes the global path, point clouds (from LiDAR or stereo camera), segmented images, and proprioceptive sensors data. The output is the optimum travel direction and speed at the current time. For each possible travel direction, this module calculates a ground spline using the point cloud data and a moving kernel averaging procedure. The terrain roughness and positive/negative obstacle heights along each direction are estimated from the deviations of the points from the ground curve. A direction is blocked (assigned infinite cost) if an impassable ditch or mound is detected, or if the vehicle body will collide with an obstacle within the vehicle braking distance (based on speed). The vehicle steering direction is chosen to be the minimum cost direction based on: deviation from global path, deviation from current heading, time to reach the obstacle, terrain topography, surface conditions (soil strength, vegetation, and surface type); and leader speed/path. The algorithm estimates the speed of an obstacle along a travel direction by taking the difference between consecutive point-clouds. This allows the vehicle to avoid dynamic objects and to follow a moving leader more closely. The desired vehicle speed is calculated based on the roughness of the terrain, time to reach the obstacle in the vehicle path, side slope, long slope, and steering wheel angle (to avoid rollover due to turning too fast).
-
Mission Manager. manages and synchronizes the positions and tasks of each vehicle in a team of vehicles, including leader-follower and vehicle formations. The scenario program for each vehicle in the simulation along with a list of waypoints can be used to perform autonomous navigation scenarios where each vehicle in a team of vehicles is required to go from a starting position to an ending position while visiting a set of waypoints either as a leader or follower. The input to this module consists of:
-
Waypoints. Those points can be static (fixed) or dynamic (defined by the position of an object determined by the Object Detector module).
-
Mission program. Each vehicle is a team has its own mission program. The program consists of commands that the vehicle executes based on a vehicle reaching a target point in the environment (within a set tolerance) or based on a time or an action. The program commands include: MoveToPoint, MoveOnPath, CirclePoint, ReachPoint, Wait, Stop, Resume, SetAsLeader, and SetAsFollower. For example, a typical program can include commands for the vehicle to move to a certain waypoint, then wait for the vehicle and other vehicles in the team to reach that point. It can also tell some vehicles to follow one vehicle while maintaining a certain offset to the leader thus creating a vehicle formation.
-







