SPOC

A Deep Learning-based Terrain Classifier for Mars Rovers

What is SPOC?

Like on Earth, the mobility of Mars rovers is highly sensitive to terrain type. SPOC (which stands for Soil Property and Object Classification) is a class of software capabilities that utilizes machine learning to classify terrain types from imagery.

Why SPOC?

Terrain has been a major source of risk for Mars rovers. Spirit was embedded in sand and ended its mission. Opportunity and Curiosity also have experiences of getting stuck in sand, although they were able to escape.

SPOC’s autonomous terrain classification capability helps human rover drivers on Earth, as well as on-board algorithms, to drive safely on the red planet.

SPOC Implementations

SPOC is a family of algorithms.

SPOC-HiRISE and SPOC-NAVCAM are based on deep convolutional neural networks, and are intended for use on ground operations. SPOC-Lite and SPOC-HPSC/SNPE are intended for on-board deployment, with SPOC-Lite employing a simpler ML model runnable on CPUs, while SPOC-HPSC/SNPE are based on deep learning and use GPUs.

SPOC-Lite

SPOC-Lite is an on-board terrain classifier compatible with a RAD750-class CPU. Taking monaural images as an input, it outputs the probability of sand on the image.

SPOC-Lite software is open-source, and is available here.

SPOC-HiRISE

SPOC-HiRISE classifies every pixel of HiRISE imagery into 17 terrain classes at 25 cm resolution. The overall accuracy compared to human labels is more than 90%, sometimes even outperforming humans.

People-Powered Innovation

Numerous volunteers over the Internet are contributing to improving SPOC by proving training data through a crowdsourcing project, AI4Mars, which has already collected 10K+ labels on Curiosity’s NAVCAM images. Anyone on Earth can help NASA explore Mars from home.

Testimonials

Hear from three members of the M2020 team about SPOC's ability to aid both engineers and scientists on Mars missions.

More Testimonials

"Drones, and autonomous robots more generally, need to be able to understand the environment around them. Deep learning and semantic segmentation are key tools that make this happen, and my work on SPOC with these technologies helped lay the foundation for the projects I'm working on today."

- Ryan Kennedy, Skydio (Former 347 and a SPOC member)

"The team's work at NASA JPL focused on Martian terrain, but machine learning is at the core of autonomous driving here on Earth, too. My SPOC experience gave me the fundamentals of non-deterministic outcomes, model training, and system testing that apply to my role at Waymo, where I analyze fault protection architectures and system testing."

- Amanda Steffy (Formerly at Waymo, currently at 382)

"Planning optimal paths that avoid obstacles is critical for mobile robots, whether they’re on Mars or in a Galaxy Far, Far Away. Working on SPOC taught me that deep learning can be used even in highly constrained real-world applications. It has become a core part of my work at Disney."

- Jeremie Papon (Formerly at Waymo, currently at 382)

"Machine Learning has the tremendous capability of bridging the gap between the macroscopic and the microscopic world. The lessons learned from SPOC still inspire our work on cancer segmentation and detection on microscope slides in pathology."

- Prof. Thomas Fuchs (Former 347 and a SPOC member)

Publications

Team

JPL

Lead

: Hiro Ono

Development and Testing

Deegan Atha, Shreyansh Daftry, Mike Swan, Henry Leopold, Yumi Iwashita, Kyohei Otsu, Annie Didier, Tanvir Islam, Jacek Sawoniewicz, Olivier Lamarre, Chris Mattmann, Travis Brown, Sami Sahnoune

M2020

Matt Heverly, Erisa (Hines) Stilley, Rich Rieber, Fred Calef, Tariq Soliman

MSL

Jeng Yeng, Nick Tooles, Junggon Kim , Mark Maimone, Chris Roumeliotis, John Wright, Doug Alexander, Amy Culver, Vu Nguyen, Adrian Tinio, John Michael Morookian, Alex Cervantes, Bob Deen, Abigail Fraeman

Former JPL

Development and Testing

Brandon Rothrock (PAIGE), Thomas Fuchs (Memorial Sloan Kettering Cancer Center), Amanda Steffy (Waymo), Ryan Kennedy (Skydio), Jeremie Papon (Disney Research), Oktay Arslan (Tesla)

Labeling

Anthony Campbell (Brigham Young University), Hiroka Inoue (JAXA)

External

Raymond Arvidson (Washington University at St. Louis), Catherine Weitz (Planetary Science Institute), David Rubin (USGS), Nathaniel Stein (California Institute of Technology)