Home
This project aims to provide a simple means of working with publicly-available raw image data from NASA's Curiosity Rover, Perseverance Rover, and InSight lander. The project provides a set of command line tools for downloading, calibrating, and performing various functions on Mars surface images. As the images are downlinked from the spacecraft, they are uploaded to the mission websites in their unprocessed form (EDR, or Experimental Data Record). Images are made available shortly after the downlink, or up to 24 hours after downlink, depending on requirements set by instrument leads. The Mars Raw Utils (MRU) interfaces with the websites to fetch these images and their associated metadata. Calibration is performed depending on the camera specifications, image compression applied by NASA, and user preference.
NOTE: These images have had some level of lossy compression applied to them or stretched in some way, reducing their value for full science. These images are best suited for the space and planetary imaging enthusiasts along with artists. The calibration methods in MRU are simplified and adapted to deal with the public formats used by NASA online. They have not been vetted or reviewed by the individual instrument teams. Should you require the full data products, I recommend waiting until they are available in the Planetary Data System and have been processed using the full mission calibration pipeline.
Curiosity: https://mars.nasa.gov/msl/multimedia/raw-images/
Perseverance: https://mars.nasa.gov/mars2020/multimedia/raw-images/
InSight: https://mars.nasa.gov/insight/multimedia/raw-images/
The release process is defined here
Download the latest .deb file, then in a terminal:
cd <directory where you downloaded the file>
sudo apt install ./mars_raw_utils_0.1.3_amd64.deb
MRU is installed via Homebrew. Please refer to the Homebrew website for instructions if you don't yet have Homebrew installed. Once Homebrew is installed and working, open a terminal and type:
brew tap kmgill/homebrew-mars-raw-utils
brew install marsrawutils
The installation will take a few minutes as it will first install the Rust compiler then download, build, and install the Mars Raw Utils and accompanying calibration data.
MRU can be used either in Windows PowerShell and the Windows Subsystem for Linux (WSL). If you choose the WSL route, refer to Microsoft's installation instructions, select the Ubuntu distribution, then follow relavant instructions above.
For Windows PowerShell, download the latest .msi installation package from the releases page.
Check the top-level Readme for the latest instructions as to installing from source. This will require a working and updated Rust installation.
All the individual commands have been rolled into a single binary mru as subcommands. They all work more-or-less the same. Run mru -h to see an up-to-date list of subcommands. Then use help <subcommand> for specific details (e.g. mru help anaglyph).
-
mru msl-fetch- Batch fetch of raw images from NASA public website -
mru msl-date- Current MSL mission date & time -
mru msl-latest- Query NASA public website for sols with images tagged as 'latest'
-
mru m20-fetch- Batch fetch of raw images from NASA public website -
mru m20-date- Current Mars2020 mission date & time -
mru m20-latest- Query NASA public website for sols with images tagged as 'latest'
-
mru nsyt-fetch- Batch fetch of raw images from NASA public website -
mru nsyt-date- Current InSight mission date & time -
mru nsyt-latest- Query NASA public website for sols with images tagged as 'latest'
-
mru anaglyph- Generate a red/blue 3d anaglyph from a stereo pair -
mru calibrate- Batch calibration on raw images -
mru focus-merge- Focus merge on a stack of images -
mru hpc-filter- Runs hot pixel detection and correction -
mru inpaint- Runs an inpaint filter on an image -
mru levels- Apply Photoshop/Gimp-like levels correction -
mru crop- Crop an image -
mru debayer- Debayers an image -
mru diffgif- Generate a differential gif on a set of images -
mru mean-stack- Generate a mean (average) of a set of images
Fetch images from each mission using the mission-appropriate command, such as mru msl-fetch, mru m20-fetch, and mru nsyt-fetch. To see which instruments to use when downloading, type mru msl-fetch -i (replacing msl with the m20 and nsyt as needed). Some instruments have shortcuts in there, such as MASTCAM which will send both MAST_LEFT and MAST_RIGHT to the server. Multiple instruments can be used at a time.
NAV_LEFT
NAV_LEFT_A
NAV_LEFT_B
NAV_RIGHT
NAV_RIGHT_A
NAV_RIGHT_B
CHEMCAM
CHEMCAM_RMI
MAHLI
MAHLI
MASTCAM
MAST_LEFT
MAST_RIGHT
MARDI
MARDI
HAZ_FRONT
FHAZ_RIGHT_A
FHAZ_LEFT_A
FHAZ_RIGHT_B
FHAZ_LEFT_B
HAZ_REAR
RHAZ_RIGHT_A
RHAZ_LEFT_A
RHAZ_RIGHT_B
RHAZ_LEFT_B
SHERLOC
SHERLOC_ACI
EDLCAM
EDL_DDCAM
EDL_PUCAM1
EDL_PUCAM2
EDL_RUCAM
EDL_RDCAM
LCAM
WATSON
SHERLOC_WATSON
SUPERCAM
SUPERCAM_RMI
HAZ_REAR
REAR_HAZCAM_LEFT
REAR_HAZCAM_RIGHT
HAZ_FRONT
FRONT_HAZCAM_LEFT_A
FRONT_HAZCAM_LEFT_B
FRONT_HAZCAM_RIGHT_A
FRONT_HAZCAM_RIGHT_B
CACHECAM
CACHECAM
PIXL
PIXL_MCC
HELI_RTE
HELI_RTE
NAVCAM
NAVCAM_LEFT
NAVCAM_RIGHT
MASTCAM
MCZ_LEFT
MCZ_RIGHT
HELI_NAV
HELI_NAV
SKYCAM
SKYCAM
ICC
icc
IDC
idc
After a while, a single or ad-hoc directory structure for downloading raw images can get unwieldy and inefficient, especially if you prefer to hold on to images for later usage. A simple recommended structure can follow a mission/sol/instrument/images.png format, such as:
MSL/
3430/
CCAM/ -> ChemCam
ECAM/ -> Engineering Cameras
MAHLI/ -> MAHLI
MARDI/ -> MARDI
MCAM/ -> MastCam
3431/
...
M20/
0313/
CCAM/ -> CacheCam
NCAM/ -> Navigation Cameras
FHAZ/ -> Front HazCams
RHAZ/ -> Rear HazCams
HNAV/ -> Ingenuity Navigation Camera
HRTE/ -> Ingenuity RTE (color) Camera
PIXL/ -> PIXL
SCAM/ -> SuperCam
SKYCAM/ -> SkyCam
WATSON/ -> Watson
ZCAM/ -> Mastcam-Z
0314/
...
NSYT/
1198/
IDC/ -> Instrument Deployment Camera
ICC/ -> Instrument Context Camera
1199/
...
Say you look at the MSL raw image site and you identify MastCam and MAHLI images that you would like to work on from sol 3451:
cd MSL
mkdir -p 3451/MCAM
mkdir -p 3451/MAHLI
cd 3451/MCAM
mru msl-fetch -c MASTCAM -s 3451
mru calibrate -i *jpg
cd ../MAHLI
mru msl-fetch -c MAHLI -s 3451
mru calibrate -i *jpg
This will apply the standard set of decompanding, blemish repair, and debayering to the images.
Some people prefer the standard companded color that appears on the public websites. For any color images, you can use the -r option to skip the decompanding (inverse look-up table) step.
Quite often, the Curiosity Rover will look for dust devils using the navigation cameras. While, for varying reasons, there are a number of different sequences used, we will are looking for the mono subframed 21-frame dust devil movie with sequence id NCAM00595. Well, we find one that was executed on sol 3418. Because we are looking for a specific seqid, we can use the -S option to only download matching images.
mkdir -p 3418/ECAM
cd 3418/ECAM
mru msl-fetch -c NAV_RIGHT -s 3418 -S NCAM00595
The NavCam calibration routine includes hot pixel detection and correction with the -t option and a threshold value, which we'll use to clean up the movie frames.
mru calibrate -i *jpg -t 2.0
We will now put the frames together into a difference gif. This computes a mean value of all the frames, and for each frame subtracts it from that mean to derive a differential. The differential is stretched and re-added back to the mean. In this example, we will have the differential itself on the top and the differential+mean on the bottom:
mru diffgif -i *NCAM00595*-rjcal.png -o DustDevil_3418_NCAM00595_rjcal.gif -v -b 0 -w 2.0 -g 2.5 -l 5 -d 20 -p stacked
Result:
Most calibration routines support profiles which provide a standard set of parameters, or presets, each providing different output products. Say you find some nice Perseverance Mastcam-Z images on sol 401. We also just want to download the grayscale bayer-pattern images, which we can do by adding the -S ECM option on the fetch command.
mkdir -p 0401/ZCAM
cd 0401/ZCAM
mru m20-fetch -c MASTCAM -s 401 -S ECM
Now we want to calibrate them using all the available profiles to see what comes out using the -P <profiles> option.
mru calibrate -i *J0?.png -P m20_zcam_bay m20_zcam_ilt m20_zcam_rad m20_zcam_cwb m20_zcam_cb2 m20_zcam_dcc
These profiles provide:
- BAY - Debayered raw
- ILT - Inverse lookup table decompanding
- RAD - Radiometric correction
- CWB - Color white balancing
- CB2 - Color white balancing, but a little more subtle.
- DCC - Decorrelated color
Result:
Original:
BAY:
ILT:
RAD:
CWB:
CB2:
TODO