Wednesday, July 14, 2021

Adventures in Exploring the Planetary Science Data Archives

 

This week, Conor discusses that wonderful repository of US-generated planetary science data: the Planetary Data System. This data, provided for free on the web at https://pds.nasa.gov/  allows any researcher - no matter whether they are professional or amateur - to benefit from the space missions that have been funded by US taxpayer money. Sometimes, this means that discoveries made by a mission can arrive decades after that mission has ended in studies led by researchers who may not have even been alive when that mission was dispatched!

by Conor Hayes

One of my favourite occurrences in astronomy (and in science in general) is when someone manages to pull new information out of old data. For example, data collected by the Galileo spacecraft in 1997 were used in a 2018 paper (https://www.nature.com/articles/s41550-018-0450-z) to argue that Europa might have plumes of water similar to those seen on Enceladus. Of course, in order for discoveries like these to be made, old data has to be archived in a way that is easily accessible to someone who may not have intimate knowledge of how the data were originally gathered.

In an attempt to solve this problem, NASA’s Planetary Science Division founded the Planetary Data System (PDS) in 1989. The PDS was not NASA’s first attempt at an archive for its planetary missions. During the 1960s and 1970s, mission data were primarily archived at the National Space Science Data Center and the Regional Planetary Image Facilities. However, these archives were not always the most robust, focusing primarily on data storage rather than organization and documentation.

The PDS, by contrast, was designed not just to archive data, but also to present it to future researchers in a standardized format that wouldn’t require highly specialized knowledge to use. To this end, the PDS archiving standards were developed. The standards are painfully specific and in-depth (the “basic concepts” document is nearly 50 pages long, and the core reference manuals total to over 650 pages), so I won’t even attempt to explain them in full here. Instead, let’s look at an archived data project from my research to see how the standards are actually implemented.

The basic premise of the PDS archiving standards is that the data have to be accessible to any plausible future researcher. This means that the data absolutely cannot be archived in a proprietary format. Any time that you write a NumPy array to disk as a NPY file, save an image as a PNG, or export a document as a PDF, you are assuming that the technology to read those files will continue to exist. If those formats are depreciated at some point down the line and the general knowledge about how to use them is lost, then the data contained within are, for all intents and purposes, gone forever.

Of course, you have to make some assumptions somewhere, otherwise developing a standard will be nearly impossible. In this case, the PDS decided to assume that future researchers would be accessing their data using computers that could understand ASCII characters. Given that the ASCII standard itself has been a fundamental part of every computer since its creation in the 1960s, this seems like a pretty safe assumption to make.

 

Figure 1 : Some of the information you would find in a PDS label file.

Now, let’s take a look at an actual PDS data product. This product is one frame of an MSL suprahorizon movie (described elsewhere in this blog), and is archived on the PDS Cartography and Imaging Sciences Node. (The other science nodes, if you were curious, are Atmospheres, Geosciences, Planetary Plasma Interactions, Ring-Moon Systems, and Small Bodies). Each product comes in two parts: the label and the actual data. The label (seen in Figure 1) contains information about the format of the data, such as the number of bytes it contains, which byte the image data begins on, the image shape, the bit depth, and the number of bands in the image. It also lists information about the instrument used to collect the data, like the azimuth and elevation that the camera was pointed at, where on the planet the rover was located when the image was taken, and other useful information like the time of day the image was taken and the units associated with the data.

Unlike the label, which is presented in a plaintext format, the image data cannot be understood just by looking at it. If you open it in a text editor, you’ll probably get something that just looks like an incomprehensible mess of random characters (see Figure 2). That’s probably not surprising though. You wouldn’t try to open a PNG in a text editor, so why would this be any different? Well, if you try to open it in your favourite image viewing application, you likely won’t have much luck there either. 

Figure 2 : Opening a PDS image file in a text editor – a bunch of nonsense!

As it happens, both the label and the image data are presented as binary files containing no information that would help an application interpret them. A text editor assumes that you’re trying to open a text file, so the label, which is a text file, opens just fine. (This is also the reason why opening the image file in a text editor displays a bunch of random letters and symbols - the editor is interpreting the image data as ASCII characters.) But displaying an image is much more complex than plaintext, so without the guidance that your typical PNG or JPG includes, it’s unlikely that any mainstream application would be able to open a PDS image file.

This is the downside of the PDS archiving standard. Because it has to make as few assumptions as possible about the application being used to open it, the data are presented in such a general format that most common applications, used to being presented with highly structured files, have no idea what to do with them. The upside is that because the standards are so well-documented, it’s not exceptionally difficult to write your own code to read PDS files. In the interest of time, I ultimately decided to use code someone else had already written (the planetaryimage package distributed by the PlanetaryPy Project - it can be downloaded from their GitHub at https://github.com/planetarypy/planetaryimage, if you’re interested), but it could be a fun challenge to create an image viewer yourself in your language of choice.

 

Figure 3 : The results of opening a PDS image file with a tool designed specifically for the task – a beautiful image from the surface of Mars!

The PDS data archiving standards might not be as intuitive or out-of-the-box easy to use as other file formats that we might be used to, but it’s for a good cause. By standardizing our data archives, we are ensuring that future researchers will continue to have access to the vast volumes of information we have collected about our Solar System, information that may be hiding discoveries awaiting reanalysis by some scientist who might not even be born yet.

Monday, July 5, 2021

Where are all the microbes?

 

This week, our Research Associate, Dr. Haley Sapers, introduces us to the enormous hidden world of microbes all around us. Studying these organisms, the niches they inhabit, and the strategies they use to survive provide clues to the adaptability of life writ large. That, in turn, helps us to understand what kinds of planetary environments might be clement to some form of life. Above, microbes from 2.8 km below the surface of our world. (image credit: Luc Riolon, https://commons.wikimedia.org/wiki/File:Candidatus_Desulforudis_audaxviator.jpg CC-BY-SA-2.5)

by Dr. Haley Sapers

If I asked you where most of the life on Earth was, you would probably tell me it’s all around us. On the surface in forests and jungles, in the oceans around coral reefs and out there swimming around as whales and sharks. And you wouldn’t be wrong.

Macroflora and fauna – that is the large plants and animals that we can see with the unaided eye – have a lot of mass. Plants alone are massive. The cumulative weight of plant life on Earth accounts for a whopping 450 Gt (450 billion tons) of carbon. To put that in perspective, all of the cars in the world only weigh in at about 2.5 billion tons. And because of their large mass, plants and animals comprise most of the biomass on Earth. But mass isn’t the whole story –  the total number of all living organisms that we can see pales in comparison to the extraordinarily high numbers of microbes that inhabit the Earth.

A Phylogenetic Tree of all life on Earth showing relationships between large groups of organisms. Bacteria are in Blue, Archaea in Green and Eucaryotes appear in red.
(image by TimVickers https://commons.wikimedia.org/wiki/File:Collapsed_tree_labels_simplified.png)

There are 3 domains of life; the domain we, and all plants, animals, fungi, and insects are part of is called Eukarya. Prokaryotes, or “Microbes”, as they are colloquially known, form the Bacterial and Archaeal domains. Although bacteria and archaea are both microscopic, they are as different from each other as E. coli is from us! There are about 1030 individual bacterial and archaeal cells on Earth (that’s 1 nonillion or 1 thousand billion billion billion!) To throw a few more astonishingly large numbers out there, there are only an estimated 10 to the power of 24 or 1 quadrillion stars in the Universe or a measly 10 to the power of 21 (one thousand trillion) grains of sand on all the beaches and deserts of the Earth. All those thousand billion billion billion cells weigh in at approximately 77 Gt of carbon. The ~10 to the power of 10 (10 billion) people on Earth only comprise 0.06 Gt of carbon or less than 0.1% of the weight of the microbes.

So, where are all those microbial cells?

You might be surprised to learn that you’re only half human. Of all the cells that are part of your body, about half of them are microbial. They live in our mouths, stomachs, intestines, and skin (among other places…).  Don’t be alarmed – we need all these microbes – in fact, we wouldn’t be able to get any nutrients from our food without them. But if each of the 10 billion people on Earth are home to 10 to the power of 14 microbial cells – we still only end up with 10 to the power of 24 microbes, 6 entire orders of magnitude short! In 2018, a group of scientists decided to count up all the life on Earth and figure out where most of it is. There’s a great (free) publicly available book that looks at a bunch of biological statistics authored by the same group (http://book.bionumbers.org/). 

But back to where the microbes are.  

All that macroflora and fauna that we see around us every day (us included) is in some way dependent on the sun for energy. Life on the surface of the Earth is fueled by the sun, and all life needs energy. So where else could life be? It turns out almost all of those bacterial and archaeal cells (over 95% of them) are actually living deep in the Earth’s subsurface, far, far away from the energy of sun. How is that even possible? That’s actually a really good question, and one many scientists are still trying to figure out. 

There are many different metabolic strategies, or ways for life to get energy. Getting energy from the sun, or consuming other organic matter are only two – perhaps the most common to us, but by far not the most common considering the vast diversity of life on Earth. There are bacteria, for example, that ‘breathe’ iron the same way that we breath oxygen. The iron provides a different electron dumping ground instead of oxygen in anaerobic (or oxygen-free) environments. Many of the microbial subsurface dwellers use strategies like this, gaining energy directly from rocks in a metabolic process known as chemoautolithotrophy (chemo = chemical, auto = self, litho = rock or the self production of chemical energy from rocks). They’re literally living geo-electrical circuits! 

In fact, these seemingly strange metabolisms may have been the first to evolve on Earth (much before photosynthesis, or the ability to harvest energy from sunlight). The very first life on Earth may have been similar to the microbes that now live deep in the Earth’s surface. Because of the diverse energy harvesting strategies that subsurface microbes use and the evidence to suggest that these are some of the earliest metabolisms to have evolved, it’s possible that the subsurface of other planets such as Mars are also habitable in the same way. Who knows – maybe there are even microorganisms living off rocks in the deep subsurface of Mars today!