Emily Schwing: In October 2019 an international team of scientists onboard an icebreaker intentionally let Arctic Sea ice freeze up around the ship. They wante d to learn more about the ice itself. But in April 2020, just halfway through the year-long experiment, it was unclear if that ice would stay frozen for the remaining six months of the project.

[CLIP: Show music; Sea ice sounds]

Schwing: You’re listening to Scientific American’s Science, Quickly. I’m Emily Schwing.

Sea ice, according to scientists, is melting at an alarming rate—so quickly that some researchers believe traditional methods for forecasting its extent may not keep up with the pace of a changing climate. 

By the year 2050, the Arctic could be ice-free in the summer months. And shipping traffic in the region is on the rise, but predicting sea ice extent is complicated. 

Today we’re looking at how machine learning—artificial intelligence—could become the tool of the future for sea ice forecasting. 

Leslie Canavera: We build artificial intelligence and machine learning models for the Arctic, based on the science of oceanography.

Schwing: That’s Leslie Canavera. She is CEO of a company called PolArctic, and she is trying to forecast ice in a different way than science ever has.

Since the late 1970s, scientists have relied on physics and statistical modeling to create sea ice forecasts. 

Canavera: When you take two water molecules, and you freeze them together, you know, like, right, this is how they freeze together. But there’s a lot of assumptions in that. And when you extrapolate to the ocean, there’s a lot of error.... And statistical modeling is based on, like, historical things of what’s happened. But with climate change, it’s not acting like the history anymore. And so artificial intelligence really takes the best of both of those and is able to learn the system and trends to be able to forecast that more accurately.

Schwing: Of course, that foundation of statistics and historical data is still important, even with its errors and caveats. 

Holland: We can't model every centimeter of the globe.

Schwing: Marika Holland is a scientist at the National Center for Atmospheric Research in Boulder, Colorado. The center has been using physics and statistical modeling to predict sea ice extent for the past five decades. Holland says that she is confident in the methodology but that these forecasts aren’t perfect.  

Holland: You know, we have to kind of coarsen things, and so we get a little bit of a muddy picture of how the sea ice cover is changing or how aspects of the climate or the Earth’s system are evolving over time. 

Schwing: Marika says there are also a lot of smaller-scale processes that can create problems for accurate forecasting.

Holland: Something like the snow cover on the sea ice, which can be really heterogeneous, and that snow is really insulating, it can affect how much heat gets through the ice.... We have to approximate those things because we aren’t going to resolve every centimeter of snow on the sea ice, for example.... So there’s always room for improvement in these systems.

Schwing: It’s that space—the room for improvement—where Leslie says artificial intelligence can be most helpful. And that help is especially important right now because of what is happening in the Arctic.

According to the Arctic Council, marine traffic increased by 44 percent through the Northwest Passage between 2013 and 2019. Search-and-rescue capabilities in the region are limited, and there has been increased attention on the region for its vast natural resource development potential. Leslie says AI can create a forecast on a smaller scale, homing in on specific locations and timing to benefit those user groups.

Canavera : We did a seasonal forecast and then an operational forecast where the seasonal forecast was 13 weeks in advance. We were able to forecast when their route would be open..., and we were actually to the day on when the route would be able to be open and they would be able to go. And then we did operational forecasts where it was like,“All right, you’re in the route, what [are] the weather conditions kind of looking like?”

Schwing: Using AI to forecast sea ice extent isn’t a novel approach, but it is gaining traction. A team led by the British Antarctic Survey’s Tom Anderson published a study two years ago in the journal Nature Communications. In a YouTube video that year, Tom touted the benefits of his team’s model, called IceNet.

[CLIP: Anderson speaks in YouTube video: “What we found is super surprising. IceNet actually outperformed one of the leading physics-based models in these long-range sea ice forecasts of two months and beyond while also running thousands of times faster. So IceNet could run on a laptop while previous physics-based methods would have to run for hours on a supercomputer to produce the same forecasts.”] 

Schwing: One of the biggest limitations when it comes to AI-generated sea ice forecasts is what Leslie calls “the black box.”

Canavera: And you have all of this data. You put it into the artificial intelligence black box, and then you get the answer. And the answer is right. And scientists get very frustrated because they’re like, “Well, tell me what the black box did,” right? And you’re like, “Well, it gave you the right answer.” And so there's a big trend in artificial intelligence that is called XAI, and explainable AI si hwat that kind of relates to and “Why did your artificial intelligence give you the right answer?”

Sometimes, she says, AI happens upon the right answer but for the wrong reasons. That’s why Marika at the National Center for Atmospheric Research says the most effective sea ice forecasts are likely to come from combining both machine learning and five decades’ worth of physics and statistical modeling.

Holland: If machine learning can help to improve those physics-based models, that’s wonderful. And that is kind of the avenues that we’re exploring—is how to use machine learning to improve these physics-based models that then allow us to kind of predict how the climate and the sea ice system are going to change on decadal, multidecadal [kinds] of timescales. 

Schwing: And there’s one piece of the sea ice forecasting puzzle Leslie, who is Alaska Native, believes is irreplaceable: traditional Indigenous knowledge.

Canavera: What's great about traditional Indigenous knowledge and artificial intelligence is that a lot of traditional Indigenous knowledge is data, and artificial intelligence builds models on data. And that’s why it works better than these like dynamical models in being able to incorporate the traditional Indigenous knowledge. 

For Science, Quickly, I’m Emily Schwing.

Scientific American’s Science, Quickly is produced and edited by Tulika Bose, Jeff DelViscio and Kelso Harper. Our theme music was composed by Dominic Smith.

You can listen to Science, Quickly wherever you get your podcasts. For more up-to-date and in-depth science news, head to ScientificAmerican.com. Thanks, and see you next time.

Jocie Bentley (tape): PSA: don’t bring hiking boots when walking the tundra. Your feet will get soaked like a wet sponge.

Gabriel Hould Gosselin (tape): Almost there. [laughs] About halfway. Well, it’s a lot faster with snowmobiles.

Bentley: Hey, I’m Jocie Bentley, and this is the final episode of a three-part Science, Quickly Fascination series from a fast-warming Arctic.

Today I’m heading to a place called the Trail Valley Creek Research Station, high in the Canadian Arctic. I’m sloshing along with Gabriel Hould Gosselin. Gabriel is a research assistant for Wilfrid Laurier University in Ontario and the University of Montreal.

We actually did his original interviews in French, so you are listening to a combination of field audio and a new interview in English.

Gosselin: Alright, so I brought you about 60 kilometers north of Inuvik, which is a little town at the top of the Northwest Territories, out near the Tuktoyaktuk highway in the tundra.

Bentley: We passed the tree line on our drive. No more trees, just a flat carpet of orange and red tundra, covering softly rolling hills. It’s unlike any landscape I’ve ever seen.

Gosselin: There’s permafrost all over the place. It’s super deep. It’s, like, four- to 600 meters, depending, deep.

Bentley: So for all you non-Metric listeners, that’s 1,300 to almost 2,000 feet deep. That’s deeper than almost all of the world’s tallest skyscrapers.

Gosselin: And that’s permafrost being permanently frozen ground, ground that doesn’t go above zero degrees Celsius. Of course, the top layer kind of thaws over summer and then refreezes over winter.

Bentley: And that is exactly the part that Gabriel is most interested in.

Gosselin: That’s the area that is active, that has bacteria kind of decomposing organic matter and farting out carbon dioxide and methane ...

Bentley: [Laughs]

Gosselin: I mean, there’s this kind of methane bomb. That's what people are thinking about, and thinking ...

Bentley: The more there is, the more trouble we’re in.

Gosselin: Oh, boy, if things keep on warming, there’s a whole bunch of ground that’s been frozen for a long time with basically a huge pool of carbon that’s just ready to be digested by those methane-producing bacteria.

Bentley: And that’s a huge concern for researchers such as Gabriel.

Gosselin: What’s going to happen? And that’s, that’s a concern. What’s going on in the Arctic?

Bentley: And that’s why he’s here. There’s only one way to learn the answer to that question, and it’s with data.

Gosselin: There’s been very little real sampling that’s been done. I mean, there’s more and more stuff that comes out. There’s more and more satellites that are put out there that are much better and better at looking at different wavelengths. Some of them use radar. Some of them use infrared. Some of them actually see, measure directly, the amount of methane that’s in the air in certain areas.

Bentley: But those methods are less reliable without actual measurements from the ground.

Gosselin: To validate those measurements, we need ground-truthing data, so, data that comes from the area that that satellite’s looking at to kind of compare what’s being measured and from up there and then what’s being measured from the ground.

Bentley: And that’s why we’re standing among a bunch of white tents on red tundra. It’s all cleaned up for the offseason.

Bentley (tape): Is it not normally this clean?

Gosselin: I mean, normally there’s a bunch of people living here, so there’s stuff everywhere.

Uh, normally we have a bug net, like, uh, one of those, uh, kind of gazebo bug net things. Yeah. And then we set up tents all around here. I mean, normally there’s a whole bunch of chairs, and, uh, you know, when we get a heater going, people kind of, it’s where we eat and just spend our time. It’s tea! And it’s nice and clean.

Bentley: We make our way to Gabriel’s main research station. It’s called an eddy covariance tower.

Gosselin: A private company installed the tower. I just installed the instruments on it. Yeah, I spent 30 hours on it.

Bentley: So he climbed 20 meters, or 65 feet, up this skinny tower carrying giant pieces of equipment. Here’s what he installed.

Gosselin: So on the tower for eddy covariance, in principle, there’s two main instruments: one instrument that measures the concentration of gas that we’re interested in in the air and [another] that uses infrared.

Bentley: Remember those greenhouse gas farts? This is how science “sniffs” them out.

Gosselin: So basically, we know, per volume, the amount of carbon dioxide or water vapor that’s in that parcel there. And   we’re using ultrasound to measure it about 10 times a second—the speed of wind and the X, Y and Z direction. And then we do the covariance between the vertical movements of wind and the concentration of gas.

Bentley (tape): Do we have enough towers in the North to get an accurate picture?

Gosselin (tape): No, we do not. It’s incredibly difficult and costly to just get something out there. Just to go in those environments is really expensive. The biggest challenge that I found doing stuff out there is not necessarily the cold or, like, the mosquitoes or whatever. It’s getting enough power for all of those instruments that measure 24 hours a day, 10 times a second.

Bentley: Is tundra really that different that we need testing in all these different locations? How does it differ?

Gosselin (tape): You try to draw, like, not a paint-by-numbers but, you know, one of those little, and then you have to kind of draw a line.

Bentley (tape): Connect the dots!

Gosselin (tape): Yeah. But basically, you get an entire image with four points, and you have to draw an elephant. Like, it’s not going to look like an elephant. It’s going to look like a square.

So it’s the same thing. Like, you’re trying to get a detailed image of what, like, how different types of landscapes in the North behave. But if we only have four points, we’re going to be missing a lot of detail, and maybe some of the details are going to be important.

Bentley: How big of a problem is this? How worried should we be?

Gosselin: There’s been very little real sampling that’s been done. I don’t want to be alarmist. There’s always the kind of trap for scientists. They just go, “Well, I don’t know. I’m not qualified enough.” And then basically that’s taken, um, by the media saying, “Well, we’re not sure whether it’s a problem.”

Of course it’s concerning. Of course we have to do something. In fact, we’re past the point of no return. Things are going to happen. Things are warming up already beyond our control. And the consequences of that are happening now and are going to happen. By how much, I can’t say.

Bentley: Science, Quickly is produced by Jeffrey DelViscio, Tulika Bose and Kelso Harper. Our music was composed by Dominic Smith. Like and subscribe wherever you get your podcasts. And for more science news, please go to ScientificAmerican.com.

This podcast was produced in partnership with Let's Talk Science.

Thanks for joining us for our Arctic series. I’m Joc Bentley, and this is Science, Quickly.

Funding for this story was provided in part by Let's Talk Science, a charitable organization that has provided engaging, evidence-based STEM programs for 30 years at no cost for Canadian youth and educators

[CLIP: Music]

Timmy Broderick: So I’m sitting inside this stone clock tower in the small town of Castellaro Lagusello in Italy. It’s pretty old, like 800 years old. I had found a nook in this tower where I could sit and record this ethereal music coming from the speaker in front of me. And through the slit of a window behind me, I could watch Italians mill about below.

Jason Drakeford: So Timmy, why are you being a recluse in this tower instead of talking with people on the ground?

Broderick: [Laughs.] Well, it’s a fair question. I was gathering tape to be played on this podcast, but it was also my last day at the Universe in All Senses, it’s an astronomy festival. And I was pooped. For three days, I ran around this tiny, picturesque town capturing what was likely the first multisensory astronomy festival ever.

[CLIP: Music]

Broderick: Oh, does this sound familiar?

Drakeford: Yes! That’s Matt Russo’s TRAPPIST-1 sonification that we listened to in the first episode!

Broderick: Yeah! The festival organizers rigged up the clock tower to play a bunch of sonifications on a loop, and then at night, they’d project visualizations of these compositions onto the face of the clock tower.

[CLIP: Show theme music]

Drakeford: You are listening to Scientific American’s Science, Quickly. I’m Jason Drakeford.

Broderick: And I’m Timmy Broderick. In the prior episode of this three-part Fascination, we dove into the origins of turning space data into sound. In this final episode, we’re traveling to Italy to see whether astronomical sonifications can help people with disabilities better understand the awe and wonder of the cosmos.

Broderick [on tape]: Okay, so I’m walking around. Four thousand people at once is a lot…

Drakeford: Okay so how did you even hear about this festival?

Broderick: In January I wrote a story about this burgeoning movement in astronomy. One of my sources for that story was Anita Zanella. She’s an Italian astronomer who grew up playing with her relatives on the stone streets of Castellaro Lagusello. She told me about the festival.

Anita Zanella: Castellaro has a lot of historical buildings. The villa and the lake are typical ones. The other important point for this little village is the tower, which is really the key part, the heart, of the village.

Broderick: Castellaro has been holding an astronomy festival for a couple years now. But this is the first time it has really been multisensory. Every workshop, every talk, every event—all of them were available in at least two senses.

Zanella: Inclusion is the main focus this year. So being able to share the knowledge and the beauty of astronomy and the beauty of the universe with whoever, irrespective of disability and sensory limitations.

Broderick: They also had these giant QR code-like signs set up around the festival to help blind people navigate and understand a workshop or exhibit.

Drakeford: Uhuh.

Broderick: It was pretty wild. My phone picked up the code from like 10 feet away!

Drakeford: Woah, this is so cool!

[CLIP: Festival sound]

Broderick: Yeah. The highlight of the first evening was the keynote panel with Anita and two visually impaired astronomers, Nic Bonne and Enrique Pérez-Montero. You might remember Enrique from our last episode. The three of them discussed how to build a “multi-sensory discovery of the sky above us.”

[CLIP: Festival sound]

Broderick: After the discussion, I talked with Claudia Beschi. She’s 25, hails from nearby Mantova, wants to be a translator and just completed graduate school. She found the discussion fascinating. She’s also been blind since birth.

[CLIP: “The Bullet Cluster” by Matt Russo] 

Claudia Beschi: I didn’t think it was possible to translate galaxies into sounds.... I felt like nature was talking to me.

I believe that nature has its own sounds. And listening to that sound, it was as if that galaxy was telling something to me. Like this galaxy was describing itself to me.

Drakeford: Woah. The galaxy was speaking to her. This is wild!

Broderick: Yeah, I was actually really moved by that conversation. It stuck with me throughout the festival.

And so the next day was the first, like, full day. There was a lot going on. We had a bunch of workshops happening. We had a radio wave scavenger hunt, we had comet smelling, there was crafting galaxies out of felt and other fabrics, and also last but definitely not least, banging pots and pans to represent stellar energies.

[CLIP: Pots and pans banging]

All of the workshops were staffed by local kids who could teach the attendees and especially the young kids. Elisa Zaltieri goes to high school in Mantova, and she ran the pots and pans station.

Zaltieri: It's an activity about how stars are actually different. We make child play pots actually.

Broderick (tape): So what are you gonna have these kids do?

Zaltieri: We have to make them recognize how stars are different and then we have to make them play, like, if they were stars.

We were trying to explain to them that [for] the biggest star, play the hardest. And the smallest, play lower, actually, because they have less energy.

Broderick (tape): So if you play really loudly, you'll have more energy; if you play really softly, you’ll have less energy?

Zaltieri: Yep!

[CLIP: Pots and pans banging]

Broderick: While the festival was ostensibly for kids, there were many workshops and events for adults.

Mattia Grella: I’m Mattia, 33, and I’m from Verona, quite close by.

Broderick: Mattia came to the festival for a couple of reasons. He knows one of the organizers, but he is also passionate about astronomy. He’s a hardcore Trekkie, as well. I met him at one of the workshops. He was creating a kind of patch made from different fabric textures. It was supposed to represent the different parts of a galaxy.

Grella: It’s smooth and kind of wavy, soft but not really smooth, kind of like the music we listened to before. It was a piece played on the piano. It was a soft piece, but played with the piano, it also had kind of a certain rhythm to it. So these little waves, at least to me, they represent this softness but also this movement.

Broderick: Mattia has been visually impaired since birth. His version of space is definitely not the inky, black expanse that you or I perceive it as.

Grella: I know the stars are classified like yellow dwarfs, red giants. And in my head, I imagined them quite with vivid colors, but I have no idea if they’re like basically white with a slight shade of yellow, red, or if they’re as vivid as I imagine them.

[CLIP: “SgrA Chandra” by Matt Russo] 

Drakeford: So what did you think? Was the festival successful?

Broderick: To be honest, I’m not sure. It was definitely fun! Like, everyone I saw was having a great time and really engaged with astronomy. But I didn’t really see many people using those giant QR codes. I know that there was bus trouble that kept many local blind and partially sighted people from coming to Castellaro.

Drakeford: Were there a lot of visually impaired people there?

[CLIP: “The Galactic Center” by Matt Russo] 

Broderick: I don’t know how many of the 4,000 attendees were blind or visually impaired. Neither do the festival organizers. That’s just unknowable. What I do know is that for the blind people I talked with — for Claudia, for Mattia — the festival and sonifications were really helpful. Claudia was there for one night, but she was thrilled by what she heard.

Beschi: I don’t know if I will see the world in a different way in the future, but I’m sure that this experience, in a way, taught something positive to me. Because I love nature. I think that nature speaks to us in every way possible. And these translations into sound and into tactile modes is a really good way to get in touch with nature, especially for us because we can’t, we can’t see how nature is really made of.

[CLIP: Outro music]

Broderick: Science, Quickly is produced by Jeff DelViscio, Tulika Bose, Kelso Harper and Carin Leong. Our theme music was composed by Dominic Smith. Matt Russo provided the sonifications you heard in this episode.

Drakeford: Don’t forget to subscribe to Science, Quickly wherever you get your podcasts. For more in-depth science news and features, go to ScientificAmerican.com. And if you liked the show, give us a rating or review.

Broderick: For Scientific American’s Science, Quickly, I’m Timmy Broderick.

Drakeford: And I’m Jason Drakeford. See you next time!

[CLIP: Music]

Timmy Broderick: So I’m sitting inside this stone clock tower in the small town of Castellaro Lagusello in Italy. It’s pretty old, like 800 years old. I had found a nook in this tower where I could sit and record this ethereal music coming from the speaker in front of me. And through the slit of a window behind me, I could watch Italians mill about below.

Jason Drakeford: So Timmy, why are you being a recluse in this tower instead of talking with people on the ground?

Broderick: [Laughs.] Well, it’s a fair question. I was gathering tape to be played on this podcast, but it was also my last day at the Universe in All Senses, it’s an astronomy festival. And I was pooped. For three days, I ran around this tiny, picturesque town capturing what was likely the first multisensory astronomy festival ever.

[CLIP: Music]

Broderick: Oh, does this sound familiar?

Drakeford: Yes! That’s Matt Russo’s TRAPPIST-1 sonification that we listened to in the first episode!

Broderick: Yeah! The festival organizers rigged up the clock tower to play a bunch of sonifications on a loop, and then at night, they’d project visualizations of these compositions onto the face of the clock tower.

[CLIP: Show theme music]

Drakeford: You are listening to Scientific American’s Science, Quickly. I’m Jason Drakeford.

Broderick: And I’m Timmy Broderick. In the prior episode of this three-part Fascination, we dove into the origins of turning space data into sound. In this final episode, we’re traveling to Italy to see whether astronomical sonifications can help people with disabilities better understand the awe and wonder of the cosmos.

Broderick [on tape]: Okay, so I’m walking around. Four thousand people at once is a lot…

Drakeford: Okay so how did you even hear about this festival?

Broderick: In January I wrote a story about this burgeoning movement in astronomy. One of my sources for that story was Anita Zanella. She’s an Italian astronomer who grew up playing with her relatives on the stone streets of Castellaro Lagusello. She told me about the festival.

Anita Zanella: Castellaro has a lot of historical buildings. The villa and the lake are typical ones. The other important point for this little village is the tower, which is really the key part, the heart, of the village.

Broderick: Castellaro has been holding an astronomy festival for a couple years now. But this is the first time it has really been multisensory. Every workshop, every talk, every event—all of them were available in at least two senses.

Zanella: Inclusion is the main focus this year. So being able to share the knowledge and the beauty of astronomy and the beauty of the universe with whoever, irrespective of disability and sensory limitations.

Broderick: They also had these giant QR code-like signs set up around the festival to help blind people navigate and understand a workshop or exhibit.

Drakeford: Uhuh.

Broderick: It was pretty wild. My phone picked up the code from like 10 feet away!

Drakeford: Woah, this is so cool!

[CLIP: Festival sound]

Broderick: Yeah. The highlight of the first evening was the keynote panel with Anita and two visually impaired astronomers, Nic Bonne and Enrique Pérez-Montero. You might remember Enrique from our last episode. The three of them discussed how to build a “multi-sensory discovery of the sky above us.”

[CLIP: Festival sound]

Broderick: After the discussion, I talked with Claudia Beschi. She’s 25, hails from nearby Mantova, wants to be a translator and just completed graduate school. She found the discussion fascinating. She’s also been blind since birth.

[CLIP: “The Bullet Cluster” by Matt Russo] 

Claudia Beschi: I didn’t think it was possible to translate galaxies into sounds.... I felt like nature was talking to me.

I believe that nature has its own sounds. And listening to that sound, it was as if that galaxy was telling something to me. Like this galaxy was describing itself to me.

Drakeford: Woah. The galaxy was speaking to her. This is wild!

Broderick: Yeah, I was actually really moved by that conversation. It stuck with me throughout the festival.

And so the next day was the first, like, full day. There was a lot going on. We had a bunch of workshops happening. We had a radio wave scavenger hunt, we had comet smelling, there was crafting galaxies out of felt and other fabrics, and also last but definitely not least, banging pots and pans to represent stellar energies.

[CLIP: Pots and pans banging]

All of the workshops were staffed by local kids who could teach the attendees and especially the young kids. Elisa Zaltieri goes to high school in Mantova, and she ran the pots and pans station.

Zaltieri: It's an activity about how stars are actually different. We make child play pots actually.

Broderick (tape): So what are you gonna have these kids do?

Zaltieri: We have to make them recognize how stars are different and then we have to make them play, like, if they were stars.

We were trying to explain to them that [for] the biggest star, play the hardest. And the smallest, play lower, actually, because they have less energy.

Broderick (tape): So if you play really loudly, you'll have more energy; if you play really softly, you’ll have less energy?

Zaltieri: Yep!

[CLIP: Pots and pans banging]

Broderick: While the festival was ostensibly for kids, there were many workshops and events for adults.

Mattia Grella: I’m Mattia, 33, and I’m from Verona, quite close by.

Broderick: Mattia came to the festival for a couple of reasons. He knows one of the organizers, but he is also passionate about astronomy. He’s a hardcore Trekkie, as well. I met him at one of the workshops. He was creating a kind of patch made from different fabric textures. It was supposed to represent the different parts of a galaxy.

Grella: It’s smooth and kind of wavy, soft but not really smooth, kind of like the music we listened to before. It was a piece played on the piano. It was a soft piece, but played with the piano, it also had kind of a certain rhythm to it. So these little waves, at least to me, they represent this softness but also this movement.

Broderick: Mattia has been visually impaired since birth. His version of space is definitely not the inky, black expanse that you or I perceive it as.

Grella: I know the stars are classified like yellow dwarfs, red giants. And in my head, I imagined them quite with vivid colors, but I have no idea if they’re like basically white with a slight shade of yellow, red, or if they’re as vivid as I imagine them.

[CLIP: “SgrA Chandra” by Matt Russo] 

Drakeford: So what did you think? Was the festival successful?

Broderick: To be honest, I’m not sure. It was definitely fun! Like, everyone I saw was having a great time and really engaged with astronomy. But I didn’t really see many people using those giant QR codes. I know that there was bus trouble that kept many local blind and partially sighted people from coming to Castellaro.

Drakeford: Were there a lot of visually impaired people there?

[CLIP: “The Galactic Center” by Matt Russo] 

Broderick: I don’t know how many of the 4,000 attendees were blind or visually impaired. Neither do the festival organizers. That’s just unknowable. What I do know is that for the blind people I talked with — for Claudia, for Mattia — the festival and sonifications were really helpful. Claudia was there for one night, but she was thrilled by what she heard.

Beschi: I don’t know if I will see the world in a different way in the future, but I’m sure that this experience, in a way, taught something positive to me. Because I love nature. I think that nature speaks to us in every way possible. And these translations into sound and into tactile modes is a really good way to get in touch with nature, especially for us because we can’t, we can’t see how nature is really made of.

[CLIP: Outro music]

Broderick: Science, Quickly is produced by Jeff DelViscio, Tulika Bose, Kelso Harper and Carin Leong. Our theme music was composed by Dominic Smith. Matt Russo provided the sonifications you heard in this episode.

Drakeford: Don’t forget to subscribe to Science, Quickly wherever you get your podcasts. For more in-depth science news and features, go to ScientificAmerican.com. And if you liked the show, give us a rating or review.

Broderick: For Scientific American’s Science, Quickly, I’m Timmy Broderick.

Drakeford: And I’m Jason Drakeford. See you next time!

[CLIP: Wanda Díaz-Merced speaks in a TED Talk: “ Once there was a star.... Just like everything in life, she reached the end of her regular star days, when her heart, the core of her life, exhausted its fuel. But that was no end. She transformed into a supernova, and in the process, releasing a tremendous amount of energy. ”] 

Timmy Broderick: Okay, Jason, who and what am I listening to?

Jason Drakeford: This is Wanda Díaz-Merced. She is a blind astronomer and a pioneer in astronomical sonification. This is a TED Talk she gave about the massive explosions that stars release when they die. She has done a lot of work capturing these gamma-ray bursts using sound rather than sight.

Broderick: Oh, so she’s, like, the OG of astronomical sonification. Like, all of this, this entire series we’re doing, stems from her and her work.

Drakeford: Yeah, exactly!

[CLIP: Science, Quickly, intro music]

[CLIP: “Flaring Blazar” by Matt Russo] 

Drakeford: You are listening to Scientific American’s Science, Quickly. I’m Jason Drakeford.

Broderick: And I’m Timmy Broderick. In the previous episode of this three-part Fascination, we introduced you to scientists and musicians who are turning comets and galaxies and other stellar goodies into fascinating compositions. Today we’re telling you about the origins of this nascent field.

Drakeford: So, Wanda ...

Broderick: Yeah?

Drakeford: I talked with her earlier this year.

Díaz-Merced: I’m in Paris, working at the Astroparticle and Cosmology Lab at the University of Paris that is part of an institution called CERN [the European laboratory for particle physics near Geneva], and I’m here in the lab.

Drakeford: Yeah, that’s Wanda. She works at the most famous particle accelerators in the world. But for all that she’s accomplished, she’s quite humble. Growing up in Puerto Rico, Wanda had a passion for science.

Díaz-Merced: I always wanted to become a scientist. But to me, the only scientists in the universe were medicine doctors. Studying science meant that you would become a doctor. 

Drakeford: Wanda was diagnosed with diabetes pretty early in childhood and then later with diabetic retinopathy. This can cause blindness in people with diabetes. So when she was in her early 20s in college, her vision started to go.

Díaz-Merced: The condition continued deteriorating until the point when I couldn’t orientate anymore. I needed help. I used to, like, stay in one place all day and not move from there. Already I was using a cane.

Drakeford: For most of Wanda’s undergraduate years, she was focused on being a doctor, even though she was losing her sight—until one day when her friend brought her into his backyard, where he had a small radio telescope as part of NASA’s Radio JOVE project.

Díaz-Merced: This is like an antenna that looks like the wires for you to hang your clothes when you wash your clothes in the summer. So just imagine that but made of copper wires and a little bit fancier.

Drakeford: Radio telescopes can detect radio emissions from several astronomical bodies, such as the sun or Jupiter—which is a very fancy way of saying that Jupiter has radio storms, and we can literally hear them. Like, Jupiter has naturally occurring lasers near its poles that beam radio waves into space. Which is wild! And sometimes we catch them here on Earth.

[CLIP: “Jovian Radio Sounds”]

Drakeford: These “pecks, pops, and crackling swooshes” are what entranced Wanda in her friend’s backyard.

Díaz-Merced: At first I said, “Emilio, why are you listening to that?” because I thought it was an AM radio. And then he said, “No, no, no, Wandita. That is waiting to see if there is any solar emissions.” And then he says that my eyes got big! Like, my, my face changed.

And I, yes—I heard it! Yes, yes!

It was this sense of possibility at that very moment. Then, at some point, he had to say, “Wanda, you have to go to your house. You cannot stay here until tomorrow just sitting by that thing, listening to it.” I didn’t want to detach from it. I began pondering, “What would it be to listen to the data?”

Drakeford: Hearing these Jovian emissions pushed Wanda into astronomy. She worked with the Radio JOVE project, made her way to NASA and completed a Ph.D. Using sonifications, she has even made discoveries that sighted astronomers have missed. 

[CLIP: Wanda's sonification of supernova explosions]

Wanda found that star formation can affect supernovae, which suggests that these explosions are not only dependent on the mass of their host star. Converting the data into sound helped uncover the drop in volume that led to the discovery.

Díaz-Merced: How do I say, I discovered my ability to listen to the data, to listen to, as you call it—I love the way you call it—to listen to the universe, to the phenomena that have been seen in the interstellar medium.

There’s no textbooks available for us. A textbook in astrophysics is like gold dust. It’s like a diamond. It’s like platinum, a yellow diamond this size of my fist.

The scientific revolution developed in a way that just assumed that we wouldn’t participate. It just developed in a way until it got to the point that we had no ways. When I began, I didn’t have any tools to do, perform in the field, no tools, nothing.

Broderick: Her work has inspired other blind astronomers, too.

Enrique Pérez-Montero: My name is Enrique Pérez-Montero. I have two names because, you know, in Spain, we have two names.

Broderick: Enrique is an astrophysicist at the Institute of Astrophysics of Andalusia in Spain. He was not born blind, but a disease called retinitis pigmentosa has made his vision progressively cloudier. He could still see when he finished his Ph.D. But now in his 40s, he continues to study the chemical compositions of the brightest galaxies. His workflow has changed, however.

Pérez-Montero: Ten or 15 years ago, I was able to see them directly in observatories and see the spectra of the universe. And at the moment I am able to deal with the numbers of data [that] telescopes take—just listening, in my computer, these numbers.

Broderick: By using his computer to read out these data aloud, Enrique is able to lead a pretty normal life as an astrophysicist. But it’s clear the field doesn’t know how to react to his disability. Their discomfort is clear whenever Enrique goes to a scientific conference, and other scientists see his guide dog, Rocco.

Pérez-Montero: Even though they are thought to be very intelligent because of the number of papers of contributions or the relationships in projects, they are shocked before the idea that you are blind and that you are an astronomer.

Broderick: Enrique’s disability even helps him analyze data without bias. Other astronomers are ...

Pérez-Montero: Distracted by the beauty of the images. They can get wrong conclusions, maybe because they are just seeing an image. And they are not objectively analyzing what’s the content of the information. And this is one thing I can do because I’m just simply listening: What is the trend of the data, of the very simple cold data, read by my computer?

Broderick: How we choose to represent data can have far-reaching consequences. Astronomy has been associated with sight for centuries, but that does not mean the sense is necessary or even the most useful tool to do the job. It’s ultimately arbitrary, Enrique says.

Pérez-Montero: Ninety-nine percent of the energy and the matter of the universe cannot be seen at all. We can see them because people working with simulations [are] putting out this stuff about dark matter and dark energy. But, of course, this cannot be seen at all, and we can translate it to other ways than images. Images are not the main source to get information about what is the true nature of our universe.

[CLIP: Outro music]

Broderick: In the next and final episode of this series, we head overseas, where a multisensory astronomy festival takes over a small Italian town. Astronomical sonification is a very cool concept—but can it actually inspire people?

Claudia Beschi: I believe that nature has its own sounds. And listening to that sound was as if that galaxy was telling something to me. So it was like this galaxy was describing itself to me.

Drakeford: Science, Quickly is produced by Jeff DelViscio, Tulika Bose, Kelso Harper and Carin Leong. Our theme music was composed by Dominic Smith. Wanda Diaz-Merced and Matt Russo provided the sonifications you heard in this episode.

Broderick: Don’t forget to subscribe to Science, Quickly wherever you get your podcasts. For more in-depth science news and features, go to ScientificAmerican.com. And if you liked the show, give us a rating or review.

Drakeford: For Scientific American’s Science, Quickly, I’m Jason Drakeford.

Broderick: And I’m Timmy Broderick. See you next time!

SUBSCRIBE: Apple | Spotify

Jeff DelViscio: Hi Science, Quickly listeners. This is Jeff DelViscio, executive producer of the show.

The whole podcast team is out in the field, so while we’re away, we’re bringing back a few amazing oldies from the archive.

Today we have a show on a piece of research so electrifying we just had to reanimate it.

Producer Shayla Love brings us a story about research that might change the way you look at bees—forever. 

Bees, as we know, buzz. But they also are buzzing with electricity. Get a lot of them together in the air, and the mass effect of their electric flight could rival the charge in a thunderstorm cloud.

Also, if you covered a car battery with 50 million million bees, you could jump-start it. 

Real facts.

The episode was first aired on November 15, 2022—when we were still called 60-Second Science. Ah, memories. 

Enjoy!

Shayla Love: This is Scientific American’s 60-Second Science. I’m Shayla Love.

When you hear a bee buzzing along, visiting a flower, you’re hearing the movement of air made by the fluttering of its wings. But it turns out that bees are buzzing in more than one way.

Giles Harrison: I first saw this when I saw a bumblebee land on an electrode I was using, and I saw a real change in the measurement. And I thought, “This is a charged thing.” 

Love: That’s Giles Harrison, a professor of atmospheric physics at the University of Reading in England. He’s co-author of a recent paper in iScience that measured the electric charge of swarms of bees and found that the insects can generate as much electricity as storm clouds.

Ellard Hunting: We’ve known for quite a long time already that bees carried an electric charge.

Love: Ellard Hunting is a biologist at the University of Bristol in England, and he studies how different organisms use those electric fields in the environment. Plants and pollen tend to be negatively charged, and bees are positively charged.

Hunting: The bee visits a flower, and the pollen is actually electrostatically attracted to the bee, and so they stick better and they transfer better. 

Love: There are several honeybee hives that are used for research at the field station at the University of Bristol’s school of veterinary sciences. Those bees sometimes swarm, and that’s when the researchers were able to directly measure them using an electric field monitor.

Bees can also electrically sense whether a flower has been visited by another bee who already took its nectar. But until now, it hadn’t been considered that living things flying around in the atmosphere could make an impact with their own charges.

Now, an individual bee’s charge is minuscule: it takes a lot of bees to generate enough electricity to make an impact.

Hunting: Imagine that you need a billion of those to light up an LED. 

Harrison: Fifty million million bees to get enough charge to start a car. 

Love: But altogether, because there are so many insects in the atmosphere, they can have a massive effect.   

This means that bees and other large groups of insects are capable of changing the atmospheric electric fields around them—potentially impacting things such as weather events, cloud formation and dust dispersal.

Insects are not the only living thing that spends time in the atmosphere. Birds and microorganisms carry charge, too, and take up space in the lower atmosphere.

Even before the bees were measured, we knew the sky was filled with electricity. These static electric fields are found everywhere in Earth’s atmosphere. And they can be swayed by rain, lighting, aerosols, pollution, volcanoes and possibly earthquakes.

Atmospheric electricity is measured as something called the vertical potential gradient, or PG, which is the difference in voltage between the surface of Earth and any point in the air. The team found the swarms of bees could change the PG by 100 to 1,000 volts per meter.

They also modeled how atmospheric electricity might be impacted by other insects, such as desert locusts, which can form swarms of up to 460 square miles. These swarms are dense enough to cram 40 million to 80 million of the insects into less than half a square mile. Based on past measurements of locusts’  electric charges, such swarms create more charge than those reported for electrical storms.

Not all insects pack such an electrical punch. In the modeling, moths and butterflies don’t seem to have a big impact because of their low densities.

Right now insects’ electric charges aren't accounted for in climate models that look at complex interactions in the atmosphere. They probably should be: the combined electric charge of all these insects might impact the development of rain, snow, and droplet formation and maybe even how clouds are made.

Hunting: We can only speculate, but, like, that might have an impact on cloud formation. If there’s a direct link between insects and cloud formation, then we know that clouds are relevant to climate. 

Love: Insect electricity could also be influencing how dust moves around the atmosphere. This is something that atmospheric scientists are interested in because such dust cuts off incoming sunlight and can change temperature distributions locally.

Harrison: The link between dust and insects is very interesting because one of the questions in climate change is “How is it that large particles move from the Sahara?” And we just thought about it in terms of the physics of transporting them from the Sahara. What if they’re stuck to a locust because they’re charged? That really changes things, and we could think about it very differently.

Love: After learning about how much of a spark these insects can generate together, it may be time to start taking into account all that extra buzzing up in the air.

For 60-Second Science, this is Shayla Love.

SUBSCRIBE: Apple | Spotify

[Image credit: Stefania Pelfini/La Waziya Photography/Getty Images]

SUBSCRIBE: Apple | Spotify

Hi Science, Quickly listeners. This is Jeff DelViscio, executive producer of the show. 

The whole podcast team is out in the field, so while we’re away, we’re bringing back a few amazing oldies from the archive.

Today we dive into your brain during bouts of intense learning—maybe that happens to you when you listen to this podcast?

Producer Karen Hopkin brings us a study that looked at brain training—and how rest might be the key to training your brain even faster. 

The episode was first aired on July 21, 2021, when we were still called 60-Second Science. Ah, memories. 

Enjoy!

--

Karen Hopkin: This is Scientific American's 60-Second Science. I'm Karen Hopkin.

They say that practice makes perfect.

[CLIP: Piano sounds]

Hopkin: But sometimes the best practice is not on a keyboard ...

[CLIP: Piano sounds]

Hopkin: It’s all in your head. Because a new study shows that the brain takes advantage of the rest periods during practice to review new skills, a mechanism that facilitates learning. The work appears in the journal Cell Reports. [Ethan R. Buch et al., Consolidation of human skill linked to waking hippocampo-neocortical replay]

Leonardo Cohen: A lot of the skills we learn in life are sequences of individual actions.

Hopkin: Leonardo Cohen of the National Institute of Neurological Disorders and Stroke, or NINDS.

Cohen: For example, playing a piece of piano music requires pressing individual keys in the correct sequence with very precise timing.

[CLIP: Piano music]

Hopkin: That level of virtuosity requires a ton of practice and a lot of repetition. But Cohen says it also requires a certain amount of rest.

Cohen: We know from previous research that interspersing rest with practice during training is advantageous for learning a new skill. In fact, we recently showed that virtually all early skill learning is evidenced during rest rather than during the actual practice.

Hopkin: It’s during those intermittent breaks that the brain starts to sew together the individual movements that make up a seamless piece.

[CLIP: Piano music]

Hopkin: The question then becomes: How? To find out, Cohen and his colleagues turned to an imaging technique called magnetoencephalography, or MEG.

Ethan Buch: The unique advantage of MEG is that it allows us to observe neural activity across the entire brain with millisecond time resolution, which is crucial for investigating very fast widespread brain dynamics.

Hopkin: Ethan Buch, Cohen’s colleague at NINDS. They and their team had 30 volunteers sit inside an MEG scanner. And they asked them to type the sequence 41324 on a keyboard as quickly and accurately as possible.

[CLIP: Sounds of typing]

Hopkin: The participants would type for 10 seconds, then rest for 10 seconds, then repeat while the researchers monitored their neural activity.

Buch: And what we found was really quite interesting. So we actually found that the brain kept replaying much faster versions of the practice activity patterns over and over again during rest.

Hopkin: So a sequence that might take one second for fingers to type ...

[CLIP: Sounds of typing]

Hopkin: ... would take just 50 milliseconds for the brain to replay.

[CLIP: Sounds of typing]

Buch: That’s an impressive 20-fold compression.

Hopkin: The regions most active were those involved in controlling movement and representing sequences. And the more often the brain repeated the sequence, the faster the subject improved.

Buch: When the participants were beginning to learn the skill, they were initially typing about five to six repetitions of the sequence during each 10 seconds of practice. But during rest, the brain replayed about 25 repetitions of the sequence, and that’s a fivefold increase over the same amount of time.

Hopkin: That lightning-quick neural rehearsal supercharges learning and memory.

Buch: It’s as if the brain actively exploits these rest periods to amplify the effects of practice and rapidly consolidate the skill memory. And this actually appears to be this skill-binding mechanism that we were looking for.

Hopkin: So next time you sit down to practice ...

[CLIP: Piano music]

Hopkin: Give yourself a break—or a lot of little breaks. Your brain, and your audience, will thank you.

[CLIP: Piano music]

Hopkin: For Scientific American’s 60-Second Science, I’m Karen Hopkin.

SUBSCRIBE: Apple | Spotify

[Image credit: Andriy Onufriyenko/Getty Images]

SUBSCRIBE: Apple | Spotify

Sophie Bushwick: Exciting new technology is on the rise, doing the same work that humans can but faster and more cheaply. That's great news for the people rich enough to buy these machines, but it's disrupting the lives and livelihoods of less powerful workers. And some of those workers are ready to fight back.

I'm talking, of course, about the Luddites, the 19th-century textile workers known for smashing automated machinery. And if you see some similarities between their situation and today's, when the rise of Big Tech and artificial intelligence is disrupting the labor market and also inspiring a bit of a backlash, well, you're not the only one. 

And this is Tech, Quickly, the loom-smashing but tech-loving edition of Scientific American’s Science, Quickly podcast. I’m Sophie Bushwick, tech editor at Scientific American.

Brian Merchant is the tech columnist at the L.A. Times and the author of a new book called Blood in the Machine: The Origin of the Rebellion Against Big Tech. Blood in the Machine tells the story of the Luddites and what their plight tells us about automation today. 

And I've heard their movement mentioned a lot recently, particularly in relation to generative AI. For instance, news recently broke that copyrighted books, including those by Steven King, were used to train an AI language model. And King wrote an essay for the Atlantic in which he says he wouldn’t forbid this type of thing, and that anyone who would might as well be: “a Luddite trying to stop industrial progress by hammering a steam loom to pieces.” This idea that Luddites just hated new technology is pretty pervasive. But is it accurate?

Brian Merchant: It is not accurate. It's one that's been lodged in the cultural consciousness for almost 200 years now though, so it's hard to blame him. I love Stephen King. I love his writing. I love his writing on writing. But he's fallen victim to this fallacy about the Luddites, which is that they were technophobic, they wanted to stop progress itself, which couldn't be further from the truth. 

What the Luddites wanted to do was to stop the machinery that was very specifically exploiting them or being used as leverage against them to reduce their quality of life, to cut their wages, to force them into factories. 

So, the Luddites protest against machinery, again against very specific kinds of machinery. They were technologists; they loved technology themselves in many many cases. But they had an issue a specific machinery being used in specific ways, namely by elites who wanted to force them into factories and degrade their standard of living.

Bushwick: Would you say that their, um, attempt to rebel against this succeeded or failed?

Merchant: That's a really complicated question—it was illegal to form a union in those days. So they were forced to use very creative tactics to sort of register their complaint, and when they did take up arms in this very specific way for the first six months or so they were very successful in the short term in getting some of those, ah, those wages raised, getting their conditions improved, getting back some of the bargaining power. In the medium term, they were less successful because their movement was violently crushed by the British state and by the Crown and by the industrialists who the crown was assisting. Dozens of Luddites were hung. Others were killed in those protests. 

And then the victors of that battle got the opportunity to impress upon history this idea that Luddism is backwards-looking, establishing what's basically a propaganda campaign on behalf of those industrialists to equate progress with any kind of technology at all, even technology that could exploit hundreds of thousands of people as it did.

Bushwick:  And we're in a moment now when technology is again threatening people's jobs. Specifically there have been a lot of stories about generative AI being used to replace copywriters, the idea that it could threaten the jobs of software coders, that instead of hiring an illustrator, places are just going to use this AI-generated art. So can we draw any parallels? What parallels can we draw between the situation the Luddites were in and then the modern automation threatening jobs today?

Merchant: The way that I would put it is that the parallels are uncanny.

So much is aligned with what what was going on at the Luddites' time that we can really sort of go one to one in a lot of cases. So, here's this revolutionary-seeming new technology, generative AI, that its creators are claiming have vast power, can replace vast numbers of jobs. That was very similar to what was happening 200 years ago. 

You had the entrepreneurs adopting things like the power loom or the wide frame or the implements that would automate various parts of the cloth industry, saying, this is this new machinery is going to be a great boon to England; it's going to be an engine of progress. 

In reality, again, this is a parallel, neither of the technologies were quite there yet, right? And so, in a lot of cases, the technology could be used more powerfully by the entrepreneurs or by the industrialists as leverage or bargaining power or a means of sort of saying hey we need to reduce wages because we're just going to use the machinery to do it anyways.

Bushwick: And you mentioned how when the Luddites were doing their protests, they didn't have unions, they didn't have the right to do this sort of collective action. As opposed to today, when we have, for the first time in decades in Hollywood, writers and actors–they're all going on strike. Not solely because of AI, but that's definitely one of the issues that they're trying to deal with as part of these, ah, strikes. So do you think that they are more likely to succeed than the Luddites were?

Merchant: Absolutely. I mean I think the crucial difference is that, you know, this is an industry that is very well organized, that has a good deal of power. And they, they, they can force their employers to the bargaining table to hash out these issues. We should be thankful that they have sort of sounded this clarion call because in a lot of ways, you know, the same issues are going to be coming for other industries. 

So the writers and the artists stand to be more successful for that reason alone—that they are organized, and it's popular. Again, I think I don't know if I mentioned this, the Luddites were super popular in the day. They were like Robin Hood. They were cheered in the streets because just about everybody understood the way that the wind was blowing.

Bushwick: And it's not just in the case of the writers strike that I've heard the term Luddite used. Another example is that in San Francisco right now, there's a big rollout of these self-driving robo taxis. And there have been a lot of issues coming up around this, but one of them is a group has been protesting them by putting traffic cones on the hoods of these cars because it tells the censors that there's an obstacle, there's something wrong, and so the car stops with no damage to it, but it is a form of protest. And some of the protesters have been compared to Luddites. Now the issue for them is not their own employment. But they are protesting this technology. Do you think it's still fair to say that this is a sort of Luddite movement?

Merchant: Yeah, I think this is a Luddite tactic through and through and not in the derogatory sense, in the sense that these are people who are standing up for their community when democratic channels have been sort of ignored. I mean, if you look at the polling data and you look at the general sentiment, people don't want driverless cars running rampant in their communities.

We've seen very viscerally how dangerous it can be, how deadly self-driving car technology can be in some cases and what a nuisance it can be to things like emergency service providers and firefighters, policemen. They've all come out against this and said this: not yet. And it's unclear to most people why it needs to be rammed through, why the experimentation period can't be longer or can't be more careful.  So if we can't get sort of the bare level of assurances and transparency from government, when new technologies are rolled out, I think it's completely justified for people to basically do Luddism.

Bushwick: And you've mentioned that the Luddites in the short-term were successful, but in the long-term you know they were the subject of a propaganda campaign. We've got Stephen King being like, ah, being a Luddite is is futile. It's, it's, it's resisting technology for no purpose. And that's the idea that many people still have hundreds of years later. So, if you're if you were going to give advice to modern Luddites, what do you think they need to do to avoid the same fate?

Merchant: You know I think we're getting to the point to where people have sort of at least, ah, loosened their, ah, knee-jerk sort of reaction in that direction. We're kind of at this point again where the negative effects of technology in a lot of spheres, whether you're talking about in the working world or social media, overstimulation, we're seeing a lot of the detrimental effects of technology, and we're seeing sort of a resurgence of this desire to sort of be able to offer more input and more, ah, have more say and how it's, ah, how it's rolled out.

We've sort of handed the keys to a handful of Silicon Valley giants and said, “Okay, you, you see what you can come up with, and then we'll kind of deal with it, we'll take the rearguard action,” but it does not have to be that way. And that's what I want to sort of beat home: the Luddites represented an alternate path.

If we look at what they really stood for it really comes down to a more just, more equitable, and more democratic development of technology, one that more people would have benefited from. How do we do that today?  

Bushwick: Science, Quickly is produced by Jeff DelViscio, Tulika Bose, Kelso Harper and Carin Leong. Our show is edited by Elah Feder and Alexa Lim. Our theme music was composed by Dominic Smith.

Don’t forget to subscribe to Science, Quickly wherever you get your podcasts. For more in-depth science news and features, go to ScientificAmerican.com. And if you like the show, give us a rating or review!

For Scientific American’s Science, Quickly, I’m Sophie Bushwick.

SUBSCRIBE: Apple | Spotify

[Image credit: Encyclopaedia Britannica Films/Getty Images]

Tanya Lewis: I’m standing on the rooftop of NYU Langone Health, a hospital in midtown Manhattan, scanning the sky over the East River for a helicopter. It’s New York City, so there are tons of helicopters, but I’m looking for a specific one.

[CLIP: Helicopter sounds]

Finally I spot it: a white helicopter with red-and-white blades, carrying some very precious cargo. The blades whir to a halt, and some people step out. One man is carrying a big white box. Inside the box? A pig kidney.

[CLIP: Show music]

This is Science, Quickly. I’m Tanya Lewis.

I run back into the hospital to watch the small team arrive through the elevator doors wheeling the box with the kidney on ice. I follow them right up to the operating floor.

[CLIP: Organ being transported]

And that’s as far as I can go. Only surgeons and nurses and a handful of observers are allowed in the operating room. There’s a risk of the pig organ carrying a virus that could infect people. And they are scrubbed in with personal protective equipment on. The surgical staff also had to undergo special blood testing before and after the surgery to ensure they stay infection free.

So I make my way back to a small room with a live video feed of the operation. 

[CLIP: Xenotransplant control room]

Lewis: I’m here to watch an organ transplant. But not just any transplant: a pig kidney is being transplanted into a human. This is known as a xenotransplant.

The human recipient is not alive—he suffered brain death as a result of a complication from a cancerous brain tumor. The body of the decedent, as researchers refer to him, is being maintained on life support. He wasn’t able to donate his organs, but his family has generously agreed to donate his body for the experiment.

I watch the live video feed of the surgery. The surgeon starts prepping the organ to be transplanted.

Once it’s ready, he lowers it into the decedent’s abdominal cavity. He has removed the decedent’s own kidneys already, so the only kidney function will come from the pig organ.

The surgeon carefully sutures the pig kidney’s blood vessels to the deceased person’s renal artery and vein.

He also connects the ureter—the tube that connects the kidney to the bladder—from the pig organ to the human ureter. The kidney flushes pink as blood begins to flow through it and starts producing urine.

Montgomery (tape): The total ischemic time was..., which is good. And it worked right away and started to make urine almost immediately....

Lewis: That’s the transplant surgeon, Robert Montgomery.

Robert Montgomery: So my name is Robert Montgomery, and I'm the chair of the department of surgery at NYU Langone health, and the director of the transplant Institute.

Lewis: The surgery is done now. Here he is leading a debriefing with the transplant team at the decedent’s bedside.

Montgomery (tape): So we did the xenotransplant and we did a bilateral native nephrectomy. So all the urine that's coming out is coming from the xenograft, and that will continue to be the case.... So we're off to a good start....

Lewis: Montgomery knows a thing or two about transplants. Not only has he done thousands of kidney transplants, he himself had a heart transplant several years ago.

Montgomery: I had a heart transplant, it's almost five years, next month, it's five years. It's just extraordinary. And I was in really bad shape. I had seven cardiac arrests where I had to be resuscitated. Then one where I was down for a very long period of time getting CPR and I was just so lucky to recover from that. And one time, where I was in a coma for a month after I had an event. So I beat all the odds for sure.

Lewis: Montgomery suffered from a type of congenital heart disease. It claimed the life of his father. It also took one of his brothers, who was just 35 years old when he died. His other brother got a heart transplant 26 years ago.

Montgomery: So, you know, this has been with me, my whole life. I've been a patient as long as I've been a doctor, and I've had a lot of time to think about this. And certainly, it's influenced my life as a transplant surgeon tremendously. And I'm always looking for the next big thing that could make a big difference in the lives of people who need transplants.

Lewis: Montgomery wants to create a virtually unlimited source of organs for people who need them. That’s why he got interested in doing xenotransplants.

Montgomery: So, organ supply is our biggest unmet need. There are over 100,000 people waiting for organs, and only about a third of them will actually make it across the finish line. And everyone else will either become too sick to benefit from a transplant or will die waiting. 

And it doesn't even take into account all the people who die before they get listed for transplant because the allocation of organs is based on severity of illness, and you have to get really sick before you're even considered to get put on the transplant list.

Lewis: Currently, all transplanted organs come from either living or deceased human donors. But very few people die in a way that their organs could be used for transplants. 

Pig xenotransplants offer a way to potentially give more people a chance to get an organ that would save their life.

Montgomery: So the current paradigm of somebody having to die for someone else to live is not working. And we need an alternative, sustainable renewable source of organs and xenotransplantation is the most promising right now.

Lewis: The pig organs used in these transplants have been genetically modified so they don’t produce a sugar called alpha gal, which is found in pigs and causes the human immune system to attack, or reject, the organ.

Rejection is already a problem for human-to-human transplants, but pig organs pose a special challenge.

Montgomery: Two humans are much more alike than a human and a pig, right? So you can imagine, just the variation in what we call antigens, different carbohydrates and proteins that–the genetic code is just really different between a pig and human. There's a lot of potential targets for the human immune system—less so when you have two humans.

Lewis: I asked Montgomery what it was like to actually do this xenotransplant, knowing how much was at stake for the field if it didn’t go as planned.

He’s a pretty calm person—just as you’d expect a transplant surgeon to be.

Montgomery: When you're doing something that hasn't been done before, you do feel this tremendous weight. And, you know, it's very clear that we're not going to have many shots on goal here that don't end in success. I'm very aware of that. But I've been at this for 35 years, and once I get in the operating room…and it is an experience, where everything just kind of collapses down onto the task at hand. And any thoughts or doubts or anything that you have in your mind quickly go away, and you're just focused on getting the job done.

Lewis: This was actually the fifth xenotransplant he and his colleagues at NYU have done using a decedent model.

Montgomery: Up until 2021, all of the xenotransplants from genetically edited pigs had been put in primates. So there had never been a pig-to-human transplant that had been done. 

And we did the first one in September of 2021. And we put that organ into an individual who had been declared dead by neurologic criteria, so they were brain dead. But the heart was still beating, and that individual was being maintained on a ventilator. And the idea there was that we had an opportunity to test one of these organs in someone who couldn't be harmed if things didn't work out well. 

Lewis: NYU soon did another kidney xenotransplant, and a team at the University of Alabama did one as well. Then NYU did two heart xenotransplants. But these previous experiments only lasted a few days before the decedent was taken off of life support. The University of Alabama just did another kidney transplant experiment that lasted a week.

But this one has been going for more than a month, and they just got approval to extend the experiment for up to another month.

Back in early 2022, a team at the University of Maryland transplanted a pig heart into a living person—a man named David Bennett. Bennett lived for two months with the new organ, but ultimately it failed and he passed away.

The exact reason for the failure is still unclear, but a previously undetected pig virus may have played a role.

Montgomery: So there's still, you know, a lot of mystery in terms of what happened. But what we do know is the heart suddenly stopped working. One of the things that was present at that time was an infection, a virus called cytomegalovirus, CMV, that came from the pig and stayed in the pig tissue and never infected any of the human tissue or cells. 

But we know from decades of primate work, when the organ ... when the pig heart is infected with that virus, that it causes the heart to start to go rapidly downhill. So, we think that played a role, but there also appeared to be some rejection as well.

Lewis: Montgomery and his team take these risks seriously. They have developed a more accurate test to screen for pig viruses, and they monitor the recipient and anyone who comes into contact with them very closely.

Decedent experiments like the one they are currently doing will be crucial to showing the FDA that these pig organs are safe and effective enough for clinical trials in living people.

NYU hopes to be one of the first sites to participate in such a trial.

For Montgomery, it’s a personal mission as much as a professional one.

Montgomery: As I laid for a month in the ICU waiting for a human organ, it came even more clearly into focus that this is what I needed to devote the rest of my life to. I kind of feel like I'm living on borrowed time anyway. And this, has become the thing that I really feel is a chance at really changing the future in a really significant way for all the people who ... I share their level of, of desperation, you know, and understand it, when you're waiting for a transplant, it's a really extremely vulnerable and difficult position to be in.

Lewis: Science, Quickly is produced by Tulika Bose, Jeff DelViscio, Kelso Harper, Carin Leong and me. It’s edited by Elah Feder and Alexa Lim. Our music is composed by Dominic Smith.

Subscribe wherever you get your podcasts. If you like the show, give us a rating or review!

For Science, Quickly, I’m Tanya Lewis.

[Image credit: Joe Carrotta for NYU Langone Health]

SUBSCRIBE: Apple | Spotify

[CLIP: Cricket sounds]

Jacob Job: The night skies have fascinated humans for as long as we have been around. Celestial bodies have become actors in our myths and folklore. And from the stars and heavens, we draw inspiration and even religion.

But the night skies have also taught us how to keep time and coordinate our days and seasons. 

We have long used the night skies to live our lives more predictably and make our way through the world more purposefully.

Job: But we’re not the only ones.

I’m Jacob Job, and you’re listening to Scientific American’s Science, Quickly. Today, part four of our five-part series on the Nighttime Bird Surveillance Network—an informal but important global audio dragnet that tracks some of the billions of migratory birds as they fly through the night.

Benjamin Van Doren: If you’re lucky, and it’s a clear night, and the moon is illuminated, there are so many birds migrating on average on these nights that if you use a telescope and look at the moon for a few minutes, you’re likely to see a bird high overhead flying and silhouetted in front of the face of the moon.

Seeing birds fly in front of the moon or hearing the calls from above, I found really thrilling because it felt like I was tapping into this vast mysterious pulse-of-the-planet phenomenon that was just so much bigger than me.

[CLIP: Theme music]

Job: Migratory birds navigate to their summer and winter homes by way of the moon and stars. On any given night during migration, there might be thousands of birds flying in the skies above you and tens or hundreds of millions more moving across the continent.

We still don’t fully understand the true scope of this mass movement. But now science is turning to machines to unlock the secrets of nocturnal migration.

Job: The nighttime bird surveillance network all started with one six-foot sound dish, an expensive studio microphone on reel-to-reel tape and a bunch of hay bales more than 60 years ago. In time, the mics got a lot smaller, and the network grew and grew.

Today people all over the world have created a vast, informal network of night sky listeners.

Interpreting all of these data, however, has created a new challenge.

It’s one that scientists at the Cornell University Lab of Ornithology are tackling head-on.

Van Doren: My name is Benjamin Van Doren. I am a scientist at the Cornell Lab of Ornithology.

Job: Benjamin grew up in New York State and is a postdoctoral research fellow who studies the science of bird migration. His interest in birds goes back a long way, about as far back as his ties to the Lab of Ornithology.

Van Doren: I particularly got into birds when I was about eight years old in third grade, and that was thanks to a classroom program that involved watching birds at bird feeders outside the classroom and recording what we saw and actually submitting our data to the ornithologist at the Cornell Lab of Ornithology, which I thought was the coolest thing.

I was really intrigued by the puzzle of bird identification, that I could learn the tricks to identifying a bird and then be able to put that knowledge to use outside in the real world when I saw a flash of color go by or, later on, heard a sound.

Job: During high school, his interest grew deeper.

Van Doren: I also started to get really interested in bird migration, and for a bird-watcher, migration is a really exciting period of the year because each day can bring an entirely new set of birds or species to your local park or even your backyard—birds that are in the middle of these long journeys of thousands of miles.

Job: But what really sealed Benjamin’s fate as a nighttime bird migration fanatic was a talk he attended at the lab on, you probably guessed it, nocturnal migration.

Van Doren: I was captivated by that. And the additional layer of mystery is that songbird migration in large part occurs at night, so it’s also literally shrouded in darkness.

Job: He decided to start monitoring birds on his own.

Van Doren: I ended up starting a research project in high school that included making audio recordings and looking at radar data. This was a whole nother level of experiencing something that was hidden to so many other people, so I really found that exciting, and I’m still pretty much doing that today.

Job: And now he’s busy solving the puzzle of how to quickly and accurately analyze thousands of hours and many terabytes of nocturnal migration data.

But more people are joining the nighttime surveillance network. More and more data are flying in from nighttime listening stations. Benjamin is one of the few researchers trying to get arms around all of it.

And he has a big data problem on his hands.

A single night of recording produces anywhere from eight to 12 hours of audio that is about one to three gigabytes in size. And that’s only at one location. Hundreds of people are recording migration all across the U.S. and beyond. Nightly audio intelligence from the network then needs to be combed through to find moments where migratory birds emitted nocturnal flight calls, or NFCs, above the microphone. During especially busy nights, that can mean more than 20,000 NFCs in a single recording from one site.

Then comes the problem of deciphering those calls.

Van Doren: Flight calls are very short vocalizations. They might last a fifth or even a twentieth of a second, and so it takes a lot of skill and practice to learn how to identify these calls, especially by ear—something I personally don’t feel that confident at. And processing hours and hours of passive audio recordings can be extremely tedious and difficult.

Job: And if you’re in the business of studying migratory birds like Benjamin and other scientists at the Lab of Ornithology are, that can get overwhelming.

Van Doren: We have hundreds, thousands, maybe tens of thousands of hours of recordings that we may want to analyze, which is just way too much for the small number of skilled humans who can do this sort of thing by hand.

So ... we really need computers to do some of the work for us to be able to get any useful, large-scale information out of these long passive audio recordings. And so that’s why we turn to machine learning.

Job: Machine learning, a type of artificial intelligence, is something we’ve all been hearing a lot about lately. But really, it’s been around for quite some time.

Van Doren: Machine learning describes such a vast array of computational tools that it is really everywhere in our lives, everywhere from credit-card-fraud detection to facial recognition to my phone suggesting which apps to open at a certain time of day.

Job: I mean, even as I wrote this episode, Google Docs suggested, oftentimes correctly, the next word or phrase I planned on typing. That’s possible because of machine learning. But Benjamin uses a specific type for bird call data.

Van Doren: One area of machine learning is computer vision, which we can use to distinguish dogs from cats in pictures, for example—or, in our case, distinguish different birds on audio recordings by feeding the computer not the raw sound itself but actually the visual representation of sound as a spectrogram, really a picture that represents the sound.

Job: Basically the spectrograms Benjamin is referring to are visual fingerprints of nocturnal flight call audio.

Van Doren: We feed the computer these spectrograms that we’ve classified as one species or another, and then, as we repeat that thousands and thousands of times, the computer learns to be able to distinguish one species from another.

Job: So if we want a computer to learn how to identify dogs in pictures, you feed it many thousands of images of different dogs: small ones, large ones, dogs of all colors, dogs in different poses. Eventually, after enough training like this, the computer can recognize if a dog is present in most any picture you present it.

Van Doren: If we train our models well enough, it can give us an accurate, or at least useful, estimate of the numbers of birds that were passing overhead, which species they were. And so very quickly, perhaps 300 [times] faster than real time, we can begin to process thousands of hours of audio in an efficient manner with these kinds of tools.

Job: If you’re a birder, you may already be familiar with this technology. The Lab of Ornithology added a feature to its Merlin Bird ID app called “Sound ID”.

Essentially, if you hear a bird outside and want to know what it is, you can tap this feature in the Merlin app and point your phone’s microphone toward the singing bird. After analyzing the spectrogram of the bird’s song, the app spits out its best guess as to what species of bird is singing in front of you.

It’s highly accurate and really useful, kind of like Shazam for birds. I asked Benjamin if people could use this feature to identify NFCs.

Van Doren: The tools and underlying technology behind something like Merlin, they can be applied to the flight call problem, this flight call challenge, but as you say it is trickier because there’s less information encoded in a 50-millisecond chip than in [a] several-second song of a Northern Cardinal, for example.

So it makes it more difficult for the computer to make the identification accurately, but when you provide enough data, the computer gets good enough that it can hopefully overcome those types of shortcomings.

And so one thing that I’m working on right now is trying to take that next step to develop a system based on some of the same technology that underlies Merlin but applied specifically to the challenge of identifying nocturnal flight calls.

Job: Benjamin is hopeful this tool will be available in the not too distant future. And by creating such a tool, all of a sudden, the problem of ID’ing hard to identify night calls slowly begins to disappear.

And this could help unlock the greatest mysteries of migration.

Van Doren: We currently have a poor understanding of what birds are doing at the species level when they’re actively migrating, and flight calls can give us a window into how birds are interacting with the landscape, how they’re interacting with human-dominated areas like cities, and importantly, tell us how different species are behaving differently in response to the environment, to the landscape and also with each other.

I think there’s a lot that we have yet to learn about how birds are interacting during migration. They’re saying something up there, and in my view, there’s a lot we don't understand about what exactly they’re communicating as they are participating in these journeys of thousands of miles. So being able to monitor birds at such a large scale will provide us the kind of information we need to make informed conservation decisions going forward.

[CLIP: Theme music]

Job: And that’s where we’re headed in our final episode on the Nighttime Bird Surveillance Network. We illuminate some of the threats birds face during migration and how scientists are combining weather radar and night call monitoring to aid migratory bird conservation efforts.

Kyle Horton: It’s really important for us to monitor those passages, especially in a time where birds are facing many different threats.

Things tend to not look amazing for migratory birds right now.

Job: Science, Quickly is produced by Jeff DelViscio, Tulika Bose and Kelso Harper.

Don’t forget to subscribe to Science, Quickly. And for more in-depth science news, visit ScientificAmerican.com.

Our theme music was composed by Dominic Smith.

For Scientific American’s Science, Quickly, I’m Jacob Job.

SUBSCRIBE: Apple | Spotify

Meg Duff: The town of Petersham, Massachusetts is leafy. It’s green. It is not the business capital of anything. It’s not a place anyone associates with cutting-edge economic research. But there is a research forest here, where scientists study the economic dynamics of forest ecosystems. And at the edge of the forest, there’s a little greenhouse on a hill.

[CLIP: Footsteps]

Duff: This is where I came to learn about the surprising economic actors in a hidden economy that we are still just beginning to understand. This economy is being reshaped by climate change … and without it, we might not even be alive.

My name is Meg Duff, and you’re listening to Science, Quickly.

[CLIP: Intro music]

Jenny Bhatnagar: These are saplings, oak saplings, and we’re planting them in pots. We’re doing a big greenhouse experiment.

Duff: That’s Jenny Bhatnagar, an associate professor of biology at Boston University. She and her colleagues are in the Harvard Forest greenhouse setting up an experiment to study an underground economy. And when I say underground I do mean that literally, because under our feet, plants and fungi are constantly trading.

Bhatnagar: Oh, the trees provide the fungus with sugar. And in exchange, the fungus provides the tree with nutrients like nitrogen, phosphorus, calcium, sulfate, water, etcetera. 

Duff: A fungus is like a mullet. As Jenny put it, there’s a business end and a party end. 

Bhatnagar: The mushroom is the party end.... It only makes a mushroom when the conditions are right.

Duff: But I’m here to see the business end: these tiny underground threads that run throughout the soil, collecting nutrients.

Bhatnagar: So I’m opening up a cooler.... So this is a cooler full of soil.... And look, see all that white?

Duff (tape): Wait, that little ...

Bhatnagar: That white is fungus. It’s not plastic.

Duff (tape): It looks like plastic!

Bhatnagar: It’s not. And you can see they grow on the tips of the roots. See, right there..., see this yellow? That’s an ectomycorrhizal fungus that’s colonizing the roots of the oaks.

Duff: Mycorrhizae are these long threadlike fungi that connect to the roots of plants. This network is often called the “wood wide web” because it facilitates communication in the forest. But there is also an economic relationship between plants and these fungi: During photosynthesis, plants collect carbon from the atmosphere. And some of it, they trade it to fungi.

Bhatnagar: Tree roots are not very good at getting nutrients and water for themselves. 

Duff: Because of that, many trees trade with fungi to get resources they can’t otherwise reach. Jenny says that if they don’t have as many fungi to trade with, trees don’t do as well: they’re often smaller, less resilient to stress and less likely to survive. So the experiment Jenny’s working on is about trying to get more mycorrhizal fungi into urban soil. Yeah.

Duff (tape): So then these little sidewalk trees ... 

Bhatnagar: They don’t have a lot..., and so we don’t know. We don’t know how the trees are able to live in the city.... We think they grow fast..., but then they die young. 

Duff: Because they don’t have as many fungi to trade with, city trees live more of a subsistence lifestyle. Forest trees just have more resources. Or—they have had, for most of the time forests have existed. But recently, their “economy” has been changing, too. And unfortunately, it’s been changing in ways that will probably feel really familiar—because trees, like us,  have been experiencing inflation.

[CLIP: Music]

Renato Braghiere: The fungi are interested in the carbon that plants produce, and the plants will pay out this carbon to the fungi, and in turn, the fungi will mobilize, searching for nutrients, and return these nutrients to the plants. 

Duff: To learn more, I called up Renato Braghiere, a researcher at the California Institute of Technology and at NASA’s Jet Propulsion Laboratory, who has been modeling the plant-fungi economy.

Braghiere: It’s a win-win situation.... We can see carbon as this currency that plants use to benefit from fungi ... 

Duff: Except for one thing: For the past few hundred years, humans have been burning fossil fuels—filling the air with more carbon for plants to capture. But the fungi don’t always have more nutrients to trade.

Braghiere: Eventually the fungi will scavenge soil looking for nutrients, and they will just find it’s harder to gather the same amount of nutrients that the plants are requiring. 

Duff: Since the supply of nutrients can’t keep up with demand, fungi are raising their prices. And even though plants have more carbon to spend, it’s just not going as far.

Braghiere: We’re kind of causing this inflation into this trade that has been working for so many years.

Duff: You heard that right. Like us, plants are experiencing inflation. For a few million years, carbon dioxide levels in the atmosphere were pretty stable. But since the industrial revolution—and especially in the past few decades—humans have added lots more, essentially devaluing the plants’ currency. To oversimplify, there are really two options for what happens next. Just as in the human world, the plant economy could course correct or it could crash. Obviously, the crash scenario is not great for the plants.

Braghiere: Because they don’t have nutrients, the photosynthetic rates will decrease. 

Duff: Like Jenny’s street trees, forest trees may begin to grow more slowly, reproduce less often and then die young. That’s also bad for the fungi because they get less carbon. And it’s really bad for us, too, because we benefit when forests store carbon.

Braghiere: So one third of ... the atmospheric CO₂ that we put up there gets absorbed by the land. And if the system crashes, this, this fraction, third, can go down.

Duff: By absorbing our carbon dioxide, plants and fungi have actually been helping to slow global warming. That’s why planting trees is such a popular climate solution. To use an economic term, the land sink for carbon is one of the things we factor into our global “carbon budget”—which helps us decide how much carbon we can burn without overshooting climate goals. And Renato says that if it weren’t for this inconvenient problem of inflation ...

Braghiere: We would have plants assimilating more and more and more carbon forever, and we will just see a very, very strong sink of carbon in the land surface. 

Duff: But, he says, that’s probably not what we should expect. Nutrient limits, along with other challenges, like droughts and fires, paint a different picture.

Braghiere: From the end of the century on, it seems like projections are saying that this productivity will start to decrease. And eventually... the land can turn into a carbon source instead of a carbon sink. And then the feedback into the climate system will just amplify and accelerate climate change, which will be a disaster. 

Duff: So that’s what the models say right now. But there’s still a lot of uncertainty.

Braghiere: Like, inflation in economics is really hard to predict.... The future is uncertain for mainly two different reasons. There’s the uncertainty in the processes that we represent in these models. But there’s also the uncertainty in the pathways that humans will take. So we might cut emissions by 2030, and then the climate system would take other pathways. 

Duff: If humans keep burning fossil fuels and printing more money for the plants, we are making the “crash” scenario a lot more likely. But we still don’t know how the plants and the fungi will respond.

Braghiere: Yes, we’re expecting that the system will crash.... It’s also important to say that nature has this incredible capacity to adapt.

Duff: There are a few different scenarios that could play out. Among the millions of species of fungi, there may be winners and losers. But some may actually do really well with different carbon prices. Best case scenario, those fungi help forests adapt.

Braghiere: Because now the price of carbon nutrient is different, one species of fungi can benefit from a different price.... We might see a shift in the composition of different types of fungi that associate with different types of plants. 

Duff: But those changes may not come quickly enough. And if those plant fungi partnerships change, that could also change these economies in other ways too ...

Braghiere: That could have a cascading effect to the entire biodiversity of that ecosystem as well. 

Duff: Here’s the annoying thing, though: it’s really hard to get good data on underground economies. And that’s even more true when the economy is actually underground—when it’s all happening under a layer of dirt. Right now Renato is extrapolating from a few research forests, like the one I visited. The problem: these forests are mostly near well-funded universities in the U.S. and Europe. So tropical rainforests are underrepresented.

Braghiere: So, at the moment, we set one carbon nutrient price per mycorrhizal type all across the world, but we might just end up with extra data realizing that ... in one part of the globe, the symbiotic relationship has a different cost than other parts of the globe. 

Duff: Right now Renato’s models use some very back-of-the-envelope assumptions about what’s going on under the soil. And he thinks a crash is by far the most likely scenario. But to be certain, we need better data on which fungi are where and how their relationships are shifting.

In the next episode, we’ll explore how researchers are getting those data. Because, as it turns out, they are in fact mapping these nearly invisible underground fungi. Here’s the wild part: now, they’re figuring out how to do that from space.

For Scientific American’s Science, Quickly, I’m Meg Duff.

Science, Quickly is produced by Tulika Bose, Jeff DelViscio and Kelso Harper. Edited by Eleh Feder and Alexa Lim. Music by Dominic Smith.

[CLIP: “A fair casting process: AI protection and fair compensation!”]

Sophie Bushwick: I’m sure you know what that is. 

Tulika Bose: Yeah, for the first time since the 1960’s—both Hollywood actors and writers are on strike. 

[CLIP: “Now striking Hollywood actors hold their largest demonstration to date, and it happened in Times Square.”]

Bushwick: At the same time!

Bose: At the same time.

Bushwick: Then and now, the problem is about technology disrupting the world of Hollywood...

Bose: And the question of whether actors and writers are being paid fairly. 

[CLIP: “We need to come to the table in good faith.”]

Bushwick: In the 1960s, the technology was television. 

[CLIP: “Here comes Flipper!”]

Today it’s streaming platforms—but also the rise of powerful new artificial intelligence models. 

Bose: Generative AI can make it way easier to use a performer’s likeness or voice without having that person there at all. And in that case, who gets to profit from the performance? 

Bushwick: This is Tech, Quickly, the digital double of Scientific American’s Science, Quickly podcast. I’m Sophie Bushwick, tech editor at Scientific American.

Bose: And I’m Tulika Bose, senior multimedia editor.

[CLIP: Music cue]

Bose: So, Sophie, I actually kind of spoke to a few actor friends about what’s happening right now, and why people are striking.

Bushwick: I get the impression that some people are worried they’re going to lose their jobs to an AI.  

Wolfgang Hunter: The worst case scenario would be there’s no more extra work, which is just like such a useful thing for a lot of like working actors, like even nonspeaking roles, extra work, like, that’s money, And if you can just AI generate a crowd and then doesn’t look like unrealistic—the thought of that is very scary. 

Bose: That’s my friend Wolfgang Hunter, an actor, comic, and writer in New York City who has written for shows like Mr. Beast. 

[CLIP: “I love the Midwest. Even the hecklers are nice people.”]

A lot of actors—even if you’ve seen them before—need to do everything they can to pay the bills. 

Hunter: So it’s scary to think that if you sign a waiver, you could just like, be never getting work again, even as like an extra.

Bose: So Sophie didn’t you speak with someone about this too?

Bushwick: That’s right — I spoke with Hany Farid, a professor at the University of California Berkeley who studies this type of technology. He says this idea, that generative AI can take an actor’s image and insert it into any scene, is just putting a new name on existing tech. 

Hany Farid: We used to call it deepfakes which was scary sounding and was largely associated with things like non-consensual sexual imagery and fraud and disinformation. Somebody in the industry did some very good rebranding and can now we call it generative Ai which sounds less scary and less awful. It is the same core underlying technology.

Bushwick: I tend to associate deepfakes with video. But generative AI can also be used to make still images, text, music, and other audio.

Bose: And this is also kind of worrying for actors who also rely on voice work. 

Hunter: What's on the line right now is reoccurring work. So especially with like voice acting and stuff like that — AI can’t like generate a new voice or a new character, but it can mimic existing ones.

Bose: So this character that Wolfgang voices, for example...

[CLIP: “I don’t know The Little Mermaid. I don’t listen to hip-hop.”]

Bose: Could be easily sampled and replaced, right? 

Bushwick: Exactly. And Hany talked about how easy this is with AI tools.

Farid: If I for example, Sophie record 2 minutes of your voice just 2 minutes nothing sophisticated just on this call right now: I can now go over to a commercial website and for $5 a month I can clone your voice and have you say anything I want you to say.

Bushwick: Of course, my voice clone might sound more realistic if the AI developers gave it more clips of my voice. In general, the more data you canprovide, the more realistic the digital double will look or sound.

Bose: Oh, digital double; kind of creepy.

Bushwick: Very creepy.

Farid: With a single image I can insert somebody who’s linking this into a video but because I've only got one view of them. It's going  to be gonna be a little glitchy, and it will not hold up to Hollywood style. It's fine for Tiktok and Youtube—but it’s not gonna it’s not gonna pass muster for Hollywood. 

Bose: The technology might get better eventually, but for now, actors have to go through something called the orb!

Bushwick: Okay, so it’s technically called a photogrammetry stage. But a video game actor who has performed on one of these stages, and who asked to remain anonymous, described it as “the orb” when he spoke to our tech reporting fellow, Lauren Leffer.

Bose: Very science fictional!

Bushwick: Oh, absolutely. I’m going to quote Lauren directly here: “Inside the orb, the world is reduced to a sphere of white light and flashes. Outside the orb’s skeletal frame is darkness. Imagine you are strapped into a chair inside this contraption. A voice from the darkness suggests expressions: ways to pose your mouth and eyebrows, scenarios to react to, phrases to say and emotions to embody. At irregular intervals, the voice also tells you not to worry, explains that more flashes are coming soon.”

Bose: Well, that sounds freakin’ terrifying.

Bushwick: Yeah, it does not seem like a pleasant experience! But if you’re a working actor, you have to put up with a lot of stuff like that for the job. And, on the one hand, AI is making it way easier to duplicate an actor’s image even without the orb—which just ramps up the pressure for performers to sign away their images.

Hunter: I am, like, such a guy that has such a bad habit of just signing something and not reading it. Absolute whatever. Yeah, cameras right there. Yeah, you're just like, lizard. But I think if I knew what I was looking at, I don't know. Because it's like, at the end of the day, that's all you have is yourself, you know, when you can't even get work or you're not even like in the industry, like fully. You only have this. You're basically selling, like, it's the closest thing I think we've actually had to selling your soul. 

Bose: So one goal of the strike is for actors to protect themselves from having to sell their image—or to make sure that, if their digital double is going to show up in a bunch of future media, that they’ll be fairly compensated for that.

Bushwick: And it’s not just their own digital doubles that they need protection from. Remember how Wolfgang said AI can’t generate a new character?

Bose: Uh oh.

Bushwick: Yeah, I don’t know about audio-only characters. But AI can absolutely generate images of humans who have never existed.

Farid: I can now just whole cloth create the next generation of performers. That's not a replacement. That's a creation. I can decide, look, we're going to do market research and we're going to say, well you know a blend between George Clooney and Brad Pitt and fill in your favorite actors. There is you know? So maybe you create this sort of this chimera right? This hybrid. 

Bose: Or, say, Timothee Chalamet?

[CLIP: “Janie’s hella tight. Maybe I’ll see you at the Deuce.”]

Bushwick: And if you have an AI-generated fake actor, it can be customized to different markets. Maybe there’s one version designed to appeal to Americans... 

Bose: Another version for the Middle East and another one for Chinese markets.

Farid: So the studios, they may not even need performers. These things can be created whole cloth. Um, so it's not even a replacement anymore. It’s something different.

Bose: Okay, yeah, this is a lot worse than we thought.

Bushwick: I understand why actors are striking! But also, if the studios want to go all-in on AI, then sure, they might be able to cut out some of the humans in front of the camera. They might even be able to save some money. But it could also majorly backfire for them.

Farid: If this technology really gets good where I can go to ChatGPT and say, please write me a movie script with this storyline and these sorts of themes and then I can synthesize the voices of the characters and synthesize the videos and do all of that on my laptop, do we democratize access to filmmaking? It's an interesting question. Maybe it’s not going to be Hollywood studios right now. The performers are scared for their jobs, and I think they have the right to be. But what if the studio should be fearful? What if the very studios that are jamming up the performers—what if they’re next?

Bushwick: Science, Quickly is produced by Jeff DelViscio, Tulika Bose, Kelso Harper and Carin Leong. Our show is edited by Elah Feder and Alexa Lim. Our theme music was composed by Dominic Smith.

Bose: Don’t forget to subscribe to Science, Quickly wherever you get your podcasts. For more in-depth science news and features, go to ScientificAmerican.com. And if you like the show, give us a rating or review!

Bushwick: For Scientific American’s Science, Quickly, I’m Sophie Bushwick. 

Bose: I’m Tulika Bose. See you next time!

Gary Stix: For Science, Quickly, I'm Gary Stix. 

Dreams may say something about the psyche, but they also say something about disease states. Dreams may even be something we can take some control over. To find out about all this, Scientific American got in touch with sleep expert Isabelle Arnulf, who is head of the Sleep Disorders Clinic at Pitié Salpêtrière University Hospital in Paris.

During her career, Arnulf, who is also a professor of neurology at Sorbonne University in France, has researched a broad range of sleep conditions. Sleepwalking, rapid eye movement, sleep behavior disorder, lucid dreaming, sleep and Parkinson's disease and hypersomnia, hypersomnia being excessive daytime sleepiness. Professor Arnulf is with us now to talk about sleep and dreams.

Hello, Professor Arnulf. Welcome to the podcast.

Isabelle Arnulf: Hello. Thank you.

Stix: So REM Sleep Behavior Disorder has been a real focus of your research. Can you tell us what that is and how you've found that it interacts with dreams and also how it might help us to better understand dreams?

Arnulf: So REM sleep behavioral disorders correspond to people having some violent behaviors during sleep, including boxing, kicking, shouting. This is the most common kind of dream behavior we see. And this is a disorder that should be treated because people are injuring themselves or their loved one. Most of them are older than 50 years old and they can go up to 80.

So it's not young people. And it has been discovered in the last 15 years that these behaviors, when they occurred in the elderly, is a strong predictor of Parkinson's disease and related disorders. So we started to do a lot of cohort of patients to follow them over time and looked whether they would develop Parkinson's disease or not.

This is the main research direction presently, but in addition to that was to use these behaviors as a window onto the dream process. Because if you think about it, during this process of waking up and remembering your dream, you lose a lot of information. In contrast, if you look at people during REM sleep behavior disorder, you'll see that dream in action at the time it's happening. People speak exactly like they would speak during wakefulness. They behave like they would during wakefulness. It's a narrow window, but it's a very, very proficient window into dreams.

Stix: Is there certain content of these dreams? And my understanding is that in some of the neurodegenerative conditions like Parkinson's that you just mentioned, there are certain themes, there's certain types of dreams that they have. Can you talk about them?

Arnulf: The most recurring themes of these dreams are aggression. They are mostly aggressed by animals, or by other humans. Or it's their loved ones who were aggressed, and they fight back to protect their loved ones, to protect themselves. We see that in 80% of the dreams, but 20% of them are quite normal.

Stix: Can this be used by clinicians for diagnosis?

Arnulf: If somebody is enacting dreams, shouting at nights, I think these dreams of being attacked and enacting them in the bed in their fifties or sixties, yes, it can be really used in the diagnosis process. You must think about Parkinson's disease that is coming, but enacting part is most important because all people can have nightmares but enacting them is is the key point that is caused by the beginning of Parkinson's disease.

Stix: Another area you study has to do with lucid dreams. Can you tell us what lucid dreams are and how they might provide a better understanding of dream states, and how they can be used in sleep medicine?

Arnulf: Lucid dreams is being aware of dreaming while you are still dreaming without awakening. Sometimes you realize you're dreaming and it wakes you up, but a lucid dreamer can remain asleep. And as soon as they realize that they are dreaming, many of them can act or change some of their dreams. Most of them try, for example, to fly and you can also change your dream to make them more agreeable, avoid enemies, or making nice travel all over the world, or sex with great (movie) stars.

So it's it's a condition that occurs in the normal population, it's more frequent in children than in adults. And this ability to change your dreams is very useful when you have nightmares. If you can realize that it's a bad dream or a nightmare and change it, it will make your nightmares disappear.

It's something we have developed with patient with narcolepsy, which is a another disorder when people are sleepy all the time, and they used it a lot to change their nightmare to make them more agreeable.

Stix: So you communicate with the self-aware patient and try to get them to take certain actions? Is that what you're saying?

Arnulf: Until our recent work on lucid dreaming, it was thought that you could not communicate with somebody asleep and dreaming. But we were able to do that in the series of patients. We were able to ask them some questions where they were asleep and they were able to answer, not with their mouth, but with some signs from their body, like pointing or smiling.

So lucid dreamers, in addition to changing their dreams, can be able to send signals or codes to the investigators. And in this case, they are just great subjects for helping us about research in dreams. Lucid dreaming is also just a scientific tool to explore what happens during our dreams.

For example, it was possible to know if time was the same during our dreams and doing reality using some signals sensed by the lucid dreamers like counting from 0 to 10, during dreams and doing the same during wakefulness. And it was shown that it was the same duration.

And we use it more recently to ask our lucid dreamers to send us codes about the emotions that were feeling in their dreams, and what we saw is that these are the emotions going extremely fast during lucid REM sleep, and they switch very rapidly from smiling to crying as if it was one of the functions of dreams to regulate emotion.

Stix: Does it also have any kind of clinical use in helping patients?

Arnulf: Yes, it has some clinical use, but you must be a lucid dreamer, which is something ... if you're not spontaneously a lucid dreamer, it may be difficult to acquire. You need three to six months of training to get to proficient lucid dreamers. But there are some people trying to develop some other techniques to accelerate the ability to become aware that you are dreaming when you are dreaming.

Stix: What are you most excited about in your research going forward?

Arnulf: The most exciting things I think is lucid dreaming--such a potent way to answer a lot of question we have about dreams for a long time, and because I found that this patient with narcolepsy where proficient lucid dreamers during naps, during daytime, we have built a lot of experiments around this idea and it worked very well to answer all the question we had about dreams. So it's coming one paper after the other because it takes time to publish, but I'm surprised by how efficacious this model is to understand dreaming.

Stix: This is really fascinating. Thank you so much.

Arnulf: Thank you. And goodbye.

Stix: Science, Quickly is produced by Tulika Bose, Jeff DelViscio, Kelso Harper and Carin Leong. Our show's music was composed by Dominic Smith. Please consider supporting independent journalism like this. Become a Scientific American subscriber today, and don't forget to subscribe to the podcast on Apple or Spotify.

For Science, Quickly, I'm Gary Stix.

Tanya Lewis: Hi, this is Your Health, Quickly, a Scientific American podcast series!

Josh Fischman: We highlight the latest vital health news: discoveries that affect your body and your mind.  

Lewis: And we break down the medical research to help you stay healthy. 

I’m Tanya Lewis.

Fischman: I’m Josh Fischman.

Lewis: We’re Scientific American’s senior health editors. 

On today’s show, we’re talking about how the stuff that goes on in your brain is connected to symptoms in your gut. Scientists have found molecular pathways that trigger inflammatory bowel disease flare-ups. And those paths start in the brain, triggered by stress. This raises the possibility of a new type of treatment.

[CLIP: Show theme music]

Fischman: Hey, remember the other day when we were comparing belly aches?

Lewis: Yeah, I do remember that. I was feeling a bit off, but drinking ginger ale really helped settle my stomach. Did your antacid tablets end up helping?

Fischman: Yeah they did. But neither of us was really sick. I mean, we both showed up at work, after all. 

Lewis: Yeah, I mean, speak for yourself—I wasn’t feeling great. But I definitely wasn’t extremely ill. 

Fischman: There is another kind of gastrointestinal illness, though, that’s in a whole different league. It’s inflammatory bowel disease, or IBD. That’s bouts of disabling pain, bleeding, weight loss, diarrhea and hospitalizations that sometimes require surgery. 

Lewis: That really doesn’t sound like fun. I actually know some people with IBD. Is it a super common illness?

Fischman: Unfortunately, it is. About 3 million adults in the U.S. have it. 

Lewis: I’ve heard of Crohn’s disease, which can be pretty disabling. Isn’t that a type of IBD? 

Fischman: Yeah, it is. The other type is called ulcerative colitiis. 

Lewis: Oh, I’ve heard of that one too. I think it mainly affects the large intestine.

Fischman: And Crohn’s often damages the small intestine, although it can hurt other parts of the GI tract as well. 

What makes them both IBD is inflammation. IBD is often linked to an overactive immune system, where the body’s own immune cells attack the digestive tract. And it may have a genetic component. 

Lewis: Yeah, I’ve heard that Crohn’s is more common among people with a family history of it. But are there any effective treatments?

Fischman: That’s usually some variety of immune suppressant such as a steroid drug or a medication that soothes inflammation. But even these controlled cases have periodic flare-ups.

The causes of these flares have been a real mystery. But I’ve been reading that the reason—or at least one of the main reasons—can be psychological stress.

Lewis: That does make sense. But it seems a little bit obvious that stress triggers a lot of diseases. So what’s the new thing here?

Fischman: It’s the precise connections. Starting in the brain, researchers traced two different pathways, made up of molecules and cells that kind of bang into one another. And the paths led all the way down to the intestines. 

Lewis: So is that kind of like a series of falling dominoes, each one knocking the next one over? 

Fischman: Yeah that’s a good way of thinking of it.

Lewis: But I just want to be clear: you’re not saying that IBD is all in your head, right?

Fischman: No, no. Really no. It’s a biological disease tied to those hyperactive immune cells. But...

Christoph Thaiss: Psychological stress will greatly exacerbate the severity of the disease. So stress by itself is not causing the disease, but it's greatly increasing the magnitude of the disease.

Fischman: That’s Christoph Thaiss.

Thaiss: I'm an assistant professor of microbiology here at the Perelman School of Medicine at the University of Pennsylvania.

Fischman: Thaiss was one of the scientists who traced out the rows of dominoes, as you put it. And his team found that the first domino to fall with IBD patients was some kind of stressful experience.

Thaiss: If they had broken up with their partner or lost their jobs or lost someone in the family or any other major event in the lives of these patients ... if someone goes through a very stressful period in their lives, it feels like their disease flaring up, but the physician doesn't see microscopic signs of inflammation yet. But then very often what will happen is a few weeks down the road, then there will be a flare. So there is definitely a very strong component of how the brain or the psychological state of the patient can control inflammatory diseases. 

Lewis: That is really validating, and I can say from personal experience that that intense stress can affect the body. But it sounds kind of anecdotal. I’m guessing Thaiss did some more scientific studies?

Fischman: Hey, I don’t blame you for a little skepticism. There’s a lot of loose talk about how stress affects the body, and you’re right: Thaiss and his team did go after this more scientifically. First they looked at mice with IBD-like conditions in order to identify the rows of molecular dominoes.

Then they verified that these same dominoes fell down in people. The pattern held true in three different groups of IBD patients. They even went so far as to do colonoscopy exams on some of these people, and saw these molecular signs of inflammation after stressful events.

Lewis: So what’s the next domino after someone has a fight with their spouse, for example?

Fischman: That’s the release of glucocorticoids, those are hormones that the brain triggers when you feel threatened. And these hormones reach two different kinds of cells in the gut, with two different effects.

Lewis: You mean hormones like cortisol? How does that affect cells in the gut?

Fischman: Well, first, did you know the gut had its own nervous system?

Lewis: Yeah, I think I did. Isn’t it called the enteric nervous system? I think it has neurons and supporting cells. 

Fischman: So those supporting cells are called glial cells. They do a bunch of different things and one is to signal those hyperactive immune cells, the ones I mentioned earlier, when the body is stressed. 

Those cells arrive, kind of like an attacking army, and they hit the lining of the intestines. And bingo, you get inflammation and a flare-up of IBD.

Lewis: And what about the second cell type you mentioned?

Fischman: Those are the enteric neurons. They control the muscles of the intestines and therefore how quickly or slowly food moves through them. Long exposure to glucocorticoids blocks these neurons from developing fully. They kind of stay in an immature state. 

And immature neurons aren’t able to make muscles squeeze very hard. So food moves very slowly. And IBD patients feel badly bloated or constipated or crampy. It just makes everything worse. 

Lewis: That sounds really horrible. But if stress leads to IBD flare-ups, could treating the stress help patients?

Fischman: Thaiss thinks so, although he cautions these are early days. Nobody has rigorously tested interventions like psychotherapy on IBD. 

But he believes that stress might affect how well other treatments, like drugs, actually work.

Thaiss: The treatment response of a patient might very strongly depend on psychological factors, which would mean that we should assess the psychological factors and tailor the treatment accordingly.

Lewis: So that’s not something doctors already do?

Thaiss: No, it's not a routine practice to basically quantify a patient's level of psychological stress when they're first seen by a physician or when the decision about the treatment is being made. So that's something that we're now following up on.

Fischman: Thaiss also thinks that psychotherapy itself, including stress management techniques, might help.

Thaiss: Stress mitigation strategy, which I think would be potentially even more beneficial than the molecular interventions at the downstream end of the pathway ... it may suggest that we can block this inflammatory pathway altogether by improving a patient's mental state or emotional state. 

Lewis: I guess I finally need to do that mindfulness meditation that I’ve been putting off. But it totally makes sense that treating stress itself could help with physical symptoms. And it seems like a promising place to start. 

Do you think it’ll work?

Fischman: Well, I hope it does. Like Thaiss said, they really need to test it. But I do have a gut feeling there may be something to this. 

[CLIP: Show theme music]

Fischman: Your Health, Quickly is produced by Tulika Bose, Jeff DelViscio, Kelso Harper and Carin Leong. It’s edited by Elah Feder and Alexa Lim.  Our music is composed by Dominic Smith.

Lewis: Our show is a part of Scientific American’s podcast, Science, Quickly. You can subscribe wherever you get your podcasts. If you like the show, give us a rating or review!

And if you have ideas for topics we should cover, send us an email at Yourhealthquickly@sciam.com. That’s your health quickly at S-C-I-A-M dot com.

Lewis: I’m Tanya Lewis.

Fischman: I’m Josh Fischman.

Lewis: We’ll be back in two weeks. Thanks for listening!