main

Podcasts turn 21 next month and they’re extremely popular. I have earbuds in most of the day, and by my best estimate listen to a couple of hours of podcasts whilst walking and doing chores over the same period. I’m not the only one. Over half of voting age adults in America told Edison Research, a market research firm, that they had listened to at least one podcast in the month prior to the election for the President of the United States in 2024.

They were important in both major campaign’s media mixes in that election. For instance, unlike Harris, Trump eschewed an interview “60 Minutes”, a traditional stop-off, and neither candidate spoke to large media outlets like NPR, PBS, the New York Times or the Washington Post. Yet, Trump sat for 16 hours of podcast interviews and Harris for 7. Both campaigns used these appearances to micro-target specific subsections of the undecided, news unaware electorate.

They are also a unique form of media. Big podcast publishers like the New York Times and NPR share space with independent voices. Hosts make money partly on the closeness of their audience, with direct support like live recordings, direct financial support and purchasing merchandise alongside advertisements (depending on which academic you ask they’ll call this “parasocial” or “intimacy” – I discuss this more in my diss).

Two of the people the campaigns sat down with exemplify this. Alex Cooper, the host of the “Call me Daddy” podcast, the second most listened to podcast on Spotify during the election, apologised to listeners for discussing politics on the show when Kamala Harris featured in the run up to the election. And, although Joe Rogan, host of the “Joe Rogan Experience” endorsed Trump he said he was open to having Harris on his show.

My study: An Analysis of Bias in Podcasts During the 2024 United States Presidential Election

My study focussed on partisan political speech in podcasts as a whole, rather than those which claim that they are political based on their tags, or those that feature political people. It is the first attempt (admittedly done quite cruedely by an undergrad) to use natural language processing methods to estimate bias within podcasts to answer a very basic question: did podcasts systematically favour one party over another during last year’s election?

What did I find out?

I initially thought that podcasts would exhibit the traits of the electorate at large, ie. leaning Republican and being more bias than those calling themselves ‘News’ podcasts. I was wrong. Instead, I found that:

  1. Podcasts devoted substantially more attention to Democratic figures (67.6%) than Republican ones (32.4%) overall.
  2. When Republicans were the subject of sentences, they expressed negative sentiment 61% more frequently than when discussing Democrats. (Positive stance was 26.5% less likely to accompany Republican references.)
  3. Podcasts that tagged themselves as ‘News’ had far more pronounced partisan tendencies.
  4. The association between positive sentiment and Democratic figures grew significantly stronger in the final month of the campaign compared to the first.
  5. High-popularity shows exhibited significantly more pronounced Democratic favouritism than less popular ones.

How did I find this out?

  1. I used Castbox’s everest API (which is very poorly documented see this thread) to get a list of all feeds and associated all-time download metrics.
  2. Filtered the top 100 feeds that were released between September 10th 2024 and November 5th 2024.
  3. Produced transcripts for each episode within the downloaded feeds (n = 1566) using Deepgram’s Nova 2, which offered an inexpensive, fast and accurate API.
  4. Used a very broad filter to extract episodes (2%) that did not contain any political language with keywords from the Comparative Agendas Project’s codebook.
  5. I then randomly sampled 500 episodes, and filtered for sentences which contained fewer than five words to produce n = 483,884 sentences for analysis.
  6. I then classified stance and political affiliation using two BERT-style transformers that had been trained on a different domain: cardiffnlp/xlm-twitter-politics-sentiment and m-newhauser/distilbert-political-tweets. I couldn’t afford the tagging and compute needed to train my own, but I assumed that social media speech was quite similar to podcast speech.
  7. I used a hierarchical logistic regression approach using mixed‐effects models because my data was nested (ie. sentences being grouped within episodes, grouped within feeds).

(Note on some obvious flaws: This does exclude YouTube/Spotify exclusives as well as differences that might come from hosting on this platform that may not be generalisable to the others; some firms do offer their own metrics of feed popularity but their methods are opaque and the best ones charge. I was already spending a lot on compute so I couldn’t afford this.)

You can download my dissertation here. The models that I used are available on Hugging Face (1 and 2). Most of the codebase for the project was in R, although I used Python for some of the more complicated parsing of the RSS feeds and to run the models. I’m planning on releasing replication code when I have time to tidy it up, but do email me with questions in the meantime!