How to parse RSS and Atom feeds in python

Feedparser is available for python version 2.x and 3.x which provides a handy functionality for retrieving RSS, CDF and Atom feeds. I will use python 2 version, so make sure to adjust your code for python 3.

import sys
print(sys.version)
2.7.12 (default, Jun 28 2016, 08:31:05)
[GCC 6.1.1 20160602]

Now, let's see a simple example using feedparser library.

import feedparser
d = feedparser.parse('http://gewhere.github.io/rss.xml')
print(d['feed']['title'])
print(d.feed.subtitle)
print(len(d['entries']))
aucotsi's blog
music & code
5

In such a way you could easily track you favorite feeds and search for information that you are interested in. The following script extracts keywords from RSS feeds, by searching for keywords in the title and the summary of each entry.

import re
import feedparser

def entrylog(entry):
    "publication-time&date:author:link"
    msg = 'title: ' + entry.title + ' - link: ' + entry.link + ' - author: ' + entry.author # + '\ncontent:\n' + entry.summary
    print msg

regex_key = re.compile('entrainment|music* performance|interpersonal|multimodal*|expressiv*|coordination')
rssfeed = ['http://mp.ucpress.edu/rss/recent.xml','http://pom.sagepub.com/rss/recent.xml','http://www.cell.com/trends/cognitive-sciences/inpress.rss','http://www.cell.com/trends/cognitive-sciences/current.rss','http://journal.frontiersin.org/journal/psychology/rss']

total = 0

for i in rssfeed:
    d = feedparser.parse(i) # read link at a time
    for post in d.entries:  # read one RSS entry at a time
        try: # some feeds may not have summary
            m = re.search(regex_key, post.title)  # match regex in title
            n = re.search(regex_key, post.summary)  # match regex in summary
        except:
            print 'WARNING: The entry ' + post.title + ' at: ' + post.id + ' has no summary'

        if ((m != None)|(n != None)):
            entrylog(post)

    print str(len(d['entries'])) + " entries scanned from: " + d['feed']['title']
    total = len(d['entries']) + total
print "Total entries = ", total
title: Tempo in Baroque Music and Dance - link: http://mp.ucpress.edu/cgi/content/short/33/5/523?rss=1 - author: Coorevits, E., Moelants, D.
title: Singing Without Hearing: The Use of Auditory and Motor Information When Singers, Instrumentalists, and Nonmusicians Sing a Familiar Tune - link: http://mp.ucpress.edu/cgi/content/short/33/5/546?rss=1 - author: Erdemir, A., Rieser, J. J.
title: Examining Rater Precision in Music Performance Assessment: An Analysis of Rating Scale Structure Using the Multifaceted Rasch Partial Credit Model - link: http://mp.ucpress.edu/cgi/content/short/33/5/662?rss=1 - author: Wesolowski, B. C., Wind, S. A., Engelhard, G.
title: Interaction of Sight and Sound in the Perception and Experience of Musical Performance - link: http://mp.ucpress.edu/cgi/content/short/33/4/457?rss=1 - author: Vuoskoski, J. K., Thompson, M. R., Spence, C., Clarke, E. F.
title: Playing with the Edge: Tipping Points and the Role of Tonality - link: http://mp.ucpress.edu/cgi/content/short/33/3/344?rss=1 - author: Chew, E.
34 entries scanned from: Music Perception: An Interdisciplinary Journal recent issues
title: What you see is what you hear: The importance of visual priming in music performer identification - link: http://pom.sagepub.com/cgi/content/abstract/44/6/1361?rss=1 - author: Mitchell, H. F., MacDonald, R. A. R.
title: Music intervals in speech: Psychological disposition modulates ratio precision among interlocutors nonlocal f0 production in real-time dyadic conversation - link: http://pom.sagepub.com/cgi/content/abstract/44/6/1404?rss=1 - author: Robledo, J. P., Hurtado, E., Prado, F., Roman, D., Cornejo, C.
title: Acceptance and Commitment Therapy for the treatment of music performance anxiety: A single subject design with a university student - link: http://pom.sagepub.com/cgi/content/abstract/44/5/935?rss=1 - author: Juncos, D. G., Markman, E. J.
title: "Dont touch that dial": Accommodating musical preferences in interpersonal relationships - link: http://pom.sagepub.com/cgi/content/abstract/44/5/1193?rss=1 - author: Denes, A., Gasiorek, J., Giles, H.
title: Effects of applause magnitude and musical style on listeners evaluations of wind band performances - link: http://pom.sagepub.com/cgi/content/abstract/44/4/742?rss=1 - author: Springer, D. G., Schlegel, A. L.
title: The effectiveness of a multimodal concept of audition training for music students in coping with music performance anxiety - link: http://pom.sagepub.com/cgi/content/abstract/44/4/893?rss=1 - author: Spahn, C., Walther, J.-C., Nusseck, M.
title: Book review: The Emotional Power of Music: Multidisciplinary perspectives on musical arousal, expression, and social control. By Cochrane, T., Fantini, B., & Scherer, K. R. - link: http://pom.sagepub.com/cgi/reprint/44/4/910?rss=1 - author: Küssner, M. B.
title: Assessing jazz big band performance: The development, validation, and application of a facet-factorial rating scale - link: http://pom.sagepub.com/cgi/content/abstract/44/3/324?rss=1 - author: Wesolowski, B. C.
title: How do musicians evaluate their musical performances? The impact of positive and negative information from normative, ipsative, and expectation standards - link: http://pom.sagepub.com/cgi/content/abstract/44/3/399?rss=1 - author: Denton, E.-g., Chaplin, W. F.
title: "She did miracles for me": An investigation of dissonant studio practices in higher education music - link: http://pom.sagepub.com/cgi/content/abstract/44/3/466?rss=1 - author: Burwell, K.
title: The evolutionary origin of pitch centre recognition - link: http://pom.sagepub.com/cgi/content/abstract/44/3/527?rss=1 - author: Podlipniak, P.
81 entries scanned from: Psychology of Music recent issues
title: Oscillatory Dynamics of Prefrontal Cognitive Control - link: http://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(16)30153-X?rss=yes - author: Randolph F. Helfrich, Robert T. Knight
6 entries scanned from: Trends in Cognitive Sciences
title: Oscillatory Dynamics of Prefrontal Cognitive Control - link: http://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(16)30153-X?rss=yes - author: Randolph F. Helfrich, Robert T. Knight
WARNING: The entry Editors, Contents, Cover details at: http://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(16)30159-0?rss=yes has no summary
18 entries scanned from: Trends in Cognitive Sciences
20 entries scanned from: Frontiers in Psychology | New and Recent Articles
Total entries =  159

Latest posts