Python demonstration (scraping metadata from fivethirtyeight.com)

Back in spring of 2023 I wrote a Python script that would scrape metadata from the features pages on fivethirtyeight.com. At the time I was very new to Python.

The big draw is the function I wrote that would scrape the number of comments from each post. The comments are part of a Facebook plugin and I had to use an IP rotator to access it repeatedly.

The output is a .csv file with the metadata. Here is a folder with output files, if you're interested.

The bigger picture is I wanted to use this metadata in a data science project for the Erdös Institute's Data Science Boot Camp. You can find everything I did for this project in my Github repository, 538Project.

Here is the Jupyter notebook.