Building the Reddit Time Machine

View the final product: Reddit Time Machine

As the front page of the internet, Reddit is the ultimate collection of internet folklore. I'm a big fan of memes, copypastas and stories like Bean Dad. Recently I saw an old meme and started thinking about how internet humor has changed over the past decade or so, from simple block text to pop culture references to poorly-photoshopped meta memes.

There are hundreds of thousands of posts per day, with hundreds of millions of active users. Because it's been a top website for over a decade, I realized I could make a sort of time capsule to show what the internet found the most funny/interesting/infuriating at a particular point in time.

I originally called the project Reddit Time Capsule and used it to get familiar with AWS CDK. After I discovered NextJS and Vercel, I changed the name to Reddit Time Machine and made changes to improve SEO.

The Reddit API

Instead of using Reddit's own API, I found an even better search functionality in pushshift.io. The Pushshift API was created by a legend named Jason Baumgartner, who basically scraped the entire Reddit dataset and built an API on top of it. As far as I can tell it's funded by donations (I donated) and r/pushshift is an active community for using the API.

Retrieving posts

The Pushshift API is better than the Reddit API in that you can search the most upvoted posts by subreddit in a specific date range. The website's documentation has the list of possible parameters, and you can also find help in the r/pushshift subreddit. Here's how I create the url for searching the different subreddits:

// startDate is the date in unix time
const endDate = (startDate || new Date().getTime() / 1000) + 86400; // 1 day later
const baseURl =
  'https://api.pushshift.io/reddit/search/submission/?sort_type=score&sort=desc&size=25';
const url = baseURl + `&after=${startDate}&before=${endDate}&subreddit=`;
ts

That will return 25 posts with the most points (in decreasing order) between the two dates. So to search for memes, for example, I just use that url and add a comma-separated list of subreddits to search in:

const [memes, setMemes] = useState<Post[]>([]);
const [loading, setLoading] = useState(false);

const fetchData = async () => {
  setLoading(true);
  fetch(url + 'memes,memeeconomy,dankmemes,adviceanimals')
    .then((response) => response.json())
    .then((res) => {
      setMemes(res.data.slice(0, 10));
    })
    .catch(() => {
      setMemes([]);
    })
    .finally(() => {
      // set loading to false
    });
};
ts

Basically whenever the date is changed by the user, we call that API for the news, memes, etc and set them in our state to display as posts.

The UI

Instead of building out components from scratch, I decided to go with Ant Design for the design system. AntD is an open-source React design system that gives you tons of reusable components out of the box. Because I knew it would be a simple website, one of the most important factors for choosing Ant Design is that it has a nice DatePicker. It makes it easy to create and handle interactions:

For displaying the posts, I utilize the AntD Card component. I found that it looks best to have images on their own cards, and the News/Predictions/Politics look better as a list on a Card:

Dealing with images

Because of the age and scope of Reddit posts, dealing with images is a bit of a nuisance. Every post can have a thumbnail and a preview object, which has a nested object images that has an array of resolutions in increasing size. Posts that are just photos also include the image in the main url with a separate field indicating the image source (Reddit, imgur, etc).

To add to the fun, sometimes the post.thumbnail is "default", so if you check if the thumbnail exists, you'll be looking for an image named "default" which throws an error. That's what led to gross functions like this for trying to get a valid image url:

const getThumbnail = (post: Post) => {
  const imgResolutions = post.preview?.images.resolutions ?? [];
  const thumbnailUrl =
    post.thumbnail && post.thumbnail !== 'default' ? post.thumbnail : undefined;
  const previewUrl =
    imgResolutions && imgResolutions.length ? imgResolutions[0].url : undefined;
  return thumbnailUrl ?? previewUrl ?? 'default_thumbnail.png';
};
tsx

The ListView component shows the thumbnail, post title, and other details for each post, using AntD components to maintain styling:

const ListView: React.FC<ListViewProps> = (props) => {
  const { title, posts } = props;

  return (
    <>
      <h3>{title}</h3>

      <Card style={{ textAlign: 'left' }}>
        <List
          dataSource={posts}
          locale={{ emptyText: `No ${title} to show` }}
          renderItem={(item) => (
            <List.Item>
              <List.Item.Meta
                avatar={
                  <Avatar
                    shape='square'
                    src={getThumbnail(item)}
                    size='large'
                    alt={item.title}
                  />
                }
                title={
                  <a href={item.url} target='_blank' rel='noopener noreferrer'>
                    {text}
                  </a>
                }
                description={
                  <a href={item.full_link} target='_blank' rel='noopener noreferrer'>
                    {text}
                  </a>
                }
              />
            </List.Item>
          )}
        />
      </Card>
    </>
  );
};
tsx

Notice the title is a link that directs to the post.url i.e. the news article, and clicking the text below (the author, subreddit and number of points) directs to the full_link which is the post on Reddit.

The layout

I use Ant Design's Grid to layout the content using Rows and Cols, which can easily change width with the size of the screen using the breakpoints (i.e. 'sm' is when screen ≥ 576px, 'md' is screen ≥ 768px, etc).

I struggled for a while to decide on how to order the content. I decided to keep the images at the top because pictures catch your eye, and I decided that news should be first to capture major events, followed by politics.

Adding Predictions

After sending the link in my college GroupMe, my friend Hugh made a suggestion: along with news, I could display "predictions" from subreddits like r/futurology. It would be interesting to see what people thought would be possible in the future, and how clearly wrong some predictions can be. After messing around with the API, I realized that searching "by [date]" gives the best results, and "predicts", "goal", or "prediction" result in a lot of false positives.

Here's an interesting example from two random days in 2012 and 2016:

r/Futorology is an optimistic bunch, huh?

Have an other suggestions? Let me know on Github, or feel free to create a PR!