Social

Media

Aggregative

Simplifying

Helper

S.M.A.S.H.

picture

Social Media: Reddit

picture
  • Total Users: over a quarter billion
  • Over 850,000 subreddits
  • Average Daily Comments: over 5 million

Aggregation

  • Reddit API available for streaming
  • Daily and Monthly archives available in JSON format.

picture

Aggregation

  • Reddit API available for streaming
  • Daily and Monthly archives available in JSON format.
  • A single day loaded into a MongoDB collection.

picture picture

Aggregation

  • Reddit API available for streaming
  • Daily and Monthly archives available in JSON format.
  • A single day loaded into a MongoDB collection.
  • Over 3.5 million comments.
picture picture

Simplification

picture
  • How do we simplify a large collection of documents?
  • Dimensionality Reduction
  • Using NLP techniques/algorithms: LDA, LSA, NMF, etc.
  • S.M.A.S.H. works best with TF-IDF and NMF.

Simplification - a Demonstration

  • A small sample chosen for demonstration - three subreddits:
    • marvelstudios
    • dataisbeautiful
    • Competitiveoverwatch

t-SNE Plot

Intertopic Distance Map

Helper - Topic Identification

  • man ant iron spider strange doctor space hawkeye wasp shield
    • Ant Man, Iron Man, Spider Man, Doctor Strange, Hawkeye
    Avengers - Men
  • trailer war infinity avenger movie marvel cap wanda vision black
    • Infinity War Trailer

Helper - Time Channel Plot

Helper - Time Channel Plot

Helper - Front Page Topics

  • tide ad pod commercial eat super bowl clean detergent meme america superbowl clothe company actually roll watch buy campaign strange
    • Super Bowl Commercials or Tide Pod Meme
  • good pretty really bot look point thing luck bad bitcoin job work idea oh damn play little old hope guess
    • BitCoin

Helper - Time Channel Plot

Helper - Time Channel Plot

  • Over 3.5 million comments.

    • S.M.A.S.H

    picture

    Appendix

    picture
    • Let's try another group:
      • news
      • worldnews
      • politics
      • theDonald
    • Look at that similarity matrix.
    • Those guys really are in their own world.

    Topics - News and Politics

    • trump obama market president donald supporter stock economy credit lie
      • Presidents and the Economy
    • good bitcoin point thing bad luck pretty job market stock
      • BitCoin

    Helper - Time Channel Plot

    Helper - Time Channel Plot