Projects

(03) Team research · STEM Club

Echo Chambers in Recommender Systems

Do recommender algorithms narrow what users see, and what does a simulation need before it can tell?

Pham Ha Khanh Chi · research mentor

2026 · BVIS Hanoi STEM Club · CREST Silver Award applicant

In short

The early results looked clean, and every one of them turned out to be a bug. Once the simulation was fixed, the three standard algorithms were no different from random. The real contribution is knowing why, and what a simulation needs before it can detect an echo chamber at all.

Research overview

This project asked whether recommender algorithms (popularity-based, content-based and collaborative filtering) gradually narrow what users see, creating an “echo chamber”. We tested this in a simulation: 100 MovieLens users, each receiving ten recommendations per round, “clicking” according to their genre preferences, with that behaviour fed back into the system. We tracked genre diversity and Shannon entropy over time to compare how each algorithm behaved.

Echo chambers get blamed for political polarisation and radicalisation online, and for people discovering less new content. But much of that is assumed rather than tested. I wanted to see whether the effect really shows up under controlled conditions, and what you would need to measure it properly.

The early results looked clean: collaborative filtering broadened exposure, content-based filtering narrowed it, and popularity swung widely. Every one of those turned out to be a bug in the simulation rather than a real effect. Once we had found and corrected the flaws, the three original algorithms showed no difference from random recommendation.

Line chart of genre diversity over 30 rounds for the original baseline: collaborative filtering rises from about 11 to 12.7 genres, content-based falls from about 11.7 to 8, and popularity swings between 9 and 15.
Before: the original baseline The first results looked like three distinct stories: collaborative filtering broadening, content-based narrowing, and popularity swinging wildly. All three trends turned out to be created by bugs in the simulation, not real effects.

Approach

(01) Control

Simulate, against random

A simulation let us isolate mechanisms under controlled conditions. Alongside the three baseline algorithms we added a random-recommendation control, so changes were compared against noise rather than against zero.

(02) Check

Could the design show an effect?

When the corrected baseline showed nothing, I tested the design itself. Content-based filtering at click rates 100× apart gave bit-identical recommendations, so it had no feedback loop. Collaborative filtering’s signal came from neighbouring users, not the individual.

(03) Redesign

Separate taste from the guess

No algorithm both read one user’s preferences and updated from that user’s behaviour. So I split each user’s true taste (which drives clicks, hidden from the algorithm) from the algorithm’s inferred profile, and built an adaptive content-based algorithm whose clicks update only the inferred profile.

Recommendations Clicks driven by true taste Click history Inferred profile true taste (hidden)
The redesigned loop Clicks come from the user’s hidden true taste, but only update the algorithm’s guess. Measuring the gap between the two shows how far an algorithm’s model of a user drifts from who that user really is.

Evaluation: five defects

Each defect was isolated with a diagnostic and confirmed with a controlled comparison.

  1. 01

    Shared rating matrix. Users’ simulated clicks leaked into later users’ collaborative-filtering calculations. This created the whole +1.57-genre “broadening”; giving each user their own copy made it vanish.

  2. 02

    Inconsistent exclusion. Popularity and content-based filtering were recommending films users had already rated (17.8% and 3.3% of recommendations).

  3. 03

    Profiles measuring sparsity, not taste. Preference concentration correlated with rating count at ρ = +0.715. Rebuilding profiles from viewing proportions made the click model about 3.4× better at telling a user’s top genre apart from the rest.

  4. 04

    Unrealistic click rate. Click rate had been set equal to the affinity score, giving 49–69% against a realistic 1–5%, so we recalibrated it.

  5. 05

    Biased tie-breaking. Ties went to the film with the higher ID, which favours newer films. This affected 91–97% of content-based picks.

After all that, there was one real effect. The adaptive algorithm narrowed genre variety significantly more than the fixed one, by about 1.3 genres, and it was the only version that clearly beat the random baseline. Users saw more different films, but from fewer genres, and the narrowing levelled off after about 30 rounds instead of spiralling.

Line chart of genre diversity over 100 rounds after the fixes. Popularity and collaborative filtering cycle around or above the random-control band near 10.8 genres; static content-based cycles between about 4 and 5; adaptive content-based falls from 5 to about 3 and levels off.
After: corrected genre diversity, 100 rounds Only the adaptive content-based algorithm narrows over time, dropping to about 3 genres per list and levelling off after roughly 30 rounds rather than spiralling. The other three show no narrowing beyond the cycle created by the 10-round cooldown. The shaded band (rounds 90–100) is the last full cycle, which we used as the main comparison window.
Line chart of per-round Shannon entropy over 100 rounds with a corrected estimator. Popularity and collaborative filtering sit around 3.9 to 4.3 bits, above the random-control band near 3.65; static content-based cycles between about 1.7 and 2.2; adaptive content-based settles near 1.5.
After: corrected Shannon entropy Entropy tells the same story as genre diversity from a different angle: the adaptive content-based algorithm settles at the lowest genre spread. The repeating sawtooth comes from the 10-round cooldown, so we averaged the last full cycle (rounds 90–100) to compare algorithms fairly.
Line chart of cumulative item-level exposure concentration over 100 rounds. It is zero for the first 10 rounds, then rises to about 0.30 for popularity, static content-based and collaborative filtering, and about 0.26 for adaptive content-based; the random control stays near zero.
After: how often films repeat This tracks how often the same films are shown to a user again (0 means every impression is a different film). The adaptive algorithm has the lowest repetition of the four, so it narrows genres without showing the same titles more often. Concentration stays at zero for the first 10 rounds by design, because films cannot return until the cooldown ends.

My role

I was the team’s research mentor, so checking that the work was valid was my job. When collaborative filtering kept trending upward and popularity was jumping around too much to be believable, I did not just accept the explanations my teammates gave me. I audited the code myself and walked them through each bug as it came to light.

From there I directed the investigation, deciding what to test and which results not to trust, and I caught mistakes in my own analysis too. I made the main design calls: separating true taste from the algorithm’s guess, building the adaptive algorithm and adding the random baseline. I also helped my teammates explain their parts of the project, and wrote up the evaluation for our CREST Silver report.

Context