23.2  The MovieLens Dataset

There are a number of datasets that are available for recommendation research. Amongst them, the MovieLens dataset is probably one of the more popular ones. MovieLens is a non-commercial web-based movie recommender system. It is created in 1997 and run by GroupLens, a research lab at the University of Minnesota, in order to gather movie rating data for research purposes. MovieLens data has been critical for several research studies including personalized recommendation and social psychology.

23.2.1 Getting the Data

MovieLens 100K contains \(100{,}000\) ratings on a one-to-five scale, contributed by 943 users for 1,682 movies (Herlocker et al. 1999). Every included user rated at least 20 movies, so the dataset cannot evaluate a truly new user. The archive also contains timestamps and limited demographic and genre fields; this chapter uses the ratings and timestamps in u.data. The GroupLens documentation describes the remaining files.

To begin with, let’s import the packages required to run this section’s experiments.

PyTorch

from d2l import torch as d2l
import numpy as np
import os
import pandas as pd
import random
import torch

MXNet

from d2l import mxnet as d2l
from mxnet import gluon, np
import os
import pandas as pd
import random

Then, we download the MovieLens 100k dataset and load the interactions as DataFrame.

PyTorch

d2l.DATA_HUB['ml-100k'] = (
    'https://files.grouplens.org/datasets/movielens/ml-100k.zip',
    'cd4dcac4241c8a4ad7badc7ca635da8a69dddb83')


def read_data_ml100k():
    data_dir = d2l.download_extract('ml-100k')
    names = ['user_id', 'item_id', 'rating', 'timestamp']
    data = pd.read_csv(os.path.join(data_dir, 'u.data'), sep='\t',
                       names=names, engine='python')
    num_users = data.user_id.unique().shape[0]
    num_items = data.item_id.unique().shape[0]
    return data, num_users, num_items

MXNet

d2l.DATA_HUB['ml-100k'] = (
    'https://files.grouplens.org/datasets/movielens/ml-100k.zip',
    'cd4dcac4241c8a4ad7badc7ca635da8a69dddb83')


def read_data_ml100k():
    data_dir = d2l.download_extract('ml-100k')
    names = ['user_id', 'item_id', 'rating', 'timestamp']
    data = pd.read_csv(os.path.join(data_dir, 'u.data'), sep='\t',
                       names=names, engine='python')
    num_users = data.user_id.unique().shape[0]
    num_items = data.item_id.unique().shape[0]
    return data, num_users, num_items

23.2.2 Statistics of the Dataset

Inspecting a few records checks both the delimiter and the column order before downstream indexing turns identifiers into array positions.

PyTorch

data, num_users, num_items = read_data_ml100k()
sparsity = 1 - len(data) / (num_users * num_items)
print(f'number of users: {num_users}, number of items: {num_items}')
print(f'matrix sparsity: {sparsity:f}')
print(data.head(5))

number of users: 943, number of items: 1682
matrix sparsity: 0.936953
   user_id  item_id  rating  timestamp
0      196      242       3  881250949
1      186      302       3  891717742
2       22      377       1  878887116
3      244       51       2  880606923
4      166      346       1  886397596

MXNet

data, num_users, num_items = read_data_ml100k()
sparsity = 1 - len(data) / (num_users * num_items)
print(f'number of users: {num_users}, number of items: {num_items}')
print(f'matrix sparsity: {sparsity:f}')
print(data.head(5))

number of users: 943, number of items: 1682
matrix sparsity: 0.936953
   user_id  item_id  rating  timestamp
0      196      242       3  881250949
1      186      302       3  891717742
2       22      377       1  878887116
3      244       51       2  880606923
4      166      346       1  886397596

Each record contains a user identifier, item identifier, rating, and timestamp. Placing the observed ratings in an \(m\times n\) user–item matrix leaves most entries missing: the observed fraction is \(100{,}000/(943\cdot1{,}682)\), or about \(6.3\%\). Missing is not the numerical value zero, nor does it mean dislike. A separate observation mask is therefore required whenever the matrix is represented densely.

We then plot the distribution of the count of different ratings. As expected, it appears to be a normal distribution, with most ratings centered at 3-4.

PyTorch

d2l.plt.hist(data['rating'], bins=5, ec='black')
d2l.plt.xlabel('Rating')
d2l.plt.ylabel('Count')
d2l.plt.title('Distribution of Ratings in MovieLens 100K')
d2l.plt.show()

MXNet

d2l.plt.hist(data['rating'], bins=5, ec='black')
d2l.plt.xlabel('Rating')
d2l.plt.ylabel('Count')
d2l.plt.title('Distribution of Ratings in MovieLens 100K')
d2l.plt.show()

23.2.3 Splitting the dataset

The split defines the prediction problem. In random mode, each interaction is assigned independently to training or test data, with 90% used for training by default. The same users and items can therefore occur in both sets, and later interactions may be used to predict earlier ones. This is a warm-start interpolation protocol; it should not be interpreted as a prospective recommendation test.

In seq-aware mode, each user’s most recent interaction is held out and the earlier interactions are ordered by time for training. This protocol asks whether a history predicts a later event, although it still excludes cold-start users and items. A production evaluation should also preserve a validation set for model selection and reserve the test set for the final report. The compact experiments below use a single holdout to keep the implementation short. They therefore demonstrate the mechanics of the metrics but do not provide a clean estimate after hyperparameter selection.

PyTorch

def split_data_ml100k(data, num_users, num_items,
                      split_mode='random', test_ratio=0.1):
    """Split the dataset in random mode or seq-aware mode."""
    if split_mode == 'seq-aware':
        train_items, test_items, train_list = {}, {}, []
        for line in data.itertuples():
            u, i, rating, time = line[1], line[2], line[3], line[4]
            train_items.setdefault(u, []).append((u, i, rating, time))
            if u not in test_items or test_items[u][-1] < time:
                test_items[u] = (i, rating, time)
        for u in range(1, num_users + 1):
            train_list.extend(sorted(train_items[u], key=lambda k: k[3]))
        test_data = [(key, *value) for key, value in test_items.items()]
        # O(N) set-membership filter instead of O(N^2) list-membership.
        test_set = set(test_data)
        train_data = [item for item in train_list if item not in test_set]
        train_data = pd.DataFrame(train_data)
        test_data = pd.DataFrame(test_data)
    else:
        # Seed for deterministic splits across frameworks; uses Python's
        # `random` for cross-framework portability (some frameworks' numpy
        # shim lacks `np.random.default_rng`).
        rng = random.Random(0)
        mask = [rng.random() < 1 - test_ratio for _ in range(len(data))]
        neg_mask = [not x for x in mask]
        train_data, test_data = data[mask], data[neg_mask]
    return train_data, test_data

MXNet

def split_data_ml100k(data, num_users, num_items,
                      split_mode='random', test_ratio=0.1):
    """Split the dataset in random mode or seq-aware mode."""
    if split_mode == 'seq-aware':
        train_items, test_items, train_list = {}, {}, []
        for line in data.itertuples():
            u, i, rating, time = line[1], line[2], line[3], line[4]
            train_items.setdefault(u, []).append((u, i, rating, time))
            if u not in test_items or test_items[u][-1] < time:
                test_items[u] = (i, rating, time)
        for u in range(1, num_users + 1):
            train_list.extend(sorted(train_items[u], key=lambda k: k[3]))
        test_data = [(key, *value) for key, value in test_items.items()]
        # O(N) set-membership filter instead of O(N^2) list-membership.
        test_set = set(test_data)
        train_data = [item for item in train_list if item not in test_set]
        train_data = pd.DataFrame(train_data)
        test_data = pd.DataFrame(test_data)
    else:
        # Seed for deterministic splits across frameworks. Use Python's
        # `random` module rather than `np.random.default_rng` because
        # mxnet.numpy.random doesn't expose the modern numpy RNG API.
        rng = random.Random(0)
        mask = [rng.random() < 1 - test_ratio for _ in range(len(data))]
        neg_mask = [not x for x in mask]
        train_data, test_data = data[mask], data[neg_mask]
    return train_data, test_data

The returned second split is named test_data for compatibility with the chapter code. If its metrics guide architecture or hyperparameter choices, it is functioning as validation data and a separate untouched test split is still required for a final estimate.

23.2.4 Loading the data

After splitting, load_data_ml100k maps the one-based identifiers to zero-based array indices and returns aligned user, item, and rating lists. For explicit feedback it also constructs a dense rating matrix; for implicit feedback it records each user’s observed item set. These representations retain the split chosen above and must not be recombined before evaluation.

PyTorch

def load_data_ml100k(data, num_users, num_items, feedback='explicit'):
    users, items, scores = [], [], []
    inter = np.zeros((num_items, num_users)) if feedback == 'explicit' else {}
    for line in data.itertuples():
        user_index, item_index = int(line[1] - 1), int(line[2] - 1)
        score = int(line[3]) if feedback == 'explicit' else 1
        users.append(user_index)
        items.append(item_index)
        scores.append(score)
        if feedback == 'implicit':
            inter.setdefault(user_index, []).append(item_index)
        else:
            inter[item_index, user_index] = score
    return users, items, scores, inter

MXNet

def load_data_ml100k(data, num_users, num_items, feedback='explicit'):
    users, items, scores = [], [], []
    inter = np.zeros((num_items, num_users)) if feedback == 'explicit' else {}
    for line in data.itertuples():
        user_index, item_index = int(line[1] - 1), int(line[2] - 1)
        score = int(line[3]) if feedback == 'explicit' else 1
        users.append(user_index)
        items.append(item_index)
        scores.append(score)
        if feedback == 'implicit':
            inter.setdefault(user_index, []).append(item_index)
        else:
            inter[item_index, user_index] = score
    return users, items, scores, inter

Afterwards, we put the above steps together and it will be used in the next section. The results are wrapped with Dataset and DataLoader. We keep the partial last batch in both frameworks (last_batch='keep' for MXNet, drop_last=False for PyTorch) so that no training samples are silently dropped from each epoch, and orders are shuffled.

PyTorch

def split_and_load_ml100k(split_mode='seq-aware', feedback='explicit',
                          test_ratio=0.1, batch_size=256):
    data, num_users, num_items = read_data_ml100k()
    train_data, test_data = split_data_ml100k(
        data, num_users, num_items, split_mode, test_ratio)
    train_u, train_i, train_r, _ = load_data_ml100k(
        train_data, num_users, num_items, feedback)
    test_u, test_i, test_r, _ = load_data_ml100k(
        test_data, num_users, num_items, feedback)
    train_set = torch.utils.data.TensorDataset(
        torch.tensor(train_u), torch.tensor(train_i),
        torch.tensor(train_r).float())
    test_set = torch.utils.data.TensorDataset(
        torch.tensor(test_u), torch.tensor(test_i),
        torch.tensor(test_r).float())
    train_iter = torch.utils.data.DataLoader(
        train_set, shuffle=True, drop_last=False,
        batch_size=batch_size)
    test_iter = torch.utils.data.DataLoader(
        test_set, batch_size=batch_size)
    return num_users, num_items, train_iter, test_iter

MXNet

def split_and_load_ml100k(split_mode='seq-aware', feedback='explicit',
                          test_ratio=0.1, batch_size=256):
    data, num_users, num_items = read_data_ml100k()
    train_data, test_data = split_data_ml100k(
        data, num_users, num_items, split_mode, test_ratio)
    train_u, train_i, train_r, _ = load_data_ml100k(
        train_data, num_users, num_items, feedback)
    test_u, test_i, test_r, _ = load_data_ml100k(
        test_data, num_users, num_items, feedback)
    train_set = gluon.data.ArrayDataset(
        np.array(train_u), np.array(train_i), np.array(train_r))
    test_set = gluon.data.ArrayDataset(
        np.array(test_u), np.array(test_i), np.array(test_r))
    train_iter = gluon.data.DataLoader(
        train_set, shuffle=True, last_batch='keep',
        batch_size=batch_size)
    test_iter = gluon.data.DataLoader(
        test_set, batch_size=batch_size)
    return num_users, num_items, train_iter, test_iter

23.2.5 Summary

• MovieLens datasets are widely used for recommendation research. They are publicly available and free to use.
• We define functions to download and preprocess the MovieLens 100k dataset for further use in later sections.

23.2.6 Exercises

• What other similar recommendation datasets can you find?
• Go through the https://movielens.org/ site for more information about MovieLens.

PyTorch

Discussions

MXNet

Discussions

Herlocker, Jonathan L, Joseph A Konstan, Al Borchers, and John Riedl. 1999. “An Algorithmic Framework for Performing Collaborative Filtering.” 22nd Annual International ACM Conference on Research and Development in Information Retrieval, SIGIR 1999, 230–37. https://doi.org/10.1145/312624.312682.