Jay Malhotra
Back to all work

Olist late deliveries

A model that flags e-commerce orders likely to arrive after the promised date, using only what is known when the order is approved.

top 10% of flags catch 36% of late orders
Role
Project lead on a team of seven: framing, repository and shared code, feature iteration, integration
Context
McGill INSY 674 final project, Fall 2026
Stack
Python, pandas, scikit-learn
Outcome
Checking the riskiest 10% of orders catches 36% of the late ones.
Share of late orders caught by checking 10% of orders a day (test period)
Random checks
10%
Two-feature baseline
33.6%
Final model
36%
The model more than triples random checks. Most of that gain comes from two features.

From the analysis

The problem

Olist is a Brazilian marketplace. Between January 2017 and July 2018, 7.0% of its delivered orders arrived after the promised date. Late orders averaged 2.21 stars, against 4.28 for orders that arrived on time.

The goal was to flag risky orders early enough for someone to act, instead of finding out from a bad review.

What I built

  • One row per order, joined from seven tables: 85,371 delivered orders after filtering.
  • Only features known at approval. Reviews were left out because they do not exist yet, and payments because we could not confirm they are known at that point.
  • Every model was scored with the same five-fold time-series cross-validation. The test set was used once, at the end.
  • Models were compared on lift, because late rates swing from 2.8% to 18.8% across folds and raw PR-AUC swings with them.

Results

  • Checking the riskiest 10% of orders each day would have caught 36% of late orders in the test period, against 10% for random checks.
  • Logistic regression beat the tree models. They overfit: their lift on training folds was 6 to 12 times their validation lift.
  • Where the parcel is going and how long the promise is matter more than what is in it.

Limits

The gain over a simple baseline is modest. A logistic regression on two features, promised days and customer state, already catches 33.6%.

No single feature is strong, and lateness comes in system-wide waves that the model cannot see coming.

See it