How REBUY priced used electronics: from live comparables to machine learning

By Lucas Espindola · October 15, 2018 · 11 min read

REBUY answered one question for people with a drawer full of old gadgets: how much is this actually worth? Type "iPhone 7 16GB", and a few seconds later you got a price, the number of listings it was based on, and roughly how long one takes to sell. Here is how that number was produced, how the same data told us which products were worth going after, and the machine-learning layer we built to replace the heuristics once enough data had built up.

The product

The flow was deliberately short. You searched for each device you owned, and a running total counted up: "Descubra quanto você pode ganhar!", "find out how much you can earn". Under the total sat two buttons, Quero vender (sell) and Quero doar (donate). Selling linked you straight to the marketplaces the price came from. Donating showed a map of e-waste collection points. The whole product depended on one thing: the price had to be believable, because people compare it with what they see on the marketplaces themselves.

Prices from comparables

We didn't set prices; the market did. For every search, the API went to the two places where used electronics actually change hands in Brazil, OLX and Mercado Livre, and read the current listings for that query:

OLX: a regional search, up to five pages of results, parsed for title, price, category, region and posting date.
Mercado Livre: the first results page, plus its "related searches" box, which we reused as suggestions in our UI.

Both were plain HTTP fetches parsed with jsdom and jQuery selectors, run in parallel with short timeouts so a slow marketplace couldn't hold the answer hostage. Nothing was stored: every search was priced from what was listed at that moment.

Matching: which listings are actually this product?

A search for "iPhone 7 16GB" returns cases, screen protectors, "iPhone 7 Plus" and broken units for parts alongside the phones. Before any statistics, every listing title was scored against the query with Levenshtein edit distance, the number of single-character edits needed to turn one string into the other:

products = products.map(x => {
    x.levenstein = searchText.toLowerCase().levenstein(x.product.toLowerCase());
    return x;
});
products = products.sort((a, b) => a.levenstein - b.levenstein);

Sorting by distance puts listings whose titles read like the query first and pushes accessories and different models down. It needs no training data and no product catalogue, and it works across both marketplaces' wildly different title styles. That mattered, because we had neither.

From comparables to one number

The closest 100 listings with a plausible price became the comparables, and the headline price was their median:

const relevantResults = products
    .filter(x => x.price_value > 0 && x.price_value < 8000)
    .splice(0, 100);
const prices = relevantResults
    .sort((a, b) => a.price_value - b.price_value)
    .map(x => x.price_value);

price: {
    min: prices[0],
    max: prices[prices.length - 1],
    median: Math.round(statistics.medianSorted(prices)),
}

The median did most of the work. Marketplace listings are noisy in both directions: optimistic asking prices, "R$ 1" placeholders, bundles, parts-only units. A mean gets dragged around by every one of them; the median of a hundred close matches barely moves. The hard price cap removed only the worst nonsense, and the distance ranking had already filtered out most accessories before price was considered.

The response also carried how many listings the price was based on and a time-to-sell estimate. That estimate was the age of the oldest listing on OLX's last results page for the query, a rough measure of how long stock sits before it sells. The UI showed both next to the price ("this search covered N listings, and this product sells in about X days on average"), because a number shown with its evidence is easier to trust.

Targeting: which products are worth it?

Pricing one search is useful to a person. Deciding what REBUY itself should focus on needed a view of the whole market, so the API grew a second system: a daily job that mirrored Mercado Livre's public catalogue into MongoDB. It walked the category tree from a root category, paged through each category's listings, then enriched items with sold quantities and how many other sellers listed the same brand and model.

On top of that we computed, per listing, how much it actually moves:

diff_months = max(ceil((now - date_created) / 1 month), 1)
revenue     = sold_quantity × price / diff_months
sales       = floor(sold_quantity / diff_months)

With monthly revenue, monthly sales and the number of competing sellers on every listing, a filterable search over the catalogue answered the targeting questions directly. Which models sell fastest used? Where is the money per month? Which products have few sellers but steady demand? Filters covered price, condition, listing type, seller reputation, rating and title keywords, and results could be sorted by any metric.

The ML layer

The heuristics had clear ceilings. A median can't tell a boxed unit from a cracked one. Monthly sales computed from a listing's lifetime average can't tell a listing that sells every day from one that sold a burst in its first week. Exact brand-and-model matching misses every listing that doesn't fill in those attributes. With the catalogue in MongoDB, a learned layer could answer three questions the raw data couldn't:

What is this item worth? Price estimation.
How fast does it sell? Sales velocity and time-to-sell.
Which listings are the same product? Product matching.

The answers fed the "how much you can earn" email, the suggestedPrice field on user-submitted products, and new filters and sort keys in the targeting search.

Data first: snapshots

The catalogue job upserts each listing, so every run overwrote price and sold_quantity and the history was lost. We added one row per listing per run to an MLProductSnapshot collection:

{ product_id, category_id, captured_at, price, sold_quantity,
  available_quantity, reviews, listing_type_id, condition }

Real sales velocity then came from the change in sold_quantity between snapshots, replacing the lifetime average, and the snapshot history became the training data for every model below.

Price estimation

A gradient-boosted regressor (LightGBM) with quantile outputs, so it predicts a range (p10, p50, p90) rather than a single number. Features: category and its path from the root, the BRAND and MODEL attributes, condition and wear level, a TF-IDF or embedding representation of the title, and seller status, listing type, free shipping and state.

The important choice was the target: the log price of listings that actually sold, meaning their sold quantity went up between snapshots. Listings nobody buys don't get to set the price. That is the main thing the median of live listings could never do, since asking prices and selling prices are different numbers. In the email, the single total became a range like "R$ 850–1.100".

Sales velocity and time-to-sell

A regressor trained on the snapshot sales changes, predicting units sold per month for a listing and days-to-sell for a used item at a given price. Its features are the price relative to the median of the product's cluster, reviews, seller status, listing type and shipping. It replaced the "age of the oldest listing" heuristic, and its prediction, _ml.predicted_sales, became something the targeting search could filter and sort on.

Product matching

Multilingual sentence embeddings of each title plus its attributes, with nearest-neighbour clustering inside each category, so Portuguese titles that describe the same phone in different words land together. This replaced both the Levenshtein ranking and the exact brand-and-model match. The resulting _ml.cluster_id gave an honest competing-seller count and pooled price data across listings for the price model.

Category classification

Products users submit are free text with no marketplace category. A title classifier maps them to a category_id, which every other model needs before it can run.

Architecture

Agenda jobs (Nest/Node)                 rebuy-ml (Python, FastAPI)
─────────────────────────               ───────────────────────────
BUILD_CATEGORY_CATALOG                  POST /predict/price
BUILD_PRODUCT_CATALOG ──► snapshots     POST /predict/velocity
BUILD_PRODUCT_DETAILED_INFO             POST /match
SCORE_PRODUCT_CATALOG ──────────────►   POST /classify/category
        │                                     ▲
        ▼                                     │ nightly retrain
MLProduct._ml { predicted_sales,        MongoDB snapshots ──► model registry (S3)
  cluster_id, price_p50, model_version, scored_at }

Training runs nightly in a separate Python service that reads the Mongo snapshots and writes versioned models to S3. Python because the API ran on Node 8 and Nest 5, which make a poor ML runtime.
Batch scoring is a new catalogue job that runs after the crawl and writes predictions onto each listing. The targeting search reads those fields directly in its aggregation, so search never calls a model at request time.
Online prediction is only for items users submit: a Nest service calls /predict/price.
Fallback: if the ML service is down or its confidence is low, the same interface returns the old heuristics, the median and the lifetime-average velocity.

Beating the baseline

Every model has to beat the heuristic it replaces before it ships, measured on a time-based split (train up to week t, test on week t+1) so it can't peek at the future:

Model	Metric	Baseline it must beat
Price	MAPE / pinball loss	Category median price
Velocity	MAE on units per month	`sold_quantity / diff_months`
Matching	Precision@k on a labelled sample of pairs	Exact BRAND + MODEL match
Category	Top-1 / top-3 accuracy	—

Each prediction records the model_version that produced it, and price-distribution drift is tracked per category.

Rollout

We introduced it in phases, so there was always a working answer:

Data: snapshots started recording. No models yet, because sales changes need a few weeks to build up.
Baselines: the existing heuristics moved behind the ML service interface, becoming both the fallback and the benchmark.
Price model: pricing for user-submitted items went live first, because it drives the email users see.
Batch scoring: matching and velocity were written onto the catalogue and exposed in the targeting search.
Feedback loop: the final sale price of items sold through REBUY went back into training as ground-truth labels.

What I'd change in the heuristics

Reading the heuristic code years later, a few things stand out:

Two scrapers, one response. Each marketplace callback could answer the request on its own, so whichever returned first won. The fix is a single Promise.all with per-source timeouts, and a later, unfinished rewrite in the repository moves to exactly that.
"Average" was a midpoint. The field called avg was (min + max) / 2, the least robust number in the response. The median was the right headline and should have been the only one.
Relevance was computed, then ignored. Each comparable had a 0–100 relevance score that never fed into the price. Weighting the median by relevance, or trimming by interquartile range instead of a fixed price cap, would have steadied prices for long-tail products well before any model.

The order mattered more than any single model. The heuristics shipped first because they were simple and explainable: "the median of the 100 closest live listings" is an answer anyone can check for themselves. When the ML layer arrived they didn't disappear. They became the fallback and the bar every model had to clear before it shipped.