How RetailIntel analyzes signals

RetailIntel continuously crawls Indian retail sources, enriches each item through a multi-agent AI council running on a frontier reasoning model, and publishes the signals its editor agent accepts. Each accepted signal gets a devil’s-advocate counter-case, and its search copy is checked against the numbers extracted from the source. Every published signal is named, dated and cited to a primary URL. No person edits a signal before it is published, but every agent’s output is stored, so we can audit why we wrote what we wrote. How RetailIntel sources, deduplicates, reviews and corrects signals is on /about.

What feeds the system

RetailIntel tracks a growing catalogue of Indian retail and business publishers. Some expose structured feeds; others require JavaScript-rendered list pages. Each source has a declared cadence, URL-level deduplication, and a per-cycle budget so one slow publisher cannot starve the rest.

The current set includes the retail-trade press (The Hindu BusinessLine, ET Retail, ET Realty · Shopping Malls, Retail4Growth, IMAGES Business of Fashion, Indian Retailer, India Retailing, BW Retail World, Storyboard18), the generalist business desks that file retail stories (Mint, Financial Express, Business Standard, Business Today, Forbes India, Fortune India, Moneycontrol, CNBC TV18, The Economic Times), the startup press (Inc42, Entrackr, YourStory, The Ken free list), and regulatory feeds like BSE Corporate Announcements for listed-retailer filings. Every signal links to its original report, and each publisher has its own page listing what we filed from it.

We do not get around paywalls. The council reads what the publisher makes public, often the feed excerpt, and cites and links the publisher. An item with no readable article behind it (a paywall page, navigation only, a bare headline) is not published.

The editorial council

A council run is a fixed sequence of model calls on one item at a time. Each agent returns a structured JSON object that is stored against the signal, so a bad signal can be traced to the agent whose read was off. There is no batch sign-off: a signal publishes as soon as its run finishes.

SCOUT — reads the article and decides whether it is about retail that matters in India. Items it marks out of scope stop here and never cost a downstream call. For the rest it extracts the primary brand, other brands, cities, event type, the figures the article states, a time horizon, and a summary of at most 45 words. The summary’s first sentence must carry a figure or date from the article and no figure the article does not state; a summary that breaks that rule is sent back once, and the first version stands if the rewrite breaks it too.

EDITOR — writes what the reader sees first: one of six themes (store-opening, retail-brand, omni-channel, retail-company, footfall, leadership), a headline of at most 140 characters, a deck of at most 400, a one-line delta, and a worthiness score. Stories it scores as too thin are not published.

FORECASTER — projects second-order effects: a probability-weighted scenario tree, the next moves to expect, and triggers to watch. The page labels these Likely next moves, RetailIntel’s automated analysis, not reported by the source.

COMPETITOR-MAPPER — names the rival brands most affected by the signal, with a 0-10 threat score and a likely counter-move for each, stored as competes_with edges in the competitor graph.

DEVIL’S ADVOCATE — writes the strongest counter-read: the bear case, the missing context, who would disagree, and a 0-10 strength score. It records dissent; it does not block publication. The page shows this counter-case only when every figure in it also appears in the source.

BRAND-LENS — writes one-sentence takeaways for three readers (operator, investor, corporate development), stored with the signal.

Clerks — short single-purpose calls. One copies up to three facts word for word from the article when it states no short figure; a fact that is not an exact match is dropped. One rewrites the summary’s opening when it only repeats the headline. One decides whether a report describes an event that already has a page. One writes the page’s search copy, which is discarded unless it matches the facts extracted from the article. A freshness check estimates when the event first became public and rewords a headline that would present an old story as new.

Brand hubs get a separate hourly model brief. A draft is stored only when every figure in it appears in the signals it cites and each paragraph links to those signals; a draft that fails is rewritten once, then dropped.

One story, one page

A story is published once. An exact link we already carry, a near-identical rewrite of a story from the last six hours in the same theme, or another publisher’s report of the same event is recorded as a further source on the existing signal instead of becoming a new page. Every other publisher that carried the event is named under the answer as Also reported by.

Data freshness

The architecture is built around the principle that writes should not trigger site builds. The council writes a new signal straight to the primary datastore, which fans out to an internal publish bus. Readers on /live hold an open Server-Sent Events connection to that bus and receive the new signal within seconds of council completion. There is no static build in the hot path — /live is server-rendered on each request, and signal detail pages like /signal/swiggy-eyes-indian-owned-status-to-flip-instamart-into-inven-1ee01bd4 read from the same database, so a correction takes effect on the next request.

On publish we also fan out to discovery surfaces: the sitemap-signals.xml entry is inserted in the same database transaction as the signal row, the RSS feed regenerates on read, and an IndexNow ping notifies Bing and Yandex that the URL is new. Google’s crawler picks up the sitemap on its own schedule.

Correction policy

If a signal is wrong, report it to us. A confirmed correction that changes the headline, deck or body stamps the signal’s revision time, which the page publishes as dateModified in its schema.org JSON-LD and article:modified_time. Nothing else moves that date: re-indexing, metadata regeneration and other housekeeping writes leave it untouched, and a signal that has never been revised carries no dateModified at all. For substantive changes — the wrong brand, the wrong number, the wrong event — we do not silently edit. The old signal is marked superseded and its URL permanently redirects (301) to the replacement, so citations to the original URL still resolve. The supersede pair stays in our records. The full correction and takedown policy is on /corrections.

What we don’t do

We don’t have browsing agents synthesising novel takes from the open web. Every signal traces back to a specific cited source row in the ingest archive.

We don’t paraphrase paywalled articles. We summarise the public excerpt and link to the publisher.

We don’t send unsolicited digests. Email goes only to readers who ask for it.

We don’t claim to predict the future with certainty. Every forecast the Forecaster writes is a scenario with an explicit probability and a horizon. When we’re wrong about a scenario, the scenario stays in the record.

The live feed: /live. The correction policy: /corrections. The providers that handle personal data: /privacy. What retail intelligence is, more broadly: /about and the companion pillar what is retail intelligence. The council’s artifacts are most legible on the signal pages themselves — pick any published item from the feed, like /signal/india-gold-discounts-hit-record-207-oz-as-15-duty-hike-empti-2fab8c52, and the scout summary, forecast, counterview, and cited source are all on one page.