Real-time Financial Data Infrastructure

Context

Kalqyl, a Swedish financial media brand owned by Spotlight Group, wanted to publish AI-driven equity research and real-time market commentary across 160+ Swedish-listed companies. Two consumer-facing products were planned on top: a real-time bot (@KalqylAI) that reacts to fresh press releases on X, and a long-form analytical bot that writes structured research reports.

Both products had the same prerequisite: a data layer that could see every market-moving disclosure within seconds, normalize it across formats, and hand it off to downstream models with enough context to actually reason.

I was the AI & ML consultant from Predli embedded on this build, and the data infrastructure is the piece I owned end-to-end before either bot existed.

The problem

Swedish disclosure data is messy in three ways that don't show up in a slide:

1. It arrives in many shapes. A press release is a body of text, but it almost always carries attachments — PDFs of investor decks, Excel tables of financials, image-based scans of legal notices. Reading just the body throws away most of the substance.
2. Volume is bursty, not steady. Quiet days have a handful of releases; quarterly reporting days have dozens of companies reporting in the same hour. The pipeline has to absorb the spike without losing or reordering events.
3. Not every release matters equally. A board appointment and a profit warning live in the same feed. Downstream agents need to know what they’re looking at before they decide whether to fire.

Most "stock news" pipelines solve (1) by scraping the body and ignoring attachments. They solve (2) by polling on a schedule and accepting minute-scale lag. They don’t solve (3) at all — they hand the consumer everything and shrug.

For a system that's going to publish to 16,000 investors with no human in the loop, none of those shortcuts work.

Architecture

The pipeline runs on Azure as a set of decoupled functions, each with one job and a clean handoff to the next.

1. Always-on WebSocket listeners

A pool of Azure Functions subscribes to disclosure feeds for 160+ tickers on the Swedish market. They’re the only stateful part of the system — long-lived connections that wake up downstream functions when a new disclosure lands.

2. Multi-format ingestion

When a PR drops, the ingestion stage extracts at three levels:

• Body text — the press release prose itself, cleaned of boilerplate.
• Attached documents — PDFs and decks get layout-aware parsing instead of being flattened to a wall of text, so headings, captions, and table boundaries survive into the downstream payload.
• Structured tables — financial statements and KPI tables are extracted as actual rows and columns with header context, so a downstream agent reasoning about a revenue figure gets numbers it can use, not OCR slop.

Everything lands in a single enriched event keyed off the disclosure.

3. Relevancy filter

Not every event in the feed deserves downstream compute. A first-pass filter drops obvious noise — routine fund holding disclosures, calendar reminders, regulatory boilerplate — before the classifier and any LLM ever see it. This keeps cost honest and keeps consumer bots from reacting to non-events.

4. Naive Bayes PR classifier

A Naive Bayes classifier sorts each surviving release into financial (earnings, dividends, capital actions, profit warnings) vs non-financial (personnel, legal, product, M&A intent). The label is a routing key, not the final output — downstream consumers branch on it to pick the right prompt template, the right schema, and the right validation rules.

The instinct in 2023 was to throw a fine-tuned transformer at every text problem. For Swedish-language financial disclosures, a well-calibrated NB was the right tool: cheap, calibratable, and tunable around the asymmetric error cost that actually mattered — a wrongly-labeled financial PR risks a hallucinated number in a published tweet, while a wrongly-labeled non-financial PR is just a slightly-off-tone summary. Picking the simplest model that respects your error costs is almost always the right call.

5. Enriched event → consumers

The final payload is a structured event — body, parsed attachments, extracted tables, classification, confidence scores, source metadata, and a stable ID for deduplication. That's the contract the Twitter bot and the Analytical bot consume; neither of them re-parses the source.

Why this matters as a data layer

Both KalqylAI consumer products got the same answer to the same question: given a fresh disclosure, what can you tell me, fast and accurately? Splitting the data infrastructure out as its own concern is what made that possible:

• One ingestion contract. Both bots read the same enriched event shape. When ingestion improves — better table extraction, a new attachment format, a smarter classifier — both products inherit it on the next release.
• Cost lives in the right place. Heavy parsing happens once, in the data layer, and is cached against the disclosure ID. Downstream LLM calls operate on already-extracted structure instead of redoing OCR per consumer.
• Failure modes are localized. A flaky upstream feed shows up as listener errors, not as a hallucinated tweet. A bad classifier threshold shows up as a queue of misrouted events, not as a corrupted research report.

It's the boring, observability-led part of the system that lets the visible product feel magic.

Results

The infrastructure has powered KalqylAI in production since 2023. 8,000+ reports and insights have been published downstream of it, generating 4.6M+ annual views and reaching 16,000+ investors. The case study is on Predli's site.

The bots that ride on it are documented separately as the KalqylAI Bot Suite.

Reflection

The hardest engineering decision wasn’t the LLM stack — the hardest decision was treating the data layer as a product, not as plumbing. Most "real-time AI" pipelines collapse ingestion, classification, and generation into one application. That works until you want a second consumer, or until something breaks and you can’t tell whether the bug is in the parse, the classifier, or the prompt. Drawing the seam at "enriched event" forced every component to have a clean contract, and every later improvement compounded across both bots.