Context
A dense dataset holds valuable signal, but it is scattered across too many rows to read in a plain table. This project turns that raw data into a visual exploration tool — a multi-page dashboard with cross-filters — and pairs it with a guided narrative for stakeholders who have two minutes, not twenty.
The working dataset is daily stock-market data (Alpha Vantage / yfinance): prices, volume, and returns across a basket of tickers and sectors. The same structure applies just as well to the Olist Brazilian e-commerce dataset (orders, categories, regions) if a commerce angle is preferred.
Business Problem
Which segments drove performance over the period, and how do they compare on risk and return?
Concretely, for the market dataset: which sectors and tickers led returns, how did volatility differ across them, and where did the unexpected moves happen?
Architecture
The pipeline stays deliberately flat: extraction and cleaning happen in Python, the BI tool consumes a clean processed CSV rather than doing transformation work of its own, and the dashboard is the only consumer-facing artifact. Staging the raw extracts in AWS S3 is an optional step that would open the door to scheduled refreshes.
Data Source
- Source: Alpha Vantage API / yfinance (daily stock-market data). Alternative: the Olist public e-commerce dataset.
- Storage: flat CSV extracts, versioned under data/, loaded into Tableau Public / Power BI.
- Volume: tens of thousands of rows — multiple tickers across several years of daily bars.
- Key fields: still being finalized alongside the ticker basket — expected to be date, ticker, sector, open, close, volume and daily_return.
- Known limitations: survivorship bias in the ticker basket, split and dividend adjustments still to verify, and rate limits on the free API tiers.
Methodology
The project follows the CRISP-DM framework:
- Business Understanding — define the question (segment performance and risk) before opening any tool.
- Data Understanding — pull the extracts from Alpha Vantage / yfinance, profile coverage, and spot gaps.
- Data Preparation — clean nulls and outliers, standardize categories and sectors, and engineer fields: daily return, moving averages, volatility.
- Modeling / Analysis — build the dashboard: an overview page plus per-segment deep-dive pages, wired together with cross-filters.
- Evaluation — pressure-test the insights against the numbers and record the top three takeaways.
- Deployment — publish to Tableau Public (or Power BI), record the 2-minute narrated demo, and ship the README.
AI-assisted narration
The first draft of the dashboard's insight narration and executive summary is generated with an LLM, prompted with the actual aggregated figures. The analyst then edits, tightens, and validates every claim against the underlying numbers before anything ships. AI removes the blank-page friction; the human owns correctness. AI assists, human validates.
Challenges
These are the problems the build has to solve — named up front so the finished project can be judged on how it handled them.
- Survivorship bias in the ticker basket. A basket assembled from today's index members quietly excludes everything that was delisted, which flatters every return figure computed from it. The selection rule has to be documented and the conclusions scoped to this basket rather than presented as a claim about the market.
- Picking a price-adjustment convention and holding it. Raw close and split/dividend-adjusted close produce different returns for the same ticker on the same day. The convention has to be chosen deliberately and verified against a known corporate action before a single return number reaches the dashboard.
- Keeping cross-filters reconcilable across pages. The value of a multi-page dashboard is also its main failure mode: it is easy to land on a deep-dive page showing a filtered subtotal that no longer ties back to the overview. Every page's figures need to reconcile to the same source aggregation.
- Extraction against free-tier rate limits. Pulling a multi-ticker, multi-year basket from a rate-limited API means the extraction step has to be chunked, cached, and restartable — otherwise a re-run either burns the daily quota or leaves data/raw/ half-populated without failing loudly.
Results
Once the numbers are in, this section will cover:
- The segments — sectors and tickers — most associated with returns, with the magnitude of the effect.
- A risk-versus-return comparison across the most relevant segments.
- Any anomalies or unexpected patterns surfaced during exploration.
Tech Stack
| Category | Tool |
|---|---|
| Data extraction | Python (yfinance, requests) / Alpha Vantage API |
| Data preparation | Python (pandas) or Power Query |
| Visualization | Tableau Public / Power BI |
| Narrated demo | Loom |
| Narration draft (AI-assisted) | LLM — first draft only, human-validated |
| Cloud (optional) | AWS S3 for staging raw CSV extracts |
| Versioning | Git / GitHub |