Back to projects

ETL & Data

Data Pipeline & Automation Infrastructure

Consolidated data engineering suite: ETL framework parsing 20+ Indonesian economic indicators from BPS, Bank Indonesia, Kemenkeu, and DJPPR archives (2003–2025); reusable scraping stack processing 30,000+ BKI ship records and 15,000+ school records across 39 provinces with retry logic, rate limiting, and checkpoint/resume; automated recurring report feeds for SPIP credit card, GAIKINDO automotive, and World Bank manufacturing indicators from multi-sheet Excel and PDF sources.

Tech stack

PythonPandaspdfplumberopenpyxlRequestsThreadPoolExecutorETLAutomation