Rewriting PySpark consumers in Go
What changed and what we measured on real-time market data streams.
I build high-throughput backend systems: real-time market data, data platforms and APIs that serve 500K+ requests/sec. Over 6 years at Angel One and Innovaccer, mostly in Go, Python and Java on AWS.
I'm a backend engineer at Angel One, India's third-largest stock broker, where I'm a core member of the Charts team. I work on the systems behind real-time market data and charts: streaming consumers, data workflows and the APIs that serve them.
Before that I was on the Data Platform team at Innovaccer. I migrated legacy healthcare data pipelines for 500+ providers with 100% data integrity, and built a query-conversion tool that translates between database DSLs.
Outside work I build and ship my own products. My Android apps have 200K+ installs, and I'm building DailyJobs.ai, a job-search autofill and tracker. I started out in open source with FOSSASIA and competitive programming at Jamia Millia Islamia.
A job-search assistant: it autofills applications on job portals and tracks every application in one place.
An audio enhancement app with 5–31 band equalizers, bass boost, a volume amplifier and real-time DSP (limiter, loudness enhancer). It has presets for 10,000+ headphones and is localized into 50+ languages.
What changed and what we measured on real-time market data streams.
How the migration went, and the ~$10K/month it saved.
Notes on building, localizing and growing small apps.
Posts are on the way. Until then I share updates on LinkedIn ↗
Have a role, a project or a hard backend problem? My inbox is open.