Prefer email? Write to hello@compliedai.com with subject line Application · Data Engineer. Attach your CV and a short note on why you want to join Complied AI. You can also use the form below the description.
Every update on Complied AI starts as a page or PDF on a regulator's website: MCA, CBDT, CBIC, RBI, SEBI, BSE, NSE and more. Our Python scrapers check each listing, pull what is new since the last run, clean it, and write it to Postgres with an audit row for every run. You would own that pipeline.
The hard part is that the sources do not want to be scraped. Listings move without notice, dates hide inside PDFs, some sites block plain HTTP clients, and a scraper that silently returns zero rows looks exactly like a quiet week. Your job is to make those failures loud, keep the dedup correct, and make sure we never publish a half-parsed document.
Day to day you would add new sources, fix the ones that broke overnight, improve PDF text extraction and date detection, and tighten the quality checks that run before anything reaches the database. You would also watch the daily runs and decide when a gap is a real problem and when a regulator simply published nothing.
You have two or more years writing production Python that moves data, ideally with some web scraping and Postgres. Experience with Playwright, PDF parsing, or OCR helps. You do not need a legal background.
Full-time, work from home anywhere in India, and paid. Some runs fail at awkward hours. We fix that with better alerts and retries, not by asking you to stay up.
Apply for Data Engineer (Scrapers)
No CV upload. A few questions about you, then a few about the work. Please answer each one. Or email hello@compliedai.com with subject Application · Data Engineer, your CV, and why you want to join Complied AI.