Senior Engineer (Automation and Web Crawling)

LoginSign Up for FreeJob listing>Job detailsSenior Engineer (Automation and Web Crawling)Full-Time|BangaloreApply for this roleJob Description‍Total Experience: 1+ yearsMode of Hire: PermanentRequired Skills Set (Mandatory): JavaScript/Node.js/Python, Web Crawling (Puppeteer/Playwright), Strong knowledge of web technologiesDesired Skills (Good if you have): HTML, Distributed Systems, Data pipelines, Proxy Management, Anti-bot TechniquesJob ResponsibilitiesDesign scalable web crawling solutions using modern technologies like Puppeteer and Playwright.Develop optimal and cost-effective methods for efficiently crawling diverse websites at scale.Implement robust proxy management and sophisticated anti-bot techniques.Create comprehensive debugging and monitoring systems for crawler performance.Actively contribute to technical discussions and architectural decision-making processes.Job RequirementsExperience in end-to-end management of large-scale distributed web crawling.Deep understanding of website internal mechanics, enabling innovative data extraction strategies.Expert-level proficiency in JavaScript/Node.js, web technologies, and anti-bot measures.Demonstrated ability to understand, adapt, and extend current system architectures. Strong collaborative skills with the ability to work effectively within existing technical frameworks.Preferred QualificationsAdvanced system optimization skills.Proven experience with cloud platforms, databases, and distributed systems.Comprehensive knowledge of data pipelines and data warehousing concepts.CultureWork with performance-oriented teams driven by ownership and passion.Learn to design systems for high accuracy, efficiency, and scalability.No strict deadlines, focus on delivering quality work.Meritocracy-driven, candid culture. No politics.Very high visibility regarding which startups and markets are exciting globally.About TracxnTracxn (Tracxn.com) is a Bangalore-based product company providing a research and deal sourcing platform for Venture Capital, Private Equity, Corp Dev, and professionals working around the startup ecosystem.We are a team of 600+ working professionals serving customers across the globe. Our clients include Funds like Andreessen Horowitz, Matrix Partners, GGV Capital, and Large Corporations such as Citi, Embraer & Ferrero.FoundersNeha Singh (ex-Sequoia, BCG | MBA - Stanford GSB)Abhishek Goyal (ex-Accel Partners, Amazon | BTech - IIT Kanpur)About Technology TeamTracxn's Technology team is 50+ members strong and growing. The technology team is subdivided into multiple smaller teams, each owning one or more services/components of the technology platform. Ours is a young team of motivated engineers with a minimal management structure where almost everyone is actively involved in technical development and design activities. We have a team-centric culture where the ownership and responsibility of a feature or module lie with a team compared to an individual.We work on an array of technologies, including but not limited to ReactJS, Next.js, Storybook, Webpack, Node, Mongo, AWS Lambda, Spring, Elastic Stack, MySQL, Kafka, Redis, Ansible, etc.We value ownership, continuous learning, consistency, and discipline as a team.‍CompanyAbout usOur InvestorsProduct ReleasesNews and Press ResourcesInvestor RelationsODR PortalTerms of UsePrivacy PolicyCookie PreferencesGDPRFAQsCareersCustomersVenture Capital FundsPrivate Equity FundsAccelerators and IncubatorsInvestment BanksCorporate InnovationCorpDev and M&A TeamsStartup FoundersJournalists and PublicationsUniversitiesGovernment AgenciesBanks and NBFCsCustomer testimonialsExplore TracxnDiscover interesting listsSector ResearchOfferingsSearch for Indian Legal EntityInsights by TracxnOur CollaborationsQuick LinksSchedule DemoGet in touchSee Pricing Plans© 2025 Tracxn Technologies Limited. All rights reserved. Registered Office: L-248, 2nd Floor, 17th Cross, Sector 6, HSR Layout, Bengaluru, Karnataka 560102. CIN:...