Lead Data Engineer - View Integrations
Klaviyo
At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying. Want to learn more about life at Klaviyo? Visit careers.klaviyo.com to see how we empower creators to own their own destiny.
At Klaviyo, we love tackling tough engineering problems and look for engineers who specialize in certain areas but are passionate about building, owning & scaling features end to end from scratch and breaking through any obstacle or technical challenge in their way. We push each other to move out of our comfort zone, learn new technologies and work hard to ensure each day is better than the last.
Team Overview:
Klaviyo operates a real-time and offline data analytics platform coded primarily in Python that is built for massive scale and hosted on Amazon Web Services (AWS). The DataLake and View Integration teams ingest billions of interactions per day to power core, Klaviyo functionality and generate data-driven insights. These interactions are ingested from disparate sources, persisted into the Data Lake, materialized into analytical storage, and used to power AI models.
As a Lead Engineer on the View Integrations Team, you will have ownership over defining the evolutionary, technical vision for materialized view frameworks. You will be responsible for the design, implementation and tuning of highly visible mission critical materialized view systems. You will be constantly challenged to improve these systems from every angle, such as monitoring, failure recovery, data quality, tooling, performance, ease of use and much more.
Team Tech Stack:
- Python
- Spark
- Airflow
- RabbitMQ, Kafka
- DynamoDB
- MySQL
- Kubernetes
- AWS
How You'll Make a Difference
- Responsible for coaching engineers, managing/reviewing technical documentation and articulating a phased approach to achieving the team's overall technical vision.
- Wholly owning features that 50,000+ customers rely on daily to reach 4B+ consumers to drive their business forward.
- Working on crafting our highly scalable real-time data pipeline that processes billions of events weekly across data sets growing to petabyte scale.
- Working on a deep analytics system that provides insight into hundreds of terabytes of data.
- Be responsible for the mission, technical direction and operations of projects that have immediate impact.
- Helping architect and design robust and high performance data processing and storage systems leveraging new databases to serve different internal and external use cases. Contribute to open source data processing technologies.
- Contributing to the company as a subject matter expert in multiple areas, constantly pushing yourself to be a better engineer and to level up all of your peers within your team and within Klaviyo.
Must Have:
- 10+ years of experience in software development, built and delivered at least one complex data processing system
- 2 years of experience in a people management role or team leadership role, coordinating the delivery of nontrivial projects.
- 5 years of experience testing and launching software products
- Experience working with reliable, fault-tolerant, and highly performant distributed systems.
- Experience with high throughput orchestration tools like temporal or cadence
The pay range for this role is listed below. Sales roles are also eligible for variable compensation and hourly non-exempt roles are eligible for overtime in accordance with applicable law. This role is eligible for benefits, including: medical, dental and vision coverage, health savings accounts, flexible spending accounts, 401(k), flexible paid time off and company-paid holidays and a culture of learning that includes a learning allowance and access to a professional coaching service for all employees.
Get to Know Klaviyo
We’re Klaviyo (pronounced clay-vee-oh). We empower creators to own their destiny by making first-party data accessible and actionable like never before. We see limitless potential for the technology we’re developing to nurture personalized experiences in ecommerce and beyond. To reach our goals, we need our own crew of remarkable creators—ambitious and collaborative teammates who stay focused on our north star: delighting our customers. If you’re ready to do the best work of your career, where you’ll be welcomed as your whole self from day one and supported with generous benefits, we hope you’ll join us.
Klaviyo is committed to a policy of equal opportunity and non-discrimination. We do not discriminate on the basis of race, ethnicity, citizenship, national origin, color, religion or religious creed, age, sex (including pregnancy), gender identity, sexual orientation, physical or mental disability, veteran or active military status, marital status, criminal record, genetics, retaliation, sexual harassment or any other characteristic protected by applicable law.
IMPORTANT NOTICE: Our company takes the security and privacy of job applicants very seriously. We will never ask for payment, bank details, or personal financial information as part of the application process. All our legitimate job postings can be found on our official career site. Please be cautious of job offers that come from non-company email addresses (@klaviyo.com), instant messaging platforms, or unsolicited calls.
You can find our Job Applicant Privacy Notice here.