Overview

We are hiring a Lead Data Engineer. Headquartered in Los Angeles, California, Right Balance provides top-tier technology talent for innovative companies in the US. We’re in the top 50 companies to watch in LA.

Engagement Details

Our client is a Series A well-funded US start-up. Currently in a product market fit stage, meaning rapidly prototyping different features for new customers and trying to adapt to their needs, not focusing on technical excellency at this moment. Their produce is all-in-one recruiting platform that provides:

1. Database of 100M US professionals
2. Email and phone number lookup
3. Email outreach and sequencing
4. Applicant tracking system
5. AI Recruiter capability Recruiting well can set a company up for long term success, while poor recruiting can set up a company for failure. We are working on a bold mission to replace the current jumble of multiple expensive and confusing systems all into a single platform at an affordable price. They have a remote global engineering team of 7 at the moment. Company leadership is based in Los Angeles. A fast moving startup that takes pride in a fast iteration style and transparency. They want to work with high-energy do-ers with heavy bias for action.

Role:

• Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.
• Hire data engineers to aid you in that journey.

Core Responsibilities

• Design scalable data pipelines processing massive record volumes
• Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)
• Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch
• Integrate new data sources into the main pipeline
• Implement advanced data matching using Splink

What’s in it for you

1. Learn and evolve your skills using the latest and greatest technology tools in a rapidly growing company.
2. Learn from the best people around you. We constantly challenge the status quo and invent new ways of building a great product.
3. Work on challenging problems, innovate, and positively impact many people's lives while having fun doing it.

Required Qualifications

• Upper-intermediate to fluent speaking and writing English. Able to have a real-time conversation.
• 5+ years of full-time hands-on Data Engineering experience.
• 5+ years of full-time hands-on Python experience.
• 5+ years of full-time hands-on SQL experience.
• 5+ years of full-time hands-on PySpark experience.
• 5+ years of full-time hands-on AWS experience.
• 2+ years of full-time hands-on Tech Lead experience.
• 2+ years of full-time hands-on Docker experience.
• 2+ years of full-time hands-on Metabase/ Athena/Glue/EMR experience.
• Good proficiency in: PySpark and distributed computing AWS data services (EMR, Glue, Athena) Docker Pandas and DataFrame manipulation Complex data format handling (JSONL, Parquet)
• Strong background in: Big data processing architectures Data warehouse design Performance optimization Advanced Python, SQL skills

Nice to haves

• Probabilistic record linking expertise
• OpenSearch/elasticsearch technologies
• Machine learning data pipeline design
• Recruitment tech ecosystem knowledge
• Bachelor’s degree in Computer Science or equivalent demonstrated ability.

Frequently Asked Questions

What are your typical clients?

The majority of our clients are venture-backed startups at the growth stage. Usually, at this stage, the company already achieved a product-market fit and is looking to expand rapidly. That’s where we bring the best engineering practices, strong architecture, the latest technologies, and consistent processes to help companies scale.

What is the length of your engagements?

Most of our long-term full-time engagements last multiple years. It allows you to evolve your career with the client company taking on more responsibilities.

What’s your company size?

The Right Balance team is 60+ engineers going to 100+ by the end of the year. The current client size team is 8+ people. The timing is great to be a part of a small team making meaningful contributions.

What happens if the engagement is completed?

Most of our engagements are long-term in nature. That said, if the current engagement is ramping down, we’ll present you with more long-term opportunities to transition into.

What are your core values?

Client First: we only win when our clients win. We treat client challenges as our own.

Ownership: we embrace responsibility, taking on challenges, getting them to completion, and enjoying getting things done.

Quality: we’re passionate about achieving quality outcomes by applying meticulous attention to detail.