Senior / Staff Software Engineer - Batch and Realtime Streaming
Berkeley, CAFull-TimeStaffSoftware Engineering
Responsibilities
- Architect, design, and implement core components for a Batch and Realtime Streaming platform, including data ingestion pipelines, storage systems, serving layers, and API interfaces
- Ship new features by collaborating across research, legal, trading, finance operations data, and infra teams for trading systems
- Collaborate with ML researchers and data scientists to understand their workflows and design intuitive interfaces (APIs, SDKs, UIs) for seamless feature discovery, access, and reuse
- Ensure data quality, consistency, and lineage for features, building robust mechanisms for versioning, monitoring, and governance
- Optimize data pipelines and storage for high performance, scalability, and reliability, considering both batch and real-time use cases
- Drive adoption of the feature store across teams by producing documentation, onboarding materials, and developer support.
- Mentor junior engineers and contribute to team best practices and technical excellence
Requirements
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
- 5+ years of software engineering experience, with a strong background in backend systems, distributed computing, or data infrastructure
- Experience in programming languages such as Python, Go, and C++
- Understanding of database technologies (Postgres, MySQL, Cassandra, DynamoDB, SQLite, DuckDB or MongoDB) and experience with APIs (REST/gRPC)
- Strong problem-solving skills, with a focus on delivering high-quality, maintainable, and well-documented solutions
- Excellent communication and collaboration skills; ability to work closely with both engineering and research teams
Preferred Qualifications
- Experience building large-scale data pipelines and storage systems (e.g., Airflow, Spark, etc)
- Exposure to modern Python data science tooling. (pandas, polars, dask, duckdb etc)
- Experience with monitoring and observability tools for distributed systems (e.g., Prometheus, Grafana, ELK Stack)
- Prior experience working with feature stores (e.g., Feast, Hopsworks, or custom solutions) is highly desirable
