The Data Infra Team

Meet the Data Infrastructure Team (aka Data Infra)

  • Tim Brown
  • Alex Hodges
  • Eugene Koontz
  • Michael Parkin
  • Thilee Subramaniam
  • Sam (Saam) Talaie
  • Scott Wilson
  • Micah Wylde

What does the team do?

Our team is responsible for building, scaling and managing the distributed systems that ingest, process, store, and make accessible trillions upon trillions of events about users, transactions, and other entity data we receive from our customers.

Over 6000 sites across 50+ countries use our APIs to power their businesses. We are in the critical conversion path for most of our customers’ businesses (think: “Should this order be processed?” or “Should this piece of content be published?”) and, thus, it’s critical for us to be highly available and performant. This is a big deal -- if we’re down, then our customers site are down.

We care about things like uptime, high availability, and low latency. Load balancing, auto-scaling, and automation are in our DNA and are what make it easy for us to keep up with the growth of our business and the growth of our customers.

What is the relationship with the other teams at Sift?

We provide infrastructure and services to help the other teams at Sift access the troves of data we acquire from our customers. This means providing real-time streaming interfaces through Kafka to perform near real-time analysis and batch-oriented interfaces through Spark and MapReduce for bigger jobs. We also build the underlying servers and components used by all applications. We develop and maintain the pipeline architecture for our ML applications.

Finally, we also index some entity data in ElasticSearch to help power our Console experience.

What does an average day on team look like?

On an average day, we are coding, testing, and deploying various infrastructure projects. Given our scale, we tend to push most of our systems to their boundaries, and often find ourselves digging deep into the internals of various open-source projects, such as HBase, to make improvements and contribute back to the community. We have short daily stand-ups and weekly meetings where we dive deep into specific projects and make longer-term plans. We prefer to focus the majority of our efforts on automation and development to minimize the time spent on operations and firefighting.

What is the culture like on the team?

We are an incredibly collaborative bunch who aim to not only build the best products, but also provide effective documentation and training for the rest of the company to best leverage our products. As engineers, we are given a great amount of freedom and responsibility to design and implement systems. To that end, we value deep discussions to tease out the most ideal solution and actively seek each others' advice and points of view. We also value giving back to the community and encourage each other to write blog posts, give talks, and contribute to open source projects. We have occasional team outings such as attending local trivia nights, boat cruises around the bay, and random froyo trips.

What are the roles on the team?

Our team is made of Software Engineers and SREs with backgrounds in distributed systems and large-scale datasets. We work on projects collaboratively and will pair SREs and and Software Engineers together on projects.

Eugene likes eating ramen, traveling, learning yoga and foreign languages.
Micah loves building reliable distributed systems, folk music and the outdoors.
Michael moved to the Bay Area in 2013 and loves working on distributed systems at scale.
When he's not out hiking or practicing his photography skills, Saam is working on problems at the intersection of engineering and machine learning.
Tim spends his free time snowboarding, surfing, and sailing.
Thilee enjoys photography, outdoors, single-malt, and cooking. Prior to Sift, he was building chatbots at NoApp.
Scott enjoys travel, hiking, yoga, and photography. He was born and raised in Alaska.

Technology

HBase

Kafka

Elastic Search

Apache Spark

Druid

OpenTSDB

Flink

Zookeeper

Terraform

Consul

MongoDB

Open Positions

See open positions on other teams