Cloud Data Engineer

Twitter
Full time Full day
San Francisco, CA

Company Description




As data engineers in Revenue Science, our mission is to build real-time and offline solutions to make data accessible and reliable while leveraging the largest-scale data processing technologies in the world -- and then apply them to the Revenue’s most critical and fundamental data problems.


Learn more about some of the challenges we tackle on this team:




  • Building a Petabyte-scale Data Warehouse (Google Cloud Next '18) https://youtu.be/APBF9Z3uBCc


  • How Twitter Migrated its On-Prem Analytics to Google Cloud (Google Cloud Next '18) https://youtu.be/sitnQxyejUg




Job Description




You are passionate about data and driven to take the data organization challenges at the scope of entire Twitter’s Revenue.


As a member of the Data Engineering team, you will build and own mission-critical data pipelines that are ‘source of truth’ for Twitter’s fundamental revenue data, as well as modern data warehouse solutions, while collaborating closely with Ads Data Science team.


You will be a part of an early stage team and have a significant stake in defining its future with a considerable potential to impact all of Twitter’s revenue and hundreds of millions of users.


You will be among the earliest adopters of bleeding-edge data technologies, working directly with Revenue Science and Revenue Platforms teams to integrate your services at scale.


Your efforts will reveal invaluable business and user insights, leveraging vast amounts of Twitter revenue data to fuel numerous Revenue teams including Ads Analytics, Ads Experience, Ads Data Science, Marketplace, Targeting, Prediction, and many others.




Qualifications





  • Strong programming and algorithmic skills

  • Experience with data processing (such as Hadoop, Spark, Pig, Hive, MapReduce etc).

  • Proficiency with SQL (Relational, Redshift, Hive, Presto, Vertica)


Nice to have:



  • Experience writing Big Data pipelines, as well as custom or structured ETL, implementation and maintenance

  • Experience with large-scale data warehousing architecture and data modeling

  • Proficiency with Java, Scala, or Python

  • Experience with GCP (BigQuery, BigTable, DataFlow)

  • Experience with Druid or Apache Flink

  • Experience with real-time streaming (Apache Kafka, Apache Beam, Heron, Spark Streaming)

  • Ability in managing and communicating data warehouse project plans to internal clients




Additional Information



All your information will be kept confidential according to EEO guidelines.

This job is expired. Please use the search form to find active jobs or submit your resume.


Post a resume

Similar jobs

Company Description Consumer Products is responsible for bringing you all the features of Twitter to Android. This team impacts hundreds of millions of users, which brings with it interesting product and technical challenges at that scale. We keep development lightweight...
San Francisco, CA
Square
Full time Full day
Company Description Square builds common business tools in unconventional ways so more people can start, run, and grow their businesses. When Square started, it was difficult and expensive (or just plain impossible) for some businesses to take credit cards. Square...
San Francisco, CA
Company Description Square builds common business tools in unconventional ways so more people can start, run, and grow their businesses. When Square started, it was difficult and expensive (or just plain impossible) for some businesses to take credit cards. Square...
San Francisco, CA
MicroTasks.io

Want to take classes online or need someone to deliver food?

All from the comfort of your home.

LEARN MORE