Publicidad

Senior Data Engineer

Publicidad
Publicidad
Senior Data Engineer
Empresa:

Mastercontrol


Publicidad

Senior Data Engineer

Publicidad

Detalles de la oferta

About MasterControl:
MasterControl Inc. is a leading provider of cloud-based quality and compliance software for life sciences and other regulated industries. Our mission is the same as that of our customers to bring life-changing products to more people sooner. The MasterControl Platform helps organizations digitize, automate and connect quality and compliance processes across the regulated product development life cycle. Over 1,000 companies worldwide rely on MasterControl solutions to achieve new levels of operational excellence across product development, clinical trials, regulatory affairs, quality management, supply chain, manufacturing and postmarket surveillance. For more information, visit www.mastercontrol.com.

Job Summary
At MasterControl we are building our next generation data platform that will leverage AI/ML techniques to help redefine how our customers bring lifesaving and lifechanging products to market. To enable this, we need your help building our Data Pipeline, Data Mart, and Data Lake.
You will be responsible for implementing the data lifecycle consumption from the data source to the end model. By gathering requirements from Product, you will transform data in near real-time which will be consumable by AI model training, business intelligence analytic tools, and self-service platforms. You will need a strong emphasis on data modeling, as well as proficiency in data transformation. You will be responsible for integration into the CI/CD pipeline as well as unit and integration testing of all processes. You will need experience with structured, semi-structured, and unstructured data sources. You will need a bachelor's degree in a STEM related field, or equivalent experience. You should have performed multiple successful deployments of production Big Data systems.
Responsibilities
Pull data from all customer databases into Kafka
Model the data for optimal performance (ETL, star-schema like, ORC, partitioning, etc ...)
Pull data from a real-time streaming architecture (Kafka) and do near-real-time aggregations and projections of the data, storing the results in S3
Help automate the provisioning of AWS Lake Formations, EMR/Spark/S3/Kafka and other services in AWS
Analyze data to find patterns worthy to expose to the end-user
Help tie corporate data to customer data from an OLTP store
Help us discover ways to use Big Data technologies in a Machine Learning pipeline (discover, clean, label, train, test)
Other assigned duties

Skills
Kafka, Spark, Airflow, Hudi, S3
Scala, Python, SQL, Java
Warehouse Architectures (Star Schema/Snowflake/Vaults/Lakes)
Familiarity with AWS and Cloud Formation or Terraform
Data Modelling
ELT/ETL Best Practices
Apache Spark with EMR and/or other Big Data tools
Kafka Streams experience is a plus
Airflow experience is a plus
Big Data Mindset (Spark/Hive/Hadoop/HCatalog/Hudi). Understanding of the Big Data landscape.
Meet multiple, challenging deadlines while communicating expectations clearly.

Physical Demands And Working Conditions
Must be able to work well with people.
Ability to operate a computer and work at a desk for extended periods of time.
Ability to communicate effectively in writing, in person, over the telephone and in e-mail.

Why Work Here?
#WhyWorkAnywhereElse?
MasterControl is a place where Exceptional Teams come together to do their best work. In fact, hiring Exceptional Teams is a core value of ours. MasterControl employees are surrounded by intelligent, motivated, and collaborative individuals. We like to call it #TheBestTeamOnThePlanet.
We work hard to develop and challenge our employees' skillsets, recognize their contributions, encourage professional development, and offer a one-of-a-kind culture. This is why we say #WhyWorkAnywhereElse?
MasterControl could be your next (and last) career move!
Here are some of the benefits MasterControl employees enjoy:
Competitive compensation
100% medical premium coverage (yes, you read that right!)
401(k) plan with company match
Generous PTO packages that increase with tenure
Schedule flexibility
Fitness clubs (you get paid to have fun and be active!)
Company parties and employee recognition programs
Wellness programs (free Fitbit, gym membership and athletic shoe reimbursements, etc.)
Onsite physician and massage therapist
Innovation center and gaming rooms at the office
Dental/vision plans
Employer paid life insurance policy
Much, much more!

Applicants must be currently authorized to work in the United States on a full-time basis.

Requisitos


Conocimientos:
Publicidad
Senior Data Engineer - Big Data

About MasterControl: MasterControl Inc. is a leading provider of cloud-based quality and compliance software for life sciences and other regulated industrie...


Desde Mastercontrol - Santa Cruz

Publicado 19 days ago

Java Engineer (Microservices)

About MasterControl: MasterControl Inc. is a leading provider of cloud-based quality and compliance software for life sciences and other regulated industrie...


Desde Mastercontrol - Santa Cruz

Publicado 7 days ago

Desarrollador Java

Requisitos:+3 años de experiencia en Angular.Js y Software development. Requerimientos adicionales (que no sean habilidades): AngularJS (versión 1.6) ...


Desde Torre - La Paz

Publicado a month ago

Android Mobile Developer

Requisitos:Scrummers continues to rapidly expand, and for this reason, we are looking to compliment our team with the best talent capable of growing with us....


Desde Torre - La Paz

Publicado a month ago

Built at: 2021-07-26T00:22:58.185Z