Primary Responsibilities:
- Developing and running pipelines for data ingress and model output egress
- Developing and running scripts for ML model inference
- Design, implement, and maintain CI/CD pipelines for MLOps and DevOps functions
- Identifying technical problems and developing software updates and ‘fixes’
- Developing scripts or tools to automate repetitive tasks
- Automate the provisioning and configuration of infrastructure resources
- Provide guidance on how to best use specific tools or technologies to achieve the desired results
- Creating documentation for infrastructure design and deployment procedures
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Required Qualifications:
- Undergraduate degree or equivalent experience
- Proficient in Python and one of PySpark or Scala. Familiarity with python tools for data processing
- Ability to develop and deploy data pipelines, machine learning models, or applications on cloud platforms (Azure, AWS,
Databricks, AzureML) - Knowledge of the software development life cycle
- Linux shell scripting
- DevOps Skills:
- Experience building and maintaining CI/CD pipelines
- Familiarity with traditional sofware monitoring, scaling, and quality management (QMS)
- Experience with DevOps tools (Github, GitHub Actions, Docker, Kubernetes, etc.)
- Experience with one or more data-oriented workflow orchestration frameworks (Prefect, Dagster, Kedro, Airbyte, KubeFlow, Airflow, Argo, etc.)
- Security and vulnerability management
Preferred Qualifications:
- Experience deploying and maintaining ML models in production
- Experience with model observability tools for insights into the behavior, performance, and health of your deployed ML models (tracking, alerting, compliance monitoring, etc.)
- MLOps Skills
- Familiarity with model versioning tools (MLFlow, etc.)
- Familiarity with data versioning tools (Delta Lake, DVC, LakeFS, etc.)