Automated End-to-end (E2E) Computer Vision Solution
Created a system that performed several things in real-time, including:• Detecting objects in the room • Classifying person poses• Automated re-training (active learning)• Model and data versioning• Dockerized pipelineUsing those models and predictions, we created a post-processing pipeline for creating reports or key performance indicators (KPIs) for clients.
Android COVID-19 Test Classification
The goal was to create a COVID-19 test classification model. We had a small dataset and had to build the best model in the shortest possible time (two weeks). I led a team of two people on this project. We used MobileNet due to size, and all business-relevant metrics were great. We used many optimization techniques to deploy the model to Android, such as quantization, pruning, and knowledge distillation.
MLOps Engineer
Participated in a project where my job was to optimize the whole machine learning system using quantization, pruning, ONNX, and more. I achieved the same accuracy with five times reduced latency, two times reduced model size, and four times reduced cost. I also changed the type of underlying EC2 instances to get more of our system.
Image Super Resolution
The goal was to improve the model for upscaling and super-resolution by researching and developing approaches from SOTA research papers. There were a lot of different custom loss function, layers, metrics, and even custom back propagations.
ETL Jobs
• Created batch ETL jobs for calculating KPIs.• Optimized solution to reduce cost and calculation time.• Scheduled jobs via Airflow and Prefect.The tech stack was: Spark, Scala, AWS S3, Kafka, Apache Airflow, and Prefect.
NLP Articles Processing
The goal of this project was to develop two stages of article processing: 1. Find all relevant tags (events, locations, names, etc.) in the article.2. Find pairs of tags that are somehow related.Hugging Face transformers were mainly used to tackle this problem (BERT-based models). Overall metrics were above 95%.
Data Ingestion
Led a team whose goal was to get data from the GraphQL database and insert it into Azure SQL. Everything was Dockerized and pushed to EKS on every push to the main branch on GitLab. Concurrent threads were used in order to optimize the solution.
Tech Leadership for the DE project
My responsibility was to make all decisions from architectural to the nitty gritty details about the implementation. We used AWS for infra (CloudWatch, Glue, S3) and Airflow to orchestrate Spark jobs. Every result of a Spark job was saved to Snowflake.