Talk: Growing our data ingestion capacity by 10x
Speakers directory
Speaker:
Sarah Usher
Talk description
Title:
Growing our data ingestion capacity by 10x
Short synopsis:
Our data pipeline didn’t have any optimisations left to offer - we had to do something new and we needed to think big - 10 times big! In this talk I’ll take you through what our team of 7 developers did to replace our data pipeline, bit by bit, with minimal business disruption.
Max size: 500 chars
Long synopsis (optional):
We’d recently completed a number of optimisations to our build pipeline, growing our capacity from 50k events per second (eps) to 80k events per second and we were feeling pretty good, but in the space of three months, the business used up this capacity due to rapid business growth. But our pipeline didn’t have any optimisations left to offer - we had to do something new and we needed to think big - 10 times big! In this talk I’ll take you through what our team of 7 developers did to replace our data pipeline, bit by bit, with minimal business disruption.
Max size: 5000 chars
Tags:
Speaker directory:
Listed in directory
Not listed
Speakers directory