New ask Hacker News story: Ask HN: Help with exponentially growing ClickHouse setup
Ask HN: Help with exponentially growing ClickHouse setup
4 by ardabulut | 0 comments on Hacker News.
Hey everyone, Long story short, we are a data analytics company from S23 batch storing currently >20 billion rows of data and it is exponentially growing. There are a lot of custom properties we have to store in JSONs (over 100 KBs per row) and many mutable datapoints (we do daily syncs with other platforms). If you've ever worked with ClickHouse then you'll know we are pushing it to its unintended limits here :D We just got three of our biggest customers last month and the trend is that it is going to keep on growing from here on out. We have 8 ClickHouse servers with 128gb memory and it still doesn't feel enough. We are looking for some og ClickHouse experts to give us some advice on how to scale this setup to handle 100x the load. I'm sure we'll hit that scale by next year. You can reach out to me from arda [at] hockeystack [dot] com if you are interested in getting your hands dirty with a big production system in 30 minutes. Even better if you are in SF, I'll buy you a coffee :)
4 by ardabulut | 0 comments on Hacker News.
Hey everyone, Long story short, we are a data analytics company from S23 batch storing currently >20 billion rows of data and it is exponentially growing. There are a lot of custom properties we have to store in JSONs (over 100 KBs per row) and many mutable datapoints (we do daily syncs with other platforms). If you've ever worked with ClickHouse then you'll know we are pushing it to its unintended limits here :D We just got three of our biggest customers last month and the trend is that it is going to keep on growing from here on out. We have 8 ClickHouse servers with 128gb memory and it still doesn't feel enough. We are looking for some og ClickHouse experts to give us some advice on how to scale this setup to handle 100x the load. I'm sure we'll hit that scale by next year. You can reach out to me from arda [at] hockeystack [dot] com if you are interested in getting your hands dirty with a big production system in 30 minutes. Even better if you are in SF, I'll buy you a coffee :)
Comments
Post a Comment