Either while training an llm or writing apis to query through millions of rows, batch streaming can be a helpful solution to go through the data with by splitting data in batches and parallel processing. What streaming solutions do you use for these purposes in your workflow?
submitted by /u/metalvendetta
[link] [comments]
