125: Data Types: Streams Part 2.

Streams provide a way to read and write potentially unlimited information and working with them is very different than data types representing a single variable. This episode continues the explanation of streams. There’s more than just reading and writing to them. One very important aspect is buffering which is like batching. Buffering allows you to save writes and when there’s been enough, then they all get written to the stream destination at once. One example that comes to mind is not really accurate at first but I’ll explain why it’s not a good example and then start changing it until it’s better. Sometimes coming up with something that’s not quite right is a great way to understand a topic as long as you know why. This episode describes some other features of streams that you’ll want to make sure to listen to. You can also read the full transcript below. Transcript This episode continues the explanation of streams. There’s more than just reading and writing to them. One very important aspect is buffering which is like batching. Buffering allows you to save writes and when there’s been enough, then they all get written to the stream destination at once. One example that comes to mind is not really accurate but I’ll explain it anyway and then explain why it’s not a good example. Sometimes coming up with something that’s not quite right is a great way to understand a topic as long as you know why. Let’s say you want to organize a camping trip but the expenses are too much for you to consider going alone. So you start asking friends and people you know if anybody is interested to go with you. You tell them that unless you get enough people, the trip will be canceled. If you get enough people, then the costs are divided and the trip becomes affordable. This is a good example of buffering because you and your friends are delayed for a bit but eventually go on the trip that would have been too expensive for anybody to afford alone. The reason this is not a good example of buffering is because it can be canceled. Stream buffering will delay writes until there are enough to justify the costs of writing the information to the stream. But it wouldn’t be used very much if the writes could be abandoned. A better example is similar. But instead of telling your friends that the trip will be canceled, the message is to try getting as many people to join in order to reduce costs and that everybody should be ready to go anyway even if not enough people join. This fits the way buffering works. Once you’ve committed to go on the trip, just like once some data has been written to a stream, then you know you’ll eventually make it to the destination. And who knows, if you get enough people to sign up, then you might be able to make two camping trips or more. Streams usually have a buffer size that will hold writes until it fills up. The information will be sent to the destination as many times as the buffer fills up. Some streams may not be buffered though. This would be like a camping trip where the camp sites can only hold one person. You might as well proceed to the destination right away if there’s only room for one person anyway. That’s a much better example and I’ll add more to it right after this message from our sponsor. It’s possible for a stream to change from buffered to unbuffered and back. I don’t remember ever needing to do this myself. I can see benefits in an adaptive stream that changes it buffer size according to outside conditions though. Normally, when you create a stream, you’ll specify if it should be buffered or not. And when you’re using an existing stream, you’ll want to know if it’s buffered or not. I mentioned earlier that you don’t have to worry about data written to a stream reaching the destination once it’s been written but this is under normal condition