Distributed Filesystem
A distributed file storage layer with replication, fault tolerance, and consistent metadata management. I'm building it because I want to really understand how real storage systems handle failure, and not just read about it.
This one comes straight out of my distributed systems class at Michigan. I got genuinely hooked on the idea that so much of systems design isn't about writing clever code, it's about deciding what you're willing to give up. You trade consistency for availability, or simplicity for performance, and then you have to live with that trade-off correctly under failure. Reading about that in a textbook only goes so far, so DistFS is me actually building a storage layer that has to make those calls for real. It chunks files and spreads them across nodes, replicates them so a single node failure doesn't lose data, and keeps metadata consistent about which node holds which chunk even as the cluster changes shape underneath it.
The honest goal is a filesystem I'd eventually trust to store my own files on my own hardware. That's a good forcing function, because "good enough for a demo" and "good enough that I'd actually rely on it" are very different bars.
DistFS is built around a set of storage primitives. Files are chunked and spread across nodes, replicated so a single node failure doesn't lose data, and tracked in metadata that records which node holds which chunk. I deliberately started with these before touching anything like consensus, because I wanted the foundational data path to be solid before building coordination logic on top of it.
Every design decision in this project eventually reduces to the same tension. How much coordination am I willing to pay for on the write path, so a read never returns stale or missing data after a node drops out? Go's goroutines and channels make the concurrent part of that manageable to reason about, since there are many nodes and many in-flight requests, but they don't make the actual trade-off decision for you. That's still the part I spend the most time thinking through before I write any code.
Failure detection and re-replication are the pieces I'm actively working through right now. The real question is how aggressively to react to a node going quiet, since a slow node and a dead node look identical for a little while, without triggering a storm of unnecessary re-replication traffic every time something hiccups. It's the same class of problem as a distributed systems paper's "failure detector." It just feels a lot more concrete when it's your own cluster timing out.
