Moving to NATS instead of dragonboat due to it's inefficiencies. #11
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I absolutely want to document this because this is a pretty major change, and it's for good reason. I recently filed a bug on dragonboat implementation on how I was able to crash state machine due to bug in the code base while it claims itself to be PRODUCTION READY. I tried visiting their Gitter, looked at the code from various aspects:
internal
inv3.x
.v4.x
claims to be beta, and given how it was absolutely killing me I am not willing to even go close to it.SQLiteLogDB
implementation was absolute nightmare to implement (In an effort to remove dependency on PebbleDB). I had to read the specifics of each function and implement the same KV based function on to SQLite based function. Now some of you might blame me for not sticking to PebbleDB, but I will lay it out right now I would rather have one storage engine than having multiple engines to deal with.Given that and the issue here which led to look at NATS to begin with, it was very obvious to me that for streaming, NATS was doing everything I needed to reliably replicate change logs, it was matter of just kicking off the server, and then talking to it. Given that there was already interest on numerous forums that people wanted to get CDC as part of bigger system, I had to use NATS client anyway, and embedding NATS server is no big deal as well.
I benchmarked with NATS for couple of hours, and the maturing of project just spoke to me. This PR takes the ugly step of adding dependency on NATS which I from my heart believe will be good future of Marmot. I know people might hate me for adding one more moving piece, but that's what UNIX philosophy is; and I am pretty sure just like me getting impressed by elaborate tooling, you will be impressed by how easy it will make your life as well.
So here is to a bright stable Marmot 🚀