Apply without generating response

[m-mueller678]

When applying requests to the state machine, responses are always produced, even though they are only used on the leader as far as I can tell. In the project I am working on, there are many requests to the state machine that are read only, but are expensive to compute. On the followers, I would like to avoid this computation, as the response will be discarded anyway and the requests do not change the state machine.

I would like a variation of RaftStateMachine::apply that produces no response and is called whenever the response would be discarded anyway. It could defer to the normal apply implementation by default to not complicate implementing RaftStateMachine:

async fn apply_without_response<I>(
    &mut self,
    entries: I,
) -> Result<(), StorageError<C::NodeId>>{
    self.apply().await.map(|_|())
}

Alternatively, an extra parameter could be added to apply to control response generation, but that would put more burden on a minimal state machine implementation.

A way for the leader to read directly from the state machine without writing a log would be preferred for my specific use case, but there could also be write-requests where the response is expensive to generate.

[github-actions]

👋 Thanks for opening this issue!

Get help or engage by:

• /help : to print help messages.
• /assignme : to assign this issue to you.

[schreter]

@m-mueller678 But why are using client_write() to start read-only requests???

You can directly read from the state machine on the leader. Ensuring the leader state is possible via several methods, for example, subscribing to metrics which will tell you when the state changes.

Alternatively, you can use Raft::ensure_linearizable() to ensure the read is linearizable across the cluster (see comments there).

Independent of that, not producing replies for state machine apply on the follower might be interesting. We do that indirectly, our reply type is () and we use a different channel to send the reply. See also Raft::client_write_ff().

[m-mueller678]

I was not aware I could read from the state machine directly. How do I obtain a reference to the state machine from a Raft<C> handle? I think the documentation on Raft::ensure_linearizable should go into more details on that.

[schreter]

You can use Raft::external_state_machine_request() to directly access the state machine. OTOH, probably only in master.

But, the implementation of RaftStateMachine is under your control. So simply implement it in a way that you keep some form of reference to it.

[SteveLauC]

OTOH, probably only in master.

Right, this new function has not been released.

So simply implement it in a way that you keep some form of reference to it.

This makes more sense, e.g., wrap your actual state machine in an Rc or Arc, like what this raft-kv-memstore-singlethreaded example does

[drmingdrmer]

Distinguishing between different response types (e.g., T or ()) can be tricky, but here’s how it can be approached:

1. Decouple it from server state:
   The response type should depend on the log entry itself, not the server state (Leader or Follower). Server state can change while the state machine is applying log entries, so relying on it could lead to issues.

2. Decide based on log entry origin:
   The response type should depend on how the log entry was created:

   • If the log entry is produced by Raft::client_write(), it needs a response.
   • If it’s created as part of the replication stream (Leader → Follower), it doesn’t need a response.

   Essentially, if a log entry has a channel sender bound to it, it should trigger a response.

3. Mark entries with a need_response attribute:

   • Add an attribute to log entries to indicate if a response is required.
   • When the entry is clone()-ed or deserialized (e.g., for transmission to a Follower), clear this attribute to show the new copy doesn’t require a response.
   • Also, you may need to update the response type from T to Option<T> for better clarity.

In short:
Only the first copy of a log entry (the one created by Raft::client_write()) should have the need_response attribute set to true. This ensures that responses are handled cleanly and avoids relying on server state.