-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable client side gRPC health check by default #18882
base: main
Are you sure you want to change the base?
Conversation
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ahrtr The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
6cc5951
to
5c8a02f
Compare
Please anyone feel free to work on this on top of this PR. |
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files
... and 20 files with indirect coverage changes @@ Coverage Diff @@
## main #18882 +/- ##
==========================================
+ Coverage 68.72% 68.73% +0.01%
==========================================
Files 420 420
Lines 35532 35537 +5
==========================================
+ Hits 24418 24428 +10
- Misses 9681 9687 +6
+ Partials 1433 1422 -11 Continue to review full report in Codecov by Sentry.
|
5c8a02f
to
af21881
Compare
@@ -218,7 +219,7 @@ func newStreamInterceptor(s *etcdserver.EtcdServer) grpc.StreamServerInterceptor | |||
return rpctypes.ErrGRPCNotCapable | |||
} | |||
|
|||
if s.IsMemberExist(s.MemberID()) && s.IsLearner() && info.FullMethod != snapshotMethod { // learner does not support stream RPC except Snapshot | |||
if s.IsMemberExist(s.MemberID()) && s.IsLearner() && !isRPCStreamSupportForLearner(info) { // learner does not support stream RPC except Snapshot |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How this change is related to health checks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gRPC client will call the Watch
RPC on the health check service automatically for the stream RPC. Refer to https://grpc.io/docs/guides/health-checking/#enabling-client-health-checking
Most likely we will do similar change for isRPCSupportedForLearner
as well,
etcd/server/etcdserver/api/v3rpc/util.go
Lines 142 to 151 in 7ab7612
func isRPCSupportedForLearner(req any) bool { | |
switch r := req.(type) { | |
case *pb.StatusRequest: | |
return true | |
case *pb.RangeRequest: | |
return r.Serializable | |
default: | |
return false | |
} | |
} |
The client side sees a grpc error, not sure why.
I am pretty sure that we have registered the health service and set the etcd/server/etcdserver/api/v3rpc/grpc.go Lines 77 to 79 in 7ab7612
|
@ahrtr Side effect of importing |
Maybe you need an import that sets Any danger to roll this out enabled by default? Do we need a config option? I think, yes. The change is minimal but if there are any bugs in grpc health impl, we might need a way to disable. |
af21881
to
b7c8ad2
Compare
Thanks both. It's a bad pattern to have a blank-import in a non-main package,
|
b7c8ad2
to
f7ab407
Compare
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
f7ab407
to
a9f846b
Compare
/retest |
@ahrtr: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
YES, we definitely need a config option for this; otherwise it will be a breaking change. If the server side health check isn't enabled, the client will will get an error something like below,
Please anyone feel free to continue to work on this task on top of this PR,
|
assigned to myself |
thx |
Followup to #16278
Previously the default gRPC service config for the resolver is
{"loadBalancingPolicy": "round_robin"}
.etcd/client/v3/internal/resolver/resolver.go
Line 44 in 7ab7612
Now I propose to change the default service config to
{"loadBalancingPolicy": "round_robin"}, "healthCheckConfig": {"serviceName": ""}
. The benefit is thatloadBalancingPolicy
isround_robin
? @dfawley But I am not whether is there any performance penalty? Probably we need to run rw-heatmaps to double confirm.