Build Super Fast gRPC APIs With Go and Java with Example

[gRPC] FieldMask — only request for needed data

gRPC with Java Server (Client) and Go Client Example with FieldMask

gRPC is not Google Remote Procedure Call 😀

gRPC stands for gRPC Remote Procedure Call and was initally developed by Google.

It is an open-source, high-performance remote procedure call (RPC) framework that can run anywhere.

It efficacy allows client applications to directly call a method on a server application on a different machine in a different data center, written in a different programming language as if it were a local method, making it easier for us to create distributed applications and services.

gRPC Communication Sample

gRPC uses HTTP/2 protocol for transport, and Protocol Buffers as the interface definition language (IDL).

It also provides features such as authentication, bidirectional streaming and flow control, blocking or nonblocking bindings, and cancellation and timeouts.

What is FieldMask?

Field Mask is a simple but very powerful concept of ProtoBuf. It can help make APIs more robust and service implementations more efficient.

It is similar to Sparse Fieldsets of JSON API specification that we can use in REST APIs.

Clients can use FieldMask to specify a list of resource data members that they need from the Servers instead of fetching the whole resource object which can be big in size and can increase network cost/latency, also server might need time to build the whole resource object ( it might be fetching data from different locations).

We will create a User message/resource which will have user’s basic details and address. We will create gRPC APIs for greeting users and fetching random user.

In fetching random user API we will use FieldMask to not fetch the Address of the user (in real time use case, the address might be stored in a different table/database/place).

We will create proto files, and implement the server in Java and the Client in Go.

Prerequisite

  • Java 8+
  • Go 1.16+ (Optional, only if you want to create client in Go)
  • Gradle or Maven

Proto Files

gRPC service is defined using protocol buffers.

Protocols are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data in a forward-compatible buffer and backward-compatible way. It’s like JSON, except it’s smaller and faster, and it generates native language bindings.

We will define protocol buffers for our use case. We will define the user message proto and greet user service pro. Let’s create a directory called user_proto and create our proto files here for now. It would look something like this:

syntax = "proto3";

package com.grpc.example;

option java_multiple_files = true;
option java_package = "com.grpc.example.greet.model";
option go_package = "./pb";

import "google/protobuf/timestamp.proto";

message User {
string firstName = 1;
optional string middleName = 2;
optional string lastName = 3;
optional uint32 age = 4;
repeated UserAddress address = 5;
google.protobuf.Timestamp currentTime = 6;
}

message UserAddress {
string addressLine1 = 1;
optional string addressLine2 = 2;
string city = 3;
string country = 4;
uint32 pincode = 5;
enum ADDRESS_TYPE {
BOTH = 0;
PERMANENT = 1;
CURRENT = 2;
}
}

syntax = "proto3";

package com.grpc.example;

option java_multiple_files = true;
option java_package = "com.grpc.example.greet.service";
option go_package = "./pb";

import "user_message.proto";
import "google/protobuf/field_mask.proto";

message GreetUserRequest {
User user = 1;
}

message GreetUserResponse {
string greetMessage = 1;
}

message GetRandomUserRequest {
google.protobuf.FieldMask field_mask = 1;
}

message GetRandomUserResponse {
User user = 1;
}

service GreetUserService {
rpc GreetUser(GreetUserRequest) returns (GreetUserResponse) {};
rpc GetRandomUser(GetRandomUserRequest) returns (GetRandomUserResponse) {};
}

We can see that in GetRandomUserRequestwe have added field_mask as a parameter.

Server-Side Implementation (Java with Gradle)

Let’s create a simple Gradle application and import it into our IDE. I will use IntelliJ IDEA for this.

Let’s make changes in build.gradle file for adding gRPC and protobuf support. We will add a few dependencies, protobuf plugin, and instructions for generating class implementations of proto files.

We will add the following dependencies :

implementation group: 'io.grpc', name: 'grpc-all', version: '1.45.0'implementation group: 'com.google.protobuf', name: 'protobuf-java', version: '3.15.0'implementation group: 'javax.annotation', name: 'javax.annotation-api', version: '1.3.2'

We will add the following plugin :

id "com.google.protobuf" version "0.8.18"

We will add the following instructions for proto files and class generation :

protobuf {
protoc {
artifact = 'com.google.protobuf:protoc:3.15.0'
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.45.0'
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
sourceSets {
main {
java {
srcDirs 'build/generated/source/proto/main/grpc'
srcDirs 'build/generated/source/proto/main/java'
}
}
}

We will now create a directory in src/main called proto, and we will copy our protocol buffer files here in this directory. This is because our sourceSet default configuration is to read proto files from this location, we can change it as per our need.

Now let’s try to build our project. It should create stubs of the proto files in the packages mentioned in the sourceSets of build.gradle.

We should create our GreetingService.java class implementation that will extend GreetUserServiceGrpc.GreetUserServiceImplBase.java class which is auto-generated, and override greetUser and getRandomUser method with our own implementation.

It should look similar to this:

public class GreetingService extends GreetUserServiceGrpc.GreetUserServiceImplBase {

@Override
public void greetUser(GreetUserRequest request, StreamObserver<GreetUserResponse> responseObserver) {
User user = request.getUser();
String greetMessage = createGreetMessage(user);
GreetUserResponse response = GreetUserResponse.newBuilder().setGreetMessage(greetMessage).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}

private String createGreetMessage(User user) {
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append("Hello, ")
.append(user.getFirstName())
.append(user.hasMiddleName() ? " "+user.getMiddleName() : "")
.append(user.hasLastName() ? " "+user.getLastName() : "")
.append("!");
if(user.getAddressCount() > 0) {
stringBuilder.append("nI hope things are good in ")
.append(user.getAddress(0).getCity())
.append(", ")
.append(user.getAddress(0).getCountry());
}
return stringBuilder.toString();
}

@Override
public void getRandomUser(GetRandomUserRequest request, StreamObserver<GetRandomUserResponse> responseObserver) {

User user = User.newBuilder().setFirstName("John").setLastName("Doe").setAge(26).build();

for (String s : request.getFieldMask().getPathsList()) {
if(s.equalsIgnoreCase("address")) {
UserAddress userAddress = UserAddress.newBuilder()
.setAddressLine1("BR 15")
.setCity("PN")
.setCountry("IN")
.build();
user = user.toBuilder().addAddress(userAddress).build();
}
}

User.Builder resUserBuilder = User.newBuilder();

FieldMaskUtil.merge(request.getFieldMask(), user, resUserBuilder);

GetRandomUserResponse response = GetRandomUserResponse.newBuilder().setUser(resUserBuilder).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}

We can see in the getRandomUser method implementation, we have used field mask to first check if address field is needed or not and after that we have used FieldMaskUtil.merge to make an object with only asked data.

We should now create our GreetingServer.java class and write code for starting the gRPC server. It should look similar to this:

public class GreetingServer {

private final int port;
private final Server server;

public GreetingServer(int port) {
this(ServerBuilder.forPort(port), port);
}

public GreetingServer(ServerBuilder serverBuilder, int port) {
this.port = port;
GreetingService greetingService = new GreetingService();

server = serverBuilder.addService(greetingService).build();
}

public void start() throws IOException {
server.start();
System.out.println("Server Started!!!");
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
try {
GreetingServer.this.stop();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
}

public void stop() throws InterruptedException {
if(server != null) {
server.shutdown().awaitTermination(30, TimeUnit.SECONDS);
}
}

public void blockTillShutdown() throws InterruptedException {
if(server!=null) {
server.awaitTermination();
}
}

public static void main(String[] args) throws InterruptedException, IOException {
GreetingServer greetingServer = new GreetingServer(9090);
greetingServer.start();
greetingServer.blockTillShutdown();
}

}

It will start our server at 9090 port. Please follow official documentation link here for more details on gRPC development with Java.

We can now write client code to consume this API.

Client Side Implementation (GoLang)

Let’s create a simple Go application named grpc-fieldmask-example-go and open it in our IDE. I will use Visual Studio Code for this.

Let’s first install the Protocol Buffer Compiler, click here for more details.

brew install protobufprotoc --version #please ensure if compiler version is 3+

Now we should install protocol compiler plugins for Go using the following commands:

go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2export PATH="$PATH:$(go env GOPATH)/bin"

First, we will create a proto directory and copy the proto files here (similar to what we did in Java)

The next step is to create the stubs from the proto files, we will use the following command for this:

protoc --proto_path=proto proto/*.proto --go_out=. --go-grpc_out=.

It will create a package/directory call pb and generate the stubs in this package. We have defined this package name in the proto files.

We will now create our main go file where we will write gRPC client code, and make requests to the server that is running in Java. It would look something like this:

package mainimport (
"context"
"flag"
"fmt"
"log"
"time"
"grpc-fieldmask-example-go/pb""google.golang.org/grpc"
"google.golang.org/protobuf/types/known/fieldmaskpb"
)
var (
addr = flag.String("addr", "localhost:9090", "the address to connect to")
)
func main() {
fmt.Println("main method")
serverAddress := flag.String("address", "localhost:9090", "the server address")
flag.Parse()
log.Printf("Connecting to %s", *serverAddress)
conn, err := grpc.Dial(*serverAddress, grpc.WithInsecure())
if err != nil {
log.Fatalf("error while connecting: %v", err)
}
defer conn.Close()
client := pb.NewGreetUserServiceClient(conn)ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
callGreetUser(client, ctx)callRandomUser(client, ctx)}func callGreetUser(client pb.GreetUserServiceClient, ctx context.Context) {
lastName := flag.String("lastName", "Doe", "last name of user")
var age uint32 = 26
user := pb.User{FirstName: "John", LastName: lastName, Age: &age}req := pb.GreetUserRequest{User: &user}r, err := client.GreetUser(ctx, &req)if err != nil {
log.Fatalf("could not greet: %v", err)
}
log.Printf("Response From Server - Greeting : %s", r.GetGreetMessage())
}
func callRandomUser(client pb.GreetUserServiceClient, ctx context.Context) {var paths []string = []string{"firstName", "lastName", "age"}field_mask := fieldmaskpb.FieldMask{Paths: paths}req := pb.GetRandomUserRequest{FieldMask: &field_mask}r, err := client.GetRandomUser(ctx, &req)if err != nil {
log.Fatalf("could not get random user : %v", err)
}
log.Printf("Response From Server - Random User : %s", r.GetUser())}

Let’s now execute the following command to create a module and add all the missing dependencies of our application:

go mod init grpc-fieldmask-example-gogo mod tidy

Everything is done now. When you execute this go code by using the following command, it should make an API request to our Java server which should be running and print a response in our console:

go run main.go

Try with different fields in the []string array, and it should reflect in the output.

As a bonus, I am going to share the same client code written in Java below

public class GreetClient {

private final ManagedChannel managedChannel;
private final GreetUserServiceGrpc.GreetUserServiceBlockingStub blockingStub;

public GreetClient(String host, int port) {
managedChannel = ManagedChannelBuilder.forAddress(host, port).usePlaintext().build();
blockingStub = GreetUserServiceGrpc.newBlockingStub(managedChannel);
}

public void shutdown() throws InterruptedException {
managedChannel.shutdown().awaitTermination(30, TimeUnit.SECONDS);
}

public String getGreetingMessage(User user) {
GreetUserRequest greetUserRequest = GreetUserRequest.newBuilder().setUser(user).build();
GreetUserResponse greetUserResponse = GreetUserResponse.getDefaultInstance();

greetUserResponse = blockingStub.greetUser(greetUserRequest);

return greetUserResponse.getGreetMessage();
}

public User getRandomUser() {
FieldMask fieldMask = FieldMask.newBuilder()
.addPaths("firstName")
.addPaths("lastName")
.addPaths("age")
.build();

GetRandomUserRequest getRandomUserRequest = GetRandomUserRequest.newBuilder().setFieldMask(fieldMask).build();

GetRandomUserResponse getRandomUserResponse = blockingStub.getRandomUser(getRandomUserRequest);
return getRandomUserResponse.getUser();
}

public static void main(String[] args) {
GreetClient greetClient = new GreetClient("localhost",9090);

User user = User.newBuilder()
.setFirstName("John")
.setLastName("Doe") .addAddress(UserAddress.newBuilder().setCity("PN").setCountry("IN").build())
.build();
System.out.println("Response Greeting Message : "+greetClient.getGreetingMessage(user));

User randomUser = greetClient.getRandomUser();

System.out.println("Response User : "+randomUser);

try {
greetClient.shutdown();
} catch (InterruptedException e) {
e.printStackTrace();
}

}

}

We just produced and consumed a gRPC API using both Java and Go. Also, we learned how to write proto files, generate stubs and use them in Java and Go.

We also used FieldMask which is one of the important features and if used correctly it can significantly increase the system performance and save some cost.

You can find the code of this example in this GitHub repository here.

Leave a Comment