Building APIs with Node.js and gRPC

Building a faster, more reliable, scalable, and efficient API requires the use of an appropriate framework and tools. The communication protocol used to build an API usually has a significant impact on its effectiveness and speed.

What is gRPC?

gRPC is a powerful remote procedure-call (RPC) framework that runs virtually in any environment. The performance of this framework in sending and receiving data between servers and clients is what distinguishes it from other frameworks with similar applications.

This framework was released by Google in 2015 as an open-source project, with the aim of encouraging the community to adopt and contribute to making a better protocol.

gRPC's high-performance capabilities rely on its use of Protocol Buffers, a language-independent data serialization format that provides strong typing and schema enforcement. Additionally, gRPC supports bi-directional streaming and flow control, which makes it an excellent choice for microservices architecture base system.

Since Google released gRPC in 2015 as an open-source project, it has become an industry standard for developing high-performance RPC-based systems. It facilitates high load times and low latency periods, making it a popular protocol for building APIs.

How does gRPC work?

gRPC enables a client to make a call request to a remote server-side procedure as a local function call. It uses Protocol Buffers to define the message structure.

A gRPC call serializes the request message using Protocol Buffers and sends it to the server. The server then deserializes the message, executes the procedure, generates a response message, serializes it, and sends it to the client.

gRPC supports server-streaming RPC, client-streaming RPC, and bidirectional-streaming RPC. In server streaming, RPC streams multiple messages from a server to a client. In client streaming, RPC streams multiple messages from a client to a server. In bidirectional streaming, RPC enables both a client and a server to send multiple or bi-directional messages over a single RPC call.

gRPC uses HTTP/2 as its underlying protocol, which provides multiplexing, flow control, and header compression, improving its performance and reducing latency.

Understanding Protocol Buffers

Protocol Buffers, which is also known as "Protobuf," is a language-independent, extensible, and efficient mechanism for serializing structured data. It was developed by Google and is used extensively in many of their internal systems, including gRPC.

Protobuf works by defining a schema for the data to be exchanged, which is written in a simple .proto file. This schema defines the structure of the data, including its fields and their types. Once the schema is defined, a compiler generates code in various programming languages, which can be used to serialize and deserialize data based on the schema.

The serialized data is compact and efficient, requiring less space to store or transmit than the equivalent in JSON or XML. It also allows for backward and forward compatibility of data, allowing new fields to be added to the schema without breaking existing code.

Protobuf is used in a wide range of applications, from data storage and communication protocols to message-passing between distributed systems. Its efficiency and flexibility make it a popular choice in systems that require high-performance data serialization and deserialization, such as gRPC.

How to create an API with gRPC and Node.js

Now that we understand what gRPC is and what it can do, lets build a gRPC-based API. In this project, we will build a bookstore. This bookstore contains the book title, author, and content. We will be retrieving the bookstore details using gRPC calls.

Setting up the Node.js project

Do the following to create a Node.js project:

1. Ensure that you have Node.js installed on your machine. The latest version of Node.js can be downloaded from the official website: https://nodejs.org/
2. Create a new directory for your project and navigate to it using the terminal or command prompt.
3. Initialize a new Node.js project by running the following command in the terminal:

npm init

This command will prompt you to enter some information about your project, such as the project name, version, and description. You can press Enter to accept the default values for most of these prompts.

Install the gRPC dependencies by running the following command:

npm install grpc @grpc/proto-loader

This command will prompt the installation of gRPC to the package.json file.

Creating the proto file

The proto file defines the structure of the data to be sent over the network from the client to the server and vice versa. Create a books.proto file and add the code below.

syntax = "proto3";

package bookPackage;

service Book {
  rpc allBooks(voidNoParam) returns (bookItems);
  rpc createBook(bookItem) returns (bookItem);
  rpc readBook(bookId) returns (bookItem);
  rpc updateBook(bookId) returns (bookItem);
  rpc DeleteBook(bookId) returns (bookItem);
}

message voidNoParam {}

message bookItem {
  int32 id = 1;
  string title = 2;
  string author = 3;
  string content = 4;
}

message bookItems {
  repeated bookItem items = 1;
}

message bookId {
  int32 id = 1;
}

The .proto code above defines the data structure for the book service. It includes the version of Protocol Buffers used (version 3), the namespace for the service and messages (bookPackage), and the book service with five methods: allBooks, createBook, readBook, updateBook, and deleteBook.

The message structure for voidNoParam includes an empty message used for the allBooks method parameter.

The message structure for bookItem contains id, title, author, and content fields.

The message structure for bookItems contains a repeated array of bookItem messages and a bookId message with a single id field for the readBook, updateBook, and deleteBook parameters.

Creating the server

The gRPC server defines the function on how the data is handled when it is sent or received. Create a file named server.js and paste the following code.

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader")
const packageDef = protoLoader.loadSync("books.proto", {});
const grpcObject = grpc.loadPackageDefinition(packageDef);
const bookPackage = grpcObject.bookPackage;

const server = new grpc.Server();

let books = [
  { id: '1', title: 'Note 1', author: "Munroe", content: 'Content 1'},
  { id: '2', title: 'Note 2', author: "Maxwell", content: 'Content 2'}
]

server.addService(bookPackage.Book.service,
  {
      "allBooks": allBooks,
      "createBook": createBook,
      "readBook": readBook,
      "updateBook": updateBook,
      "deleteBook": deleteBook
  });

function createBook (call, callback) {
  const book = call.request;
  book.id = books.length + 1;
  books.push(book);
  callback(null, { books });
}

function readBook (call, callback) {
  const book = books.find(n => n.id == call.request.id);

  if (book) {
      callback(null, book);
  } else {
      callback({
          code: grpc.status.NOT_FOUND,
          details: "Not found"
      });
  }
}

function updateBook (call, callback) {
  const existingBook = books.find(n => n.id == call.request.id);
  if (existingBook) {
      existingBook.title = call.request.title;
      existingBook.author = call.request.author;
      existingBook.content = call.request.content;
      callback(null, existingBook);
    } else {
        callback({
            code: grpc.status.NOT_FOUND,
              details: "Not found"
        });
    }
}

function deleteBook (call, callback) {
  const existingBookIndex = books.findIndex((n) => n.id == call.request.id)
  if (existingBookIndex != -1) {
     books.splice(existingBookIndex, 1)
     callback(null, {})
    } else {
        callback({
            code: grpc.status.NOT_FOUND,
            details: "Book not found"
        })
    }
}

function allBooks(call, callback) {
    callback(null, { books })
}

server.bindAsync("127.0.0.1:50000", grpc.ServerCredentials.createInsecure(), (error, port) => {
  server.start();
  console.log(`listening on port ${port}`);
  });

This code shows how to create a gRPC server using Node.js. It loads the books.proto file and creates a new gRPC server. An array of books with initial data is defined. The book service is added to the gRPC server with five methods: allBooks, createBook, readBook, updateBook, and deleteBook.

Each of these methods takes call and callback arguments. The createBook method adds a new book and returns it. The readBook method finds a book by its id and returns the book object with the input id.

The updateBook method updates an existing book and returns the book object. The deleteBook method removes a book. The allBooks method returns an array of all the books.

Finally, the server starts by using server.bindAsync and the server’s address and credentials, followed by server.start().

Creating the client

After creating the gRPC server, create a client.js file and add the following code. This code block demonstrates how to create a gRPC client using Node.js and how it can be used to make requests to a gRPC server.

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader")
const packageDef = protoLoader.loadSync("books.proto", {});
const grpcObject = grpc.loadPackageDefinition(packageDef);
const bookPackage = grpcObject.bookPackage;

const text = process.argv[2];

const client = new bookPackage.Book("localhost:50000", grpc.credentials.createInsecure())

client.createBook({
  "title": "title 3",
  "author": "Herod 3",
  "content": "Content 3"
}, (err, response) => {

  console.log("Book has been created " + JSON.stringify(response))

})

client.readBook({
  "id": 1
}, (err, response) => {

  console.log("Book has been read " + JSON.stringify(response))

})

client.updateBook({
  "id": 2,
  "title": "title 3",
  "author": "Herod 3",
  "content": "Content 3"
}, (err, response) => {

  console.log("Book has been updated " + JSON.stringify(response))

})

client.deleteBook({
  "id": 2,
}, (err, response) => {

  console.log("Book has been deleted " + JSON.stringify(response))

})

client.allBooks(null, (err, response) => {

  console.log("Read all books from database " + JSON.stringify(response))

})

We have defined the message structure for the book service using the books.proto file. The client will be able to create, read, update, and delete books, as well as retrieve a list of all the books in the database.

Next, we created a new gRPC client using new bookPackage.Book("localhost:50000", grpc.credentials.createInsecure()). This method takes two arguments: the server address and the client credentials.

It then calls the createBook, readBook, updateBook, deleteBook, and allBooks methods on the client object, passing in the appropriate request parameters and callbacks.

Finally, the client is started by calling the client.start() method.

gRPC vs REST vs GraphQL

gRPC, REST, and GraphQL are three approaches to building APIs, each with its advantages and disadvantages.

gRPC uses Protocol Buffers for message passing, and it is known for its high performance, low latency, and support for bidirectional streaming. It is well-suited for microservices and high-performance systems but requires compatible programming languages and can be challenging to set up.

REST is a set of architectural principles for building web services. RESTful APIs are stateless, use HTTP methods for resource operations, and return responses in JSON or XML format. It is widely used and easy to integrate but can be less efficient than gRPC due to HTTP overhead and lack of bidirectional streaming support.

GraphQL is a query language for APIs that allows clients to specify the data they need and receive only that specific data in response. It provides a flexible, type-safe API that allows clients to avoid over-fetching or under-fetching data. It is suitable for complex data models and known for its ease of use and client flexibility, but can be more difficult to implement due to its unique syntax and server-side complexity.

In summary, gRPC is best for high-performance, microservices architectures, while REST is widely used and easy to integrate, and GraphQL is suitable for complex data models and client flexibility. The best choice of which one to use depends on the requirements of the project and the trade-offs between efficiency, flexibility, and ease of use.

Pros and cons of grpc

gRPC has several advantages and disadvantages that should be considered when choosing it as a communication protocol for building APIs.

Pros of gRPC

• High Performance: gRPC is known for its high performance and low latency due to its use of binary serialization and HTTP/2. This makes it an ideal choice for building microservice architectures and systems that require high throughput.
• Scalability: gRPC supports bidirectional streaming, which enables clients and servers to send multiple messages over a single connection, making it more scalable than REST in certain scenarios.
• Strong Typing: gRPC uses Protocol Buffers for data serialization, which provides strong typing and schema enforcement, reducing the likelihood of errors and improving API stability.
• Multi-language support: gRPC provides support for multiple programming languages, allowing clients and servers to communicate with each other regardless of their language of choice.

Cons of gRPC

• Learning Curve: gRPC can be more challenging to learn and set up compared to other communication protocols, such as REST. Especially for beginners who aren’t knowledgeable about the framework.
• Compatibility: The client and server must use a compatible programming language, which can limit gRPC usability in some scenarios.
• Limited Browser Support: gRPC is not currently supported by web browsers, which limits its usability in web-based applications.

Conclusion

gRPC is a powerful RPC framework that uses Protocol Buffers to serialize data before sending it over a network, making it ideal for building microservices and distributed systems with high throughput. Although it is more challenging to learn and set up than REST, gRPC provides significant advantages in terms of efficiency for certain use cases. However, it requires compatible programming languages on both the client and server. gRPC is not a complete replacement for REST, but it can be used along with REST with some tweaks. Ultimately, the choice of communication protocol depends on the project requirements and trade-offs between efficiency, compatibility, and ease of use.