The Power of Data Serialization: A Deep Dive into Typical and Protobuf
In the world of software development, microservices have become the norm, offering flexibility and scalability. However, as these services communicate with each other, clients, and datastores, ensuring correct payloads is a significant challenge. This is where data serialization comes into play, converting data into a transmittable format. In this article, we’ll explore two prominent serialization technologies, Typical and Protobuf, and their applications in TypeScript.
What is Data Serialization?
Data serialization is the process of converting data into a format, such as bytes, that can be easily transmitted or stored. A schema defines the structure and criteria of the data, enabling developers to share knowledge and build utilities for a common format. This results in shipping code that is less prone to validation errors.
Typical and Protobuf: A Comparison
Both Typical and Protobuf are trusted technologies for using schemas. Typical, a relatively new player, offers modern features like asymmetric fields for backward/forward compatibility and choices for pattern-matching. Protobuf, with 15 years of experience, is maintained by Google and supports plugins, offering a more granular approach to schema evolution.
Getting Started with an Employee Data Example
Let’s consider a hypothetical service that sends employee data to other services within an organization. We’ll use TypeScript and Node.js to demonstrate the differences between Typical and Protobuf. We’ll set up our development machine, define a schema, generate serializers and deserializers, and explore schema changes and nullable fields.
Defining the Schema
Our sample payload structure includes basic employee information, such as name and hourly rate. We’ll define an Employee struct with an enum for department choices. The schema will look similar in both Typical and Protobuf, with minor differences.
Generating Serializers and Deserializers
Using the CLI tools, we’ll generate TypeScript types and code for both Typical and Protobuf. We’ll write code to serialize our sample payload into a binary format.
Making a Schema Change or Schema Evolution
Let’s say we need to add an employee phone number to the schema. This change will require careful planning to avoid breaking schema changes. We’ll explore how Typical and Protobuf handle schema changes, including backward and forward compatibility.
Using Typical’s Asymmetric Fields
Typical’s asymmetric fields ensure forward and backward compatibility, making schema changes easier to understand. We’ll add the keyword to the phone number field, making it required for writers and optional for readers.
Using Protobuf’s Optional Keyword
In Protobuf, we’ll use the optional keyword to make the phone number field required. This process is more involved, requiring multiple updates to writers and readers.
Working with Flexible Payloads
We’ll explore how to work with nullable fields using Typical’s choices and Protobuf’s oneof keyword. This allows for pattern-matching-like capabilities and flexibility in payload design.
Conclusion
Both Typical and Protobuf have their unique advantages and limitations. Typical offers modern features and a cohesive CLI tool, while Protobuf boasts a long-standing reputation and widespread popularity. The choice between the two depends on your organization’s specific needs and preferences. By understanding the power of data serialization and the capabilities of Typical and Protobuf, you can make informed decisions about your software development projects.