TwirpScript

A protobuf RPC framework for JavaScript and TypeScript

What is this? 🧐

TwirpScript is an implementation of Twirp. TwirpScript autogenerates Javascript or TypeScript clients and servers from protocol buffers. The generated clients can be used in browser or Node.js runtimes, enabling type safe communication between web applications and a server or server-to-server. TwirpScript implements the latest Twirp Wire Protocol (v7).

Table of Contents

• Overview
• Highlights 🛠
• Installation 📦
• Getting Started
• Requirements ⚠️
  • Overview 📖
  • Configuring your Twirp Runtime
    • Client
    • Server
  • Context
    • Client
    • Server
  • Middleware / Interceptors
    • Client
    • Server
  • Hooks
    • Client
    • Server
• Configuration 🛠
• JSON
• Examples 🚀
• Working with other tools
• Caveats, Warnings and Issues ⚠️
• FAQ
• Contributing 👫
• Licensing 📃

Overview

Twirp is a simple RPC framework built on protocol buffers. TwirpScript generates JavaScript or TypeScript clients and servers from .proto service specifications. The generated clients can be used in the browser or in Node.js runtimes. This enables type safe communication between the client and server, as well as reduced payload sizes when using protobuf as the serialization format.

You define your service in a .proto specification file, and TwirpScript will generate client and service handlers for that service. You fill in the business logic that powers the server, and TwirpScript handles the boilerplate.

To learn more about the motivation behind Twirp (and a comparison to REST APIs and gRPC), check out the announcement blog.

Highlights 🛠

1. Isomorphic. TwirpScript's generated serializers/deserializers can be consumed in the browser or Node.js runtimes.

2. Small. TwirpScript's runtime and generated code are built with tree shaking to minimize bundle sizes. This results in a significantly smaller bundle size than google-protobuf. TwirpScript's runtime is 2KB (1.2 gzipped). The serialization runtime, ProtoScript, is 37KB (7.2 gzipped). ProtoScript will be eliminated from bundles when only using the generated JSON clients.

3. In-editor API documentation. Comments in your .proto files become TSDoc comments in the generated code and will show inline documentation in supported editors.

4. Idiomatic JavaScript / TypeScript code. None of the Java idioms that protoc --js_out generates such as the List suffix naming for repeated fields, Map suffix for maps, or the various getter and setter methods. TwirpScript generates and consumes plain JavaScript objects over classes.

Installation 📦

1. Install the protocol buffers compiler:

   MacOS: brew install protobuf

   Linux: apt install -y protobuf-compiler

   Windows: choco install protoc

   Or install from a precompiled binary.

2. Add this package to your project: npm install twirpscript or yarn add twirpscript

Requirements ⚠️

• Node.js v16 or greater
• TypeScript v4.7 or greater when using TypeScript

Getting Started

Overview 📖

1. Define your service in a .proto file.
2. Run npx twirpscript to generate JavaScript or TypeScript code from your .proto file. This will generate JSON and Protobuf clients, a service interface, and service utilities.
3. If you only need a client, you're done! Use the generated client to make requests to your server.
4. Implement the generated service interface.
5. Add your implemented service to your application server's routes.

1. Define your service

Create a proto specification file:

src/protos/haberdasher.proto

syntax = "proto3";

// Haberdasher service makes hats for clients.
service Haberdasher {
  // MakeHat produces a hat of mysterious, randomly-selected color!
  rpc MakeHat(Size) returns (Hat);
}

// Size of a Hat, in inches.
message Size {
  int32 inches = 1; // must be > 0
}

// A Hat is a piece of headwear made by a Haberdasher.
message Hat {
  int32 inches = 1;
  string color = 2; // anything but "invisible"
  string name = 3; // i.e. "bowler"
}

2. Run npx twirpscript

This will generate haberdasher.pb.ts (or haberdasher.pb.js for JavaScript users) in the same directory as as haberdasher.proto. Any comments will become TSDoc comments and will show inline in supported editors.

npx twirpscript will compile all.proto files in your project.

3. Use the client

Use the generated clients to make json or protobuf requests to your server:

src/client.ts

import { client } from "twirpscript";
import { MakeHat } from "../protos/haberdasher.pb";

client.baseURL = "http://localhost:8080";

const hat = await MakeHat({ inches: 12 });
console.log(hat);

The above client code may be used in browser or node.js runtimes. See a Node.js client example.

If you have an existing Twirp server you're connecting to and only need a client, that's it! You're done. If you're implementing a service as well, keep reading.

4. Implement the generated service interface

src/server/haberdasher/index.ts

import { Haberdasher, createHaberdasher } from "../../protos/haberdasher.pb";

const haberdasher: Haberdasher = {
  MakeHat: (size) => {
    return {
      inches: size.inches,
      color: "red",
      name: "fedora",
    };
  },
};

export const haberdasherHandler = createHaberdasher(haberdasher);

5. Connect your service to your application server

src/server/index.ts

import { createServer } from "http";
import { createTwirpServer } from "twirpscript";
import { haberdasherHandler } from "./haberdasher";

const PORT = 8080;

const app = createTwirpServer([haberdasherHandler]);

createServer(app).listen(PORT, () =>
  console.log(`Server listening on port ${PORT}`),
);

If you're deploying to a serverless environment such as AWS Lambda, replace createTwirpServer above with createTwirpServerless. See the aws lambda example for a full project!

Configuring your Twirp Runtime

Client

Clients can be configured globally, at the RPC callsite, or with middleware. The order of precedence is middleware > call site configuration > global configuration. Middleware overrides call site configuration, and call site configuration overrides global configuration.

Configuration Options

Example

src/client.ts

import { MakeHat } from "./protos/haberdasher.pb";

const hat = await MakeHat({ inches: 12 }, { baseURL: "http://localhost:8080" });
console.log(hat);

baseURL can be globally configured, instead of providing it for every RPC call site:

import { client } from "twirpscript";

// http://localhost:8080 is now the default `baseURL` for _all_ TwirpScript RPCs
client.baseURL = "http://localhost:8080";

const hat = await MakeHat({ inches: 12 }); // We can omit `baseURL` because it has already been set
console.log(hat);

You can override a globally configured baseURL at the RPC call site:

import { client } from "twirpscript";
client.baseURL = "http://localhost:8080";

// This RPC will make a request to https://api.example.com instead of http://localhost:8080
const hat = await MakeHat(
  { inches: 12 },
  { baseURL: "https://api.example.com" },
);
console.log(hat);

// This RPC will make a request to http://localhost:8080
const otherHat = await MakeHat({ inches: 12 });
console.log(otherHat);

In addition to baseUrl, headers can also be set at via global configuration or call site configuration. headers defines key value pairs that become HTTP headers for the RPC:

import { client } from "twirpscript";

client.baseURL = "http://localhost:8080";

// setting a (non standard) HTTP "device-id" header via global configuration. This header will be sent for every RPC.
client.headers = { "device-id": getOrGenerateDeviceId() };

// setting a (non standard) HTTP "idempotency-key" header for this RPC call. This header will only be sent for this RPC.
const hat = await MakeHat(
  { inches: 12 },
  { headers: { "idempotency-key": "foo" } },
);
console.log(hat);

rpcTransport

Twirp abstracts many network details from clients. Sometimes you will want more control over the underlying network request. For these cases, TwirpScript exposes rpcTransport.

rpcTransport can be used to customize the network request made. rpcTransport can swap out the default implementation to use an https agent, or a library like axios. The transport only needs to implement the RpcTransport interface. It can also be used to "bake" in certain options, for example, setting fetch's credentials option to include (which will include cookies in cross origin requests).

rpcTransport can be set via global configuration or call site configuration:

import { client } from "twirpscript";

// sets a custom rpcTransport for all RPC calls, globally.
client.rpcTransport = (url, opts) =>
  fetch(url, { ...opts, credentials: "include" });

// sets a custom rpcTransport for this RPC call. This transport will only be used for this RPC.
const hat = await MakeHat(
  { inches: 12 },
  {
    rpcTransport: (url, opts) =>
      fetch(url, { ...opts, credentials: "include" }),
  },
);

In Node.js environments, TwirpScript automatically uses Node's http or https client instead of fetch. You can override this behavior and use fetch in Node.js environments by using the global example above.

Server

Servers can be configured by passing a configuration object to createTwirpServer.

Configuration Options

Example

import { createServer } from "http";
import { createTwirpServer } from "twirpscript";
import { haberdasherHandler } from "./haberdasher";

const PORT = 8080;

// This removes the "/twirp" prefix in the RPC path
const app = createTwirpServer([haberdasherHandler], { prefix: "" });

createServer(app).listen(PORT, () =>
  console.log(`Server listening on port ${PORT}`),
);

Context

Client

Server

Your service handlers are invoked with context as their second argument. The base fields are documented above, but you may extend this object with arbitrary fields. This means you can use context to provide information to your handler that doesn't come from the RPC request itself, such as http headers or server-side API invocations.

Custom fields can be added to the context object via middleware.

Example

If you setup middleware similiar to the authentication middleware example, you could read the currentUser username property in your service handler. See the authentication example for a full application.

import { Haberdasher, createHaberdasher } from "../../protos/haberdasher.pb";
import { Context } from "../some-path-to-your-definition";

const haberdasher: Haberdasher<Context> = {
  MakeHat: (size, ctx) => {
    return {
      inches: size.inches,
      color: "red",
      name: `${ctx.currentUser.username}'s fedora`,
    };
  },
};

export const haberdasherHandler = createHaberdasher(haberdasher);

Middleware / Interceptors

TwirpScript's client and server request response lifecycle can be programmed via middleware.

Middleware is called in order of registration, with the Twirp RPC invoked last.

Because each middleware is responsible for invoking the next handler, middleware can do things like short circuit and return a response before the RPC is made, or inspect the returned response, enabling powerful patterns such as caching.

Client

Clients can be configured via the client export's use method. use registers middleware to manipulate the client request / response lifecycle. The middleware handler will receive context and next parameters. You can set the headers and url for the RPC via context. next invokes the next handler in the chain -- either the next registered middleware, or the Twirp RPC.

Example

import { client } from "twirpscript";

client.use((context, next) => {
  const auth = localStorage.getItem("auth");
  if (auth) {
    context.headers["authorization"] = `bearer ${auth}`;
  }
  return next(context);
});

Client middleware can override both global configuration and call site configuration.

import { client } from "twirpscript";

client.baseURL = "http://localhost:8080";

client.use((context, next) => {
  const url = new URL(context.url);
  url.host = "www.foo.com";

  return next({ ...context, url: url.toString() });
});

// This will make a request to https://www.foo.com instead of http://localhost:8080 or https://api.example.com"
const hat = await MakeHat(
  { inches: 12 },
  { baseURL: "https://api.example.com" },
);
console.log(hat);

Server

Servers can be configured via your server's use method. use registers middleware to manipulate the server request / response lifecycle.

The middleware handler will receive req, context and next parameters. req is the incoming request. context is the context which will be passed to each middleware handler and finally the Twirp service handler you implemented. You may extend context to pass extra parameters to your service handlers that are not available via your service's defined request parameters. This can be used to implement things such as authentication or rate limiting. next invokes the next handler in the chain -- either the next registered middleware, or if there is no middleware remaining, the Twirp service handler you implemented.

Example

import { createServer } from "http";
import { createTwirpServer, TwirpError } from "twirpscript";
import { authenticationHandler } from "./authentication";

export interface Context {
  currentUser: { username: string };
}

const services = [authenticationHandler]
const app = createTwirpServer<Context, typeof services>(services);

app.use(async (req, ctx, next) => {
  // exception so unauthenticated users can authenticate
  if (ctx.service.name === authenticationHandler.name) {
    return next();
  }

  // extract token from authorization header
  const token = req.headers["authorization"]?.split("bearer")?.[1]?.trim();

  // a fictional helper function that retrieves a user from a token
  const currentUser = getCurrentUser(token);

  if (!currentUser) {
    return TwirpErrorResponse({
      code: "unauthenticated",
      msg: "Access denied",
    });
  } else {
    ctx.currentUser = currentUser;
    return next();
  }
};

createServer(app).listen(PORT, () =>
  console.log(`Server listening on port ${PORT}`)
);

See the authentication example for a full application.

Errors

Twirp defines a list of error codes. You can throw an error from your service handler by using TwirpError:

import { TwirpError } from "twirpscript";

if (!currentUser) {
  throw new TwirpError({
    code: "unauthenticated",
    msg: "Access denied",
  });
}

Note: You must use TwirpError. Any unhandled errors will otherwise be caught and the TwirpScript server will respond with the following JSON response: { code: "internal", msg: "server error" } and set the appropriate headers and status code.

If you want to respond with a Twirp Error from middleware, use TwirpErrorResponse. This will create a Twirp Error response while still running any remaining middleware in the chain. You may explictly define try / catch clauses in your middleware, but any unhandled errors will otherwise be caught and the TwirpScript server will respond with the error described above.

Hooks

TwirpScript clients and servers can be instrumented by listening to events at key moments during the request / response life cycle. These hooks are ideal placements for instrumentation such as metrics reporting and logging. Event handlers for both clients and servers are passed a context object as the first argument. As a best practice, this context object should be treated as readonly / immutable.

While hooks and middleware can be used to accomplish similar goals, as a best practice use hooks for instrumentation and middleware for mutation.

Client

Every client event handler is invoked with the request context.

Events

requestPrepared is called as soon as a request has been created and before it has been sent to the Twirp server.

responseReceived is called after a request has finished sending.

error is called when an error occurs during the sending or receiving of a request. In addition to the context, the error that occurred is passed as the second argument.

Example

import { client } from "twirpscript";

client.on("responseReceived", (context) => {
  // log or report
});

Server

Every server event handler is invoked with the request context.

Events

requestReceived is called as soon as a request enters the Twirp server at the earliest available moment. Called with the current context and the request.

requestRouted is called when a request has been routed to a service method. Called with the current context and the input to the service method.

responsePrepared is called when a request has been handled by a service method. Called with the current context and the response generated by the service method.

responseSent is called when all bytes of a response (including an error response) have been written. Called with the current context and the response.

error is called when an error occurs while handling a request. Called with the current context and the error that occurred.

Example

import { createServer } from "http";
import { createTwirpServer } from "twirpscript";
import { habderdasherHandler } from "./haberdasher";

const PORT = 8080;

const app = createTwirpServer([habderdasherHandler]);

app.on("responseSent", (ctx) => {
  // log or report
});

createServer(app).listen(PORT, () =>
  console.log(`Server listening on port ${PORT}`),
);

Configuration 🛠

TwirpScript aims to be zero config, but can be configured via the cli interface, or when using the npx twirpscript command, by creating a proto.config.mjs (or .js or .cjs) file in your project root.

  /src
    A.proto
    B.proto

  import "src/B.proto";

/** @type {import('twirpscript').Config} */
export default {
  root: "src",
};

import "B.proto";

   /** @type {import('twirpscript').Config} */
   export default {
     exclude: ["/bar/"]
   }

  /src
    A.proto
    B.proto

  /src
    A.proto
    A.pb.ts
    B.proto
    B.pb.ts

  /** @type {import('twirpscript').Config} */
  export default {
    dest: "out",
  }

  /src
    A.proto
    B.proto
  /out
    /src
      A.pb.ts
      B.pb.ts

  /** @type {import('twirpscript').Config} */
  export default {
    root: "src",
    dest: "out",
  }

  /src
    A.proto
    B.proto
  /out
    A.pb.ts
    B.pb.ts

JSON

TwirpScript's JSON serialization/deserialization is migrating to the proto3 specification. This is nearly complete, but still in progress.

TwirpScript will serialize JSON keys as lowerCamelCase versions of the proto field. Per the proto3 spec, the runtime will accept both lowerCamelCase and the original proto field name when deserializing. You can provide the json_name field option to specify an alternate key name. When doing so, the runtime will encode JSON messages using the the json_name as the key, and will decode JSON messages using the json_name if present, otherwise falling back to the lowerCamelCase name and finally to the original proto field name.

Example

syntax = "proto3";

message Hat {
  // this key will serialize as 'userID' instead of 'userId'
  int64 user_id = 1; [json_name="userID"];
  int64 wardrobe_id = 2;
}

The above Hat message would serialize to the following JSON:

{ "userID": "some 64bit number", "wardrobeId": "some 64bit number" }

Examples 🚀

The documentation is a work in progress. Checkout the examples in the examples directory:

• The JavaScript fullstack shows a minimal browser client and server implementation in JavaScript.
• The TypeScript fullstack shows a minimal browser client and server implementation in TypeScript.
• The authentication example extends the fullstack example to demonstrate authentication using tokens.
• The aws lambda example shows TwirpScript running on AWS Lambda, complete with the necessary CDK to deploy a full stack (API Gateway + Lambda).
• The Next.js example shows using TwirpScript in Next.js.

The examples also demonstrate testing using jest.

Working with other tools

TypeScript

As a design goal, TwirpScript should always work with the strictest TypeScript compiler settings. If your generated TwirpScript files are failing type checking, please open an issue.

TwirpScript could opt generated files out from type checking, but it leverages the TypeScript compiler to flag version mismatches between the TwirpScript runtime and generated code.

Linters

TwirpScript does not make any guarantees for tools like linters and formatters such as prettier or eslint. It generally does not make sense to run these tools against code generation artifacts, like the .pb.ts or .pb.js files generated by TwirpScript. This is because:

1. The permutations of preferences and plugins for these tools quickly explode beyond what is feasible for a single tool to target. There are always new tools that could be added as well.
2. The code is generated automatically, and not all rules are autofixable. This means there are cases that would always require manual intervention by the user.
3. Autogenerated code is readonly, and expected to be correct. Autogenerated code has a much difference maintenance cycle than code written by hand, and should generally be treated as a binary or a dependency. You don't lint your node_modules!

Here are some example snipits to opt TwirpScript generated code out of these tools:

.eslintrc.js

module.exports = {
  ignorePatterns: ["*.pb.[t|j]s"],
};

Buf

TwirpScript can be used with Buf. This will bypass TwirpScript's cli runner, so all options must be passed to buf via it's configuration yaml. proto.config.mjs is bypassed, as is automatic typescript inference.

buf.gen.yaml

version: v1
plugins:
  - name: protoc-gen-twirpscript
    path: ./node_modules/twirpscript/dist/compiler.js
    out: .
    opt:
      - language=typescript
    strategy: all

See the buf example for a full example.

Caveats, Warnings and Issues ⚠️

Protobuf2

This library supports Protobuf3. Protobuf2 should generally work, with the notable exception of Groups which are deprecated and will not be implemented.

At this time, there are no plans to support Proto2 directly. I'd like any community that grows around TwirpScript to not be concerned with alternate patterns to support proto2. TwirpScript's release came quite some time after the release of proto3,and I'm not aware of any compelling reasons to use proto2 over proto3 for new applications. The protobuf compiler supports references between proto2 and proto3 files, so existing applications can port messages incrementally to proto3 that want to leverage TwirpScript.

If there is sufficient demand for proto2 support, I will reconsider this stance. Please open an issue and add the Protobuf 2 label if you have a proto2 support request.

FAQ

Why use Twirp instead of GraphQL, gRPC or REST?

For multiple clients with distinct views, I would pull in GraphQL. For a single UI client I prefer the simpler architecture (and well defined optimization paths) found with a more traditional API server approach.

A REST or JSON API lacks type safety and the developer experience that comes from static typing. This can be mitigated to an extent with tools like JSON Schema, but that route requires stitching together (and maintaining) a suite of tools to achieve a similar developer experience.

gRPC is great, but has a large runtime (and corresponding complexity) that is unnecessary for some applications. Twirp offers many of the benefits of gRPC with a significantly reduced runtime. TwirpScript's developer experience is designed to be idiomatic for the JS/TS community, and TwirpScript's autogenerated clients are optimized for use in the browser.

To learn more about the motivation behind Twirp (and a comparison to REST APIs and gRPC), check out the announcement blog.

Contributing 👫

PR's and issues welcomed! For more guidance check out CONTRIBUTING.md

Licensing 📃

See the project's MIT License.

Changelog

v0.0.72

• Remove hardcoded compiler path on Windows to support monorepos. Thanks @lordvlad!

v0.0.71

• Fix Timestamp and Duration JSON serialization. Previously, when either seconds or nanos were 0, the Timestamp / Duration was omitted from the serialized json. Thanks @martynchamberlin!

v0.0.70

• Fix compiler path on Windows. Thanks @l1b3r!

v0.0.69

• Updates to ProtoScript 0.0.20. This updates JSON serializtion of Timestamp and Duration well known types to conform to the Protobuf3 specification.

v0.0.68

• All header names are now lowercased. This was already true for users using createTwirpServer in conjunction with Node.js' createServer, but previously users of createTwirpServerless did not receive this standardization.

v0.0.67

• Updates to v0.0.18 of ProtoScript, which includes fixes for self-referncing (recursive) messages.

• Buf users will need to update their buf.gen.yaml path: buf.gen.yaml

  version: v1
  plugins:
    - name: protoc-gen-protoscript
  -    path: ./node_modules/protoscript/compiler.js
  +    path: ./node_modules/protoscript/dist/compiler.js
      out: .
      opt:
        - language=typescript
      strategy: all

v0.0.66

• Fix intermittent EAGAIN issue encountered when compiling protos

• Use glob imports for generated messages instead of destructuring. This preserves tree shaking, but preserves module namespacing to disambiguate name collisions between protos. Previously, identically named messages in different modules could causes a name collision, eg:

  // foo.proto
  message Foo {}

  // bar.proto
  import "foo.proto";
  message Foo {}

  Would result in errors in the generated code. Now, this is namespaced and works correctly.

v0.0.65

Protocol Buffers Well-Known Types are now exported from protoscript. References to well-known types are now imported from protoscript rather than being generated. This is a non breaking change. If you have well-known types in your project, you can remove the google/protobuf directory that was generated in previous versions alongside your other .pb.js/ts files.

The output location of google/protobuf was a common reason for using dest in proto.config.mjs so this change should facilitate zero configuration for a greater number of projects.

v0.0.64

• Revert Include file extensions in generated file imports introduced in v0.0.61 for TypeScript users. Generated TypeScript imports will revert to the following:

- import { Foo } from './foo.pb.js';
+ import { Foo } from './foo.pb';

When targeting ESM, the TypeScript compiler expects .js extensions and not .ts extensions for imports because the compiler does not manipulate import paths: https://www.typescriptlang.org/docs/handbook/esm-node.html.

Including a full extension results in the following TypeScript error:

[tsserver 2691] [E] An import path cannot end with a '.ts' extension.

The TypeScript team's recommendation to use .js extensions for .ts file imports when targeting ESM causes a number of issues with the broader JavaScript ecosystem. Until this situation is rectified, TwirpScript will not emit ESM compliant extensions for TypeScript. This only impacts TypeScript users who wish to target ESM in Node.JS using the TypeScript compiler, as bundlers are not pedantic about file extensions. If you're impacted by this, please join the discussion in #202

v0.0.63

• Change configuration file format. Now, the configuration file is JS instead of JSON. This provides better discoverability and type checking for TypeScript users.

The following .twirp.json:

{
  "root": "src"
}

Would be renamed to proto.config.mjs and changed to the following:

/** @type {import('twirpscript').Config} */
export default {
  root: "src",
};

• Use relative file path for determining path to compiler instead of hard coding from project root. This should interop better with more exotic package tooling and repo setup.

• Fix: Improved map detection. Previously field types suffixed with Entry were incorrectly flagged as maps. This has been fixed.

v0.0.62

• Allow rpcTransport overrides. See #189 for more context.
• Remove process.stdin.fd usage to see if it resolves intermittent Error: EAGAIN: resource temporarily unavailable, read. See #191 for more context.

v0.0.61

• Fix JSON deserialization. #181 introduced a regression that caused TwirpScript servers' JSON serialization to fail.
• Distribute strict ESM. A CommonJS is runtime is included for legacy node clients. Code generation uses ESM and requires Node.js v14 or later. If you're using a compiler other than TypeScript such as webpack, please see these instructions
• Use ProtoScript code generation. This will result in some generated imports coming from protoscript instead of twirpscript, but this is a non breaking change. These imports were previously delegated to ProtoScript via reexports inside TwirpScript, and that indirection has now been removed.

v0.0.60

• Removes @types/node dependency. @types/node is no longer necessary when using a TwirpScript generated client.

v0.0.59

• Fixes generated JSON client when using nested messages. The generated JSON serialization names were invalid for nested messages. See #176 for more context.

v0.0.58

• Better insight into internal server Errors. TwirpScript catches any errors thrown internally. Errors that aren't an instanceof TwirpError present to users as a generic Twirp internal_error for security so that internal details don't leak. This can hurt the DX for identifying issues during development, and also can hide important debugging information when relying on error reporting using hooks.

TwirpScript now includes the thrown error as an error property in the TwirpError that is passed to the error hook, but continues not to the response.

Example:

If we have code that does the following in a service or middleware:

throw new Error("uh oh.");

The error hook will be invoked with:

TwirpError {
  code: 'internal',
  msg: 'server error',
  meta: {
    // this is kept private to the error hook and not exposed to end users
    error: Error("uh oh")
  }
}

And the response will be:

TwirpError {
  code: 'internal',
  msg: 'server error',
  // note the error is kept private: it's not exposed to end users
  meta: undefined,
}

As before, any errors that should be surfaced to end users should be created by throwing TwirpError:

throw new TwirpError({ code: "code", msg: "msg" });

v0.0.57

• Generated .pb files now opt out of eslint via eslint-disable comments
• TwirpScript now uses ProtoScript as the serialization runtime instead of google-protobuf. ProtoScript's runtime is 37KB (7.2KB gzipped) compared to google-protobuf's 231KB (46KB gzipped).

v0.0.56

Users will need to npx twirpscript to regenerate their .pb.ts / .pb.js files when adopting this version.

• The generated message serializer/deserializer objects have been split into two separate objects: one for JSON and one for Protobuf. This enables smaller client bundles when using a bundler that supports tree shaking / dead code elimination. Many users will be unaffected by this change, but this is a breaking change for users that use message encode/decode methods directly in their source code.

Previously this proto:

// A Hat is a piece of headwear made by a Haberdasher.
message Hat {
  int32 inches = 1;
  // anything but "invisible"
  string color = 2;
  // i.e. "bowler"
  string name = 3;
}

would generate an object like this in the generated pb.ts or pb.js file:

export const Hat = {
  encode: ...
  decode: ...
  encodeJSON: ...
  decodeJSON: ...
}

now two objects are generated, one for Protobuf and one for JSON (with a JSON suffix appended to the message name):

export const Hat = {
  encode: ...
  decode: ...
}

export const HatJSON = {
  encode: ...
  decode: ...
}

• TwirpScript client code is now isomorphic: Node.js clients no longer require extra configuration (using the client rpcTransport attribute). TwirpScript now uses Node's conditional exports internally.

  import { client } from "twirpscript";
  -import { nodeHttpTransport } from "twirpscript/node";
  import { MakeHat } from "../protos/haberdasher.pb";
  
  client.baseURL = "http://localhost:8080";
  
  -// This is provided as a convenience for Node.js clients. If you provide `fetch` globally, this isn't necessary and your client can look identical to the browser client above.
  -client.rpcTransport = nodeHttpTransport;
  
  const hat = await MakeHat({ inches: 12 });
  console.log(hat);

v0.0.55

• Protobuf messages now always pack packable repeated fields when serializing and can read packed or unpacked when deserializing. This will slightly decrease the size of some protobuf messages over the wire, and enable better interop with messages encoded by other protobuf serializers.

v0.0.54

• Fixes a regression introduced in v0.0.53 that caused JSON.parse to be invoked twice for JSON clients.
• Optional message deserialization had a bug impacting optional fields that are a message type. The default message value was always being supplied to the client, preventing clients from determining whether the field was set. This has been fixed. Now if a server omits or supplies an optional message field with a null value the client will read the field as undefined.

v0.0.53

• client JSON request methods now use encodeJSON and decodeJSON helpers.

v0.0.52

• Fixes a regression where nested types were not consumable:

  [tsserver 2702] [E] 'Foo' only refers to a type, but is being used as a namespace here.

• Removes dist from public import paths. This impacts users of twirpscript/dist/node and direct invocations of the compiler, like buf users. The following changes are necessary to migrate:

  -import { nodeHttpTransport } from "twirpscript/dist/node";
  +import { nodeHttpTransport } from "twirpscript/node";

  version: v1
  plugins:
    - name: protoc-gen-twirpscript
      -path: ./node_modules/twirpscript/dist/compiler.js
      +path: ./node_modules/twirpscript/compiler.js
      out: .
      opt:
        - language=typescript
      strategy: all

v0.0.51

• When using protobuf map fields, map keys are now typed as strings: Record<string, $SomeType>. Previously other types were accepted, which would cause type checking to fail when the key was boolean, bigint, or number. This is also more correct because JavaScript always encodes object keys as strings. Generated type definitions for map types are no longer exported. See #151 for more background.
• Empty messages now generate the full serialization interface implemented by other messages. This resolves an issue where messages with fields whose value was an empty message would fail code generation.
• Enum serializers now have two private serialization helpers. This resolves an issue where Enums imported into other protobuf files failed code generation. See #150 for more background.

v0.0.50

• Add typescript.emitDeclarationOnly only option. This will only emit TypeScript type definitions and not any runtime.

v0.0.49

• Enums are now represented by the enum value specified in proto instead of an integer. Eg: "FOO_BAR" instead of 0. This improves the developer experience when printing messages. This is a breaking change, but users using protobuf serialization should be unimpacted unless their code directly references enums via their integer value instead of the generated constants.

Eg:

// this code will now need to migrate to the string value instead of an integer
`if (foo.someEnum === 0)`;

// this code will continue to work without change
`if (foo.someEnum === SomeEnum.Field)`.

JSON serialization now also uses the enum value instead of an integer value, as described by the protobuf JSON specification. This is a breaking change for JSON clients.

• Bytes are now Base64 encoded when JSON serializing as described by the protobuf JSON specification above. This is a breaking change for JSON clients.

v0.0.48

This version has the following bug fixes:

• Fix nested message definitions. Previously this would cause "ReferenceError: Cannot access before initialization" error.
• Fix repeated int64 generation. The generated code would not compile for repeated bigint cases
• Fix bigint json serialization for maps
• Fix comment escaping
• Fix reserved names for internal variables

Code generation for map types are no longer inlined. This is an internal refactor that should not impact consumption.

v0.0.47

• Add json serialization options. See the README's configuration section for more context.

v0.0.46

This version has 3 breaking changes:

1. (Only impacts TypeScript users) The 'Service' naming suffix has been removed from the generated TypeScript types for services. Given the following proto:

service Haberdasher {
  rpc MakeHat(Size) returns (Hat);
}

The generated service type will now be Haberdasher instead of HaberdasherService. This enables better out of the box compatibility with buf which expects all service names to end with Service. Following this recommendation would generate TwirpScript types with 'ServiceService' suffixes.

<Service>Service => <Service>

1. The 'Handler' suffix has been removed from the generated create<Service>Handler helper.

Given the proto above, the generated helper is now createHaberdasher instead of createHaberdasherHandler.

create<Service>Handler=> create<Service>

1. (Only impacts TypeScript users) optional types now accept null and undefined. This enables better compatibility with other tools that may type optionals as some type | null

Changes:

• remove naming suffixes by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/125
• add exclude option to .twirp.json by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/127
• add null to optional ts types by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/128

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.45...v0.0.46

v0.0.45

• no longer generate _readMessageJSON for empty messages
• fix map types in _readMessageJSON

v0.0.44

This version has breaking changes between the generated code and the runtime. Run npx twrispcript to update your generated .pb.ts when updating to this version.

TwirpScript now ships with JSON serializers and supports the json_name option described here. This enables clients to specify custom JSON field names.

Breaking change: int64 types were previously encoded as strings, but are now encoded as [bigint](https://caniuse.com/bigint https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt)

Changes:

• add json_name support by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/116
• use bigint for int64 types by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/118

v0.0.43

• Fix repeated message deserialization by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/112.

v0.0.42

• More compact code generation for empty messages by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/101.

v0.0.41

• Breaking Change: field names are now camelCased by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/97. npx twirpscript will regenerate your .pb.ts files with the correct casing. If you run into significant issues with this change or prefer snake_case, please open an issue.

v0.0.40

• export MIN_SUPPORTED_VERSION_X by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/95

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.39...v0.0.40

v0.0.39

• Add TSDoc comments to generated serializer/deserializers by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/90
• Bug Fix: invalid filepaths generated when using root by @arranf in https://github.com/tatethurston/TwirpScript/pull/89

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.38...v0.0.39

v0.0.38

• Bug Fix: remove types from js map output by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/83
• The generated message protobuf encoders now accept partials by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/84
• TypeScript types for map are now inlined into the generated interface by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/85

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.37...v0.0.38

v0.0.37

• Bug Fix: map generation for JavaScript clients by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/78
• Add comment literal escaping by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/79
• Remove version comment from generated code by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/80

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.36...v0.0.37

v0.0.36

• bytesource refactor by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/70
• add version check by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/71
• add map type by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/75

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.35...v0.0.36

v0.0.35

• add body to request type for middleware and hooks by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/67

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.34...v0.0.35

v0.0.34

This release includes breaking changes:

1. The generated .pb.ts files have been restructured. You'll need to run npx twirpscript to regenerate your .pb.ts files after updating.
2. context's service and method properties now point to generated objects instead of simply being strings (eg previously these were "Haberdasher" and "MakeHat" and now these properties point to generated objects. This enables more powerful runtime reflection, eg:

app.on('requestReceived', (ctx, req) => {
   console.log(`Received request body: ${ctx.contentType === 'Protobuf' ? ctx.method.input.decode(req.body) : req.body}`);
});

1. requestRouted and responsePrepared are now invoked with the JavaScript object input / output to your handler, rather than the serialized input / output. This should improve debugging workflows, because you may now simply console log the input / output instead of needing to deserialize protobuf human readable output.

2. Bug Fix: client error hook by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/56

3. improve types for context's method and service properties by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/61
4. Tate/service object by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/62
5. context service and method now point to implementation instead of string by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/63
6. requestRouted and responsePrepared are now invoked with input/output by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/64
7. remove unnecessary isEndGroup check in serializer by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/65
8. format error output from protoc compiler by @tatethurston in https://github.com/tatethurston/TwirpScript/pull/66

Full Changelog: https://github.com/tatethurston/TwirpScript/compare/v0.0.33...v0.0.34