Tag: Efficiency

Introducing nsuv

nsuv is a C++ wrapper around libuv with the main goal of supporting compile-time type safety when propagating data.

You can find the open source package here: https://github.com/nodesource/nsuv

Here at NodeSource we are focused on fixing issues for the enterprise. This includes adding functionality and features to Node.js that are useful for enterprise-level deployments but would be difficult to upstream. One is the ability to execute commands remotely on Worker threads without the addition of running the inspector, such as capturing CPU profiles or heap snapshots. Another feature necessary to make Node.js more reliable in production is the ability to record and send metrics without being at the mercy of a busy event loop.

To achieve these, we run a separate thread that receives commands and gathers metrics from each Node.js thread. The locks and data queues in the separate thread are managed by libuv. As the codebase grew, usability issues began to come up, such as remembering the correct type of each void pointer and keeping track of the lifetime of the many shared locks and resources. Our solution was to write a wrapper for libuv to alleviate these problems.

We had a lot of existing libuv code and didn’t want to rewrite everything from scratch. So we wrote a template class library that inherits from each libuv handle or request type and uses the curiously recurring template pattern (CRTP) for inheritance. Doing so made it possible to write a wrapper that serves as a drop-in replacement, allowing for incremental improvements while supplementing the wrapper’s API with what was needed.

N|Solid has a zero-failure tolerance, so none of our code can accidentally terminate your process. One way we do this is to try our best not to perform additional allocations. If an allocation is necessary, it always does with a strong exception guarantee, which is then caught and returned as a libuv error code.

We have also enabled compile time warnings when returned error codes aren’t handled. While developing nsuv, we analyzed many existing C++ projects that use libuv and discovered that most of them assume the state of the application and lack sufficient error handling in case something unexpected occurs. This can be especially painful when working with asynchronous code, but we understand that not everyone requires the same level of caution. It can be disabled by defining NSUV_DISABLE_WUR in your flags.

Getting Started

The following code example shows the execution of a simple libuv timer, and the only change was to turn the uv_timer_t to a nsuv::ns_timer instance while still being able to use the original libuv APIs:

static void timer_cb(uv_timer_t* handle) {
Foo* foo = static_cast<Foo*>(handle->data);
delete foo;
uv_close(reinterpret_cast<uv_handle_t*>(handle), nullptr);
}

static void call_timer() {
ns_timer timer;
Foo* foo = new Foo();

timer.data = foo;
uv_timer_init(uv_default_loop(), &timer);
uv_timer_start(&timer, timer_cb, 1000, 0);
uv_run(uv_default_loop(), UV_RUN_DEFAULT);
}

As you can see, there’s no need to cast timer before being passed to libuv’s timer function since ns_timer is a derived class of uv_timer_t and upcasting is implicit. It offers the first step in converting code to be more type-safe and improve overall usability. Improvements can be made incrementally from here. Below we take advantage of the CRTP and use it to downcast the uv_timer_t to the nsuv counterpart after using libuv’s timer API:

static void timer_cb(uv_timer_t* handle) {
// Downcast the libuv handle to its nsuv counterpart.
ns_timer* timer = ns_timer::cast(handle);
// Convenience method to retrieve and cast data.
Foo* foo = timer->get_data<Foo>();

delete foo;
timer->close();
}

While this is a good first step, it still requires we know what the data value should be cast to. The call to get_data() only serves as a convenience method for easier casting.

Passing Data

One of the most painful parts of working with libuv was ensuring we didn’t accidentally cast a void pointer to the wrong type from a specific queue. While this could be verified by hand, having the compiler tell us if we did it wrong would have been more reassuring.

To accomplish this, we wrapped libuv in a way that allows any function that takes a callback to be passed an arbitrary pointer. That pointer is then passed along as an argument in the callback’s parameters. Preventing us from needing to use the uv_handle_t::data property and ensuring the callback always has the correct pointer type.

Below we have fully converted the previous code to use nsuv. As you can see, the pointer that would have been stored in the data parameter can now be passed to the method, making it available as an argument in the callback.

static void call_timer() {
ns_timer timer;
Foo* foo = new Foo();
int r;

r = timer.init(uv_default_loop());
//check r
r = timer.start(+[](ns_timer* handle, Foo* foo) {
delete foo;
handle->close();
}, 1000, 0, foo);
// check r

uv_run(uv_default_loop(), UV_RUN_DEFAULT);
}

For the sake of the example, a C++ lambda function was used. Remember that when passing a lambda function, it needs to be converted to a plain old function pointer using the + operator.

Also notice that we are assigning and handling all return values from each call. As mentioned above, the compiler will warn us if we do not check each call’s return codes. For simplicity of future examples, the return value will be assigned but not include a comment that it needs to be checked.

Locks

Because of all the communication between threads, mutexes were heavily used. To make things simpler, we added a couple of APIs for convenience. The first API of note is that init() accepts an optional boolean value. If true is passed in, the mutex is automatically destroyed when the destructor is called. The other was to add an API for scoped locking.

static void try_mutex() {
ns_mutex mutex;
// The optional boolean argument sets if the mutex should be
// automatically destroyed in the destructor.
int r = mutex.init(true);
// Convenience class to create scoped locks. Accepts either a
// pointer or reference.
{
ns_mutex::scoped_lock lock(mutex);
}
}

Having a mutex call destroy() in the destructor was kept false by default to maintain parity with the libuv API and prevent surprises while migrating to nsuv.

Example Usage

At first, we only implemented the libuv APIs that were necessary for us to use internally, but since deciding to open source the library we have begun to add as much of the remaining libuv APIs as possible. But despite not having yet ported the entire libuv API, it’s still possible to take advantage of what has been done. The following is an example from a test that includes the checks to demonstrate how class instances are being passed around.

#include “nsuv-inl.h”

using namespace nsuv;

ns_tcp client;
ns_tcp incoming;
ns_tcp server;
ns_connect<ns_tcp> connect_req;
ns_write<ns_tcp> write_req;

static void alloc_cb(ns_tcp* handle, size_t, uv_buf_t* buf) {
static char slab[1024];
assert(handle == &incoming);

buf->base = slab;
buf->len = sizeof(slab);
}

static void read_cb(ns_tcp* handle, ssize_t, const uv_buf_t*) {
assert(handle == &incoming);

handle->close();
client.close();
server.close();
}

static void write_cb(ns_write<ns_tcp>* req, int) {
assert(req == &write_req);
// Retrieve a reference to the uv_buf_t array as a std::vector.
assert(req->bufs().size() == 2);
}

static void connection_cb(ns_tcp* server, int) {
int r;
r = incoming.init(server->get_loop());
r = server->accept(&incoming);
r = incoming.read_start(alloc_cb, read_cb);
}

static void connect_cb(ns_connect<ns_tcp>* req, int, char* data) {
static char bye_ctr[] = “BYE”;
uv_buf_t buf1 = uv_buf_init(data, strlen(data));
uv_buf_t buf2 = uv_buf_init(bye_ctr, strlen(bye_ctr));
// Write to the handle attached to this request and pass along data
// by constructing a std::vector.
int r = req->handle()->write(&write_req, { buf1, buf2 }, write_cb);
}

static void do_listen() {
static char hello_cstr[] = “HELLO”;
struct sockaddr_in addr_in;
struct sockaddr* addr;
int r;

r = uv_ip4_addr(“127.0.0.1”, 9999, &addr_in);
addr = reinterpret_cast<struct sockaddr*>(&addr_in);

// Server setup.
r = server.init(uv_default_loop());
r = server.bind(addr, 0);
r = server.listen(1, connection_cb);

// Client connection.
r = client.init(uv_default_loop());
r = client.connect(&connect_req, addr, connect_cb, hello_cstr);

uv_run(uv_default_loop(), UV_RUN_DEFAULT);
}

The request types ns_write and ns_connect are also used in the above example. They inherit from uv_write_t and uv_connect_t respectively, and can be upcast and downcast the same way as handles. Each request type API is templated to identify which handle is being used and can return the correct handle type.

While the write() method does accept a uv_buf_t[] array, we’ve also added the ability to pass in a std::vector of buffers for ease of use. Once the request is complete, the list of written buffers can be retrieved via the ns_write::buf() API as a reference to the std::vector that’s stored internally.

Conclusion

One goal when creating nsuv was to reduce cognitive load by mimicking the libuv API naming and structure while adding safety features offered by C++. We’ve made it easy to transition existing projects to nsuv. By open-sourcing nsuv, we hope to give developers more confidence that their code will behave as expected when expected.

There is near zero runtime overhead using nsuv. The template function proxy pattern used can be completely optimized out by modern compilers. Combining that with the ability to enforce type checks at compile time, I won’t be using libuv in C++ without nsuv going forward.

Using nsuv is as simple as including the two header files from the project repository. We are still working on getting complete coverage of the libuv API and hope the community can help us decide what to work on next. We are also working on porting all applicable tests from libuv to nsuv, which can serve as usage examples. We hope that you’ll find nsuv as useful as we have.

NodeSource has delivered Node.js fresh to your Linux system via your package manager within hours, minutes, days, or weeks. For NodeSource, sustaining the community is essential because we want to support more people using Linux to have Node.js in production.

Also, we are looking for more community involvement in the project. Help will be appreciated! So if you have ideas or solutions or want to help us continue supporting open source, you can contribute to this GitHub Repo.

Continue the conversation with NodeSource here:
– Twitter
– LinkedIn
– Github
As always, the best place to contact us is via our website or [email protected]

Ready for more?

If you are looking for NodeSource’s Enterprise-grade Node.js platform, N|Solid, please visit https://downloads.nodesource.com/. For detailed information on installing and using N|Solid, please refer to the N|Solid User Guide.