N|Solid v4.9.0 is now available

NodeSource is excited to announce N|Solid v4.9.0 which contains the following changes:

This version of N|Solid contains amazing features like M__achine Learning support__ and SBOM support (Software Bill of Materials) , it also contains the latest Node.js versions: v14.21.1 (LTS), v16.18.1 (LTS) and v18.12.1 (LTS), few updates and stability improvements.

For detailed information on installing and using N|Solid, please refer to the N|Solid User Guide..

Changes

NodeSource is excited to announce N|Solid v4.9.0 which contains the following changes:

Machine Learning support: this feature contains common Node.js issues detection powered by machine learning analysis, also allows the users to train custom models to be used to detect similar patterns in your applications metrics, the machine learning detection is also integrated with the global notification system and the events logs.

SBOM support (Software Bill of Materials): N|Solid has added support for SBOM reporting in all applications connected to the N|Solid console, the report is offered in two formats: JSON and PDF, it contains the information for the dependency inventory of an specific application; it includes valuable information lik licensing and the security status for each dependency used.

There are three available LTS Node.js versions for you to use with N|Solid, Node.js 16 Gallium, Node.js 14 Fermium and Node.js 18 Hydrogen.

N|Solid

N|Solid v4.9.0 Fermium ships with Node.js v14.21.1.

N|Solid v4.9.0 Gallium ships with Node.js v16.18.1.

N|Solid v4.9.0 Hydrogen ships with Node.js v18.12.1.

Node.js

The Node.js 14 Fermium LTS release line will continue to be supported until April 30, 2023.
The Node.js 16 Gallium LTS release line will continue to be supported until September 11, 2023.
The Node.js 18 Hydrogen LTS release line will continue to be supported until April 30, 2025.

Supported Operating Systems for N|Solid Runtime and N|Solid Console

Please note that The N|Solid Runtime is supported on the following operating systems:

Windows:

Windows 10
Microsoft Windows Server 1909 Core
Microsoft Windows Server 2012
Microsoft Windows Server 2008

macOS:
macOS 10.11 and newer

RPM based 64-bit Linux distributions (x86_64):

Amazon Linux AMI release 2015.09 and newer
RHEL7 / CentOS 7 and newer
Fedora 32 and newer

DEB based 64-bit Linux distributions (x86_64, arm64 and armhf):

Ubuntu 16.04 and newer
Debian 9 (stretch) and newer

Alpine
Alpine 3.3 and newer

Download the latest version of N|Solid

You can download the latest version of N|Solid via http://accounts.nodesource.com or visit https://downloads.nodesource.com/directly.

New to N|Solid?

If you’ve never tried N|Solid, this is a great time to do so. N|Solid is a fully compatible Node.js runtime that has been enhanced to address the needs of the Enterprise. N|Solid provides meaningful insights into the runtime process and the underlying systems. Click [here]

Enhance Observability with Opentelemetry tracing – Part 1

Recently, conversations have been increasing around OpenTelemetry; it is gaining more and more momentum in Node.js development circles, but what is it? How can we take advantage of the key concepts and implement them in our projects?

Of note, NodeSource is a supporter of OpenTelemetry, and we have recently implemented full support of the open-source standard in our product N|Solid. It allows us to make our powerful Node.js insights accessible via the protocol.

Opentelemetry is a relatively recent vendor-agnostic emerging standard that began in 2019 when OpenCensus and OpenTracing combined to form OpenTelemetry – seeking to provide a single, well-supported integration surface for end-to-end distributed tracing telemetry. In 2021, they released V1. 0.0, offering stability guarantees for the approach.

And most important, OpenTelemetry is an open-source observability project/framework with a collection of software development kits (SDKs), APIs, and tools for instrumentation from the Cloud Native Computing Foundation (CNCF).

W3C Trace Context is the standard format for OpenTelemetry. Cloud providers are expected to adopt this standard, providing a vendor-neutral way to propagate trace IDs through their services. Organizations use OpenTelemetry to send collected telemetry data to a third-party system for analysis.

But to break down its history a bit, we think it’s important to understand the concept of __Observability__.

At Nodesource, as you likely know, we work daily in Observability, focusing exclusively on the Node.js runtime and N|Solid from N|Solid 4.8.0 supports some OpenTelemetry features. But before getting deeper in OTEL, it is important to understand Observability and try to resolve this important question: What is Observability?

Setting the foundations to talk about OpenTelemetry

It’s important to understand that when we talk about Observability, we need first to know what questions we seek to answer or clarify when detailing a system.

The first question often asked is __why__ my application has specific behavior. And to solve this and other questions, we first must instrument our system so that our application can emit signals, that is, traces, metrics, and logs. When we correctly do this, we have the necessary information needed.

Observability is the ability to measure the internal states of a system by examining its outputs. – Splunk

Detailing a system through Data Collection: Telemetry Data

Your systems and apps need proper tooling to collect the appropriate telemetry data to achieve Observability.
But what is the telemetry data that we need?

The three key concepts are :

Metrics

Logs

Traces

Ok, let’s define each of these concepts:

Metrics

__Metrics__: are aggregations over a period of time of numeric data about your infrastructure or application. Examples include system error rate, CPU utilization, and request rate for a given service.

As quoted by isitobservable.io, OpenTelemetry has three metric instruments :

__Counter__: a value that is summed over time (similar to the Prometheus counter)
__Measure__: a value that is aggregated over time (a value over some defined range)
__Observer__: captures a current set of values ​​at a given time (like a gauge in Prometheus)

The context is still very important, along with metric information like name, description, unit, kind (counter, observer, measure), label, aggregation, and time.

Logs

__Logs__: A Log is a timestamped message emitted by services or other components. They are not necessarily associated with any particular user request or transaction, but they become more valuable when they are.

The logical line would tell us that here we must jump to traces because it is part of the three key concepts. But before defining what a trace is, we must zoom in on the concept of __Span__.

Span

__Span__: A Span represents a unit of work or operation. It tracks specific operations that a request makes, painting a picture of what happened during the time in which that operation was executed.

A span is the building block of a trace and is a named, timed operation representing a piece of the workflow in the distributed system. All traces are composed of Spans.

Traces

Traces__: A Trace records the paths taken by requests (made by an application or end-user) as they __propagate through multi-service architectures, like microservice and serverless applications. It is also known as Distributed Trace. A trace is almost always an assessment of end-to-end performance.

Without tracing, it is challenging to pinpoint the cause of performance problems in a distributed system.

Suppose you realize we broke into the three pillars of Observability when introducing the concept of Span. In that case, however, the three pillars and Span conform to what is known as __Telemetry Data__, which are simple __signals emitted from applications and resources about their internal state.__

The core concept of Context Propagation

When we want to correlate events across our services’ boundaries, we look for a context that helps us identify the current trace and Span. But context is not the only thing we need; we also need __propagation__.

If you are with us following the article carefully, you will realize that in the definition of trace, we talk about the word __‘propagation’__. You might wonder what this means.

Propagation is how context is bundled and transferred in and across services, often via HTTP headers. Now, With these clear concepts, we can begin to understand the concept of __Context Propagation__.

A critical functionality required to implement Distributed Tracing is the concept of Context Propagation. We can define it as a mechanism for storing state and accessing data across the lifespan of a distributed transaction, either across execution contexts inside a process or across the boundaries of the services that conform to our system.

For In Process propagation__, we typically use something like the __AsyncLocalStorage class from the async_hooks module.

Whereas Across Processes__, it will depend on the IPC protocol used. For example, for HTTP, there’s the Trace Context specification from the _W3C__, which defines the _traceparent and tracestate headers to propagate tracing info.

Getting into a Distributed Application

Let’s say we have a distributed application like the one in the picture. It has 4 Nodejs services: API, auth, Service1 and Service2, and 1 database.

Imagine we’re having intermittent performance issues. They could come from several points:
– Database access
– Network link status,
– DNS request latency, etc.
Finding where exactly may become a very hard and time-consuming task; the harder, the more complex the system is.

Distributed tracing will help us A LOT with that, as we’ll generate tracing information on every point of the distributed system (A, B, C, D, and E). Not only that, but while the request goes through all the services, thanks to Context Propagation, some ‘tracing state’ will be passed along so all the tracing info can be linked to the very same request.

Instrument your system

To get visibility into the performance and behaviors of the different microservices, we need to instrument the code with OpenTelemetry to generate traces. But first, let’s define what Instrumentation is…

Automatic Instrumentation

With __Automatic Instrumentation__, our instrumentation libraries will automatically take the configuration provided (through code or environment variables) and do most of the work.

In the following example, using the OpenTelemetry SDK, we show how we can automatically generate spans for every HTTP transaction handled by the Nodejs HTTP core module.

Manual Instrumentation

__Manual instrumentation__, on the other hand, while requiring more work on the user/developer side, enables far more options for customization, from naming various components within OpenTelemetry (for example, spans and the traces ) to add your own attributes, specific exception handling, and more. See the following example shows how to manually generate a Span using the OpenTelemetry SDK.

How to Implement Opentelemetry in my project?

The way we historically would implement a typical observability pipeline is shown in the following picture.

In this case, having all that data at your disposal is great and can give us a valuable overview of our system, but unless we are able to correlate the observability signals somehow: metrics, logs, and traces, we won’t be able to have the best of it.
OpenTelemetry comes to solve this problem. The solution is going to come from correlating these signals. This can be done by applying the same concept of Context Propagation that was used for Traces to Metrics and Logs, so in this case, identifiers such as the trace_id and the span_id are associated with those signals.

OpenTelemetry spanId and traceId can correlate Logs and Metrics with a specific Span in a Trace.

Opentelemetry Components

OpenTelemetry is much more… Before finishing this article, it is important to describe the different components of OpenTelemetry.

Note: For more detail, read the specification overview of the Opentelemetry project.

OpenTelemetry API: It provides an API, which defines data types and operations for generating and correlating tracing, metrics, and logging data.

💚 From N|Solid 4.8.0 we provide an implementation of the OpenTelemetry TraceAPI, allowing users to instrument their own code using the de-facto standard API.

OpenTelemetry SDK: It provides Language-specific implementations of the API.

OpenTelemetry OTLP: A protocol to transport the Telemetry Data.

💚 With N|Solid 4.8.0 we support many instrumentation modules available in the OpenTelemetry ecosystem. Supporting exporting traces using the OpenTelemetry Protocol(OTLP) over HTTP.

OpenTelemetry Collector: To receive, process, and export Telemetry data.

💚 In N|Solid 4.8.0 is now possible to send N|Solid Runtime monitoring information (metrics and traces) to backends supporting the OpenTelemetry standard like multiple APMS (Dynatrace, Datadog, Newrelic).

OpenTelemetry Semantic Conventions: To have well-defined naming for the attributes associated with the signals: (service.name, http. port, etc.)

We know that there may be other key concepts to develop around Opentelemetry, and for this reason, we invite you to visit the direct website of the project or the Github Repo directly.

This introductory article gives us the basis for sharing a demo we prepared for NodeConf.EU, where we apply open-source tools to implement Opentelemetry in your project. We invite you to stay tuned for our next blog post. 😉 Wait for the second part!

Conclusions

Traces are really useful for understanding modern distributed systems.
We build better software when we get the best of our traces.
With OTel (Opentelemetry), we’re able to have maximized insights and answer future questions without having to make any code changes.
#OTel provides interoperability with observability tools.
Collect and correlate telemetry data is easy if you follow the OpenTelemetry framework.
As far as we know, the OpenTelemetry community is working hard to develop support for metrics and logs. Waiting for news soon! 🤞
Note: If you want to learn more about OpenTelemetry in Javascript, click HERE

To start getting more value out of your traces and metrics, you can use Opentelemetry with N|Solid back-end.

Achieve Your Performance Goals With N|Solid

We know that you want to get the best out of your application and to do it professionally, you will surely need a great ally to help you with various tools without affecting your performance. We do not want to stay in a ‘marketing speech’ where we tell you that we are the best… you can 👀check it directly here with this Open Source tool that also includes OTEL Results.

We’d love to hear more from you! 💚
– Feel free to TRY N|Solid and get in touch with us on Twitter at @nodesource.

Top 10 N|Solid —APM for Node— features you needed to use

Nearly a year ago, we launched N|Solid SaaS, and although there are still a few months to go before our anniversary, we wanted to share the top 10 features ofN|Solid that make us proud every day of what we have built.

N|Solid is the best way to monitor and secure your Node.js applications (including in production) that are trusted by developers, software teams, and global enterprise companies. It has an array of features like other APMs, but we go deeper with our insights and are more performant than all others. We created N|Solid to help companies build the best software with Node and save time resolving issues. Because there is significant risk in deploying open-source applications without knowing the security gaps, we provide features to prevent security issues and insights for resolving them.

We are confident we’re the best APM solution for Node. js-based applications; if you are using Node, you should be using our runtime. We’re a complete product/solution, not just an APM focused on Node.

About N|Solid

N|Solid is a toolset built on Node.js that provides a number of enhancements to improve troubleshooting, debugging, managing, monitoring, and securing your Node.js applications. It is 100% compatible with the open-source project and requires no instrumentation of your code.

N|Solid Console

N|Solid provides a web-based console, ‘N|Solid Console’ to monitor your applications but also allows you to introspect your Node.js applications, in the same way, directly in the CLI if you run the __N|Solid Runtime__.

N|Solid Runtime

If you want to introspect your Node.js applications and have the most control from your command line, you’ll run them with the N|Solid Runtime, which is shaped similarly to a typical Node.js runtime but provides some additional executables.

To install N|Solid Runtime, download and unpack an N|Solid Runtime from the N|Solid download site.

Why N|Solid is an APM

Traditionally, the acronym APM has been used to refer to application performance management. However, in recent years it also refers, perhaps more correctly, to Application Performance Monitoring, and that is exactly what N|Solid does, which is why its categorization in this spectrum of applications is correct. Something important to highlight is that it is not a polyglot APM; it is clearly an APM specialized in Node.js, which has always been our focus.

While other APMs support Node.js, none provide the level of insights N|Solid can. In many cases, the APMs can become a part of the problem by consuming significant resources due to how they are designed. But don’t take our word for it. You can check it with real data through this OS Project — The APM’s Benchmark tool —.

APM’s Benchmark Tool – Overview Screen

N|Solid APM (Self-hosted or SaaS) is the best observability and insights tool to manage Node performance and security, and the full platform access enables you to really #KnowyourNode

In this blog post, we want to wrap it up our product series, briefly telling you about the 10 main features of N|Solid. We hope you like it and it helps you get the most out of our product.

[1] Project & Applications Monitoring in N|Solid

Visually view application behavior and identify performance and security issues.

With Project & Application Monitoring, you can track a website or any application based on Node.js. This feature allows you to collect your log data to help developers detect bugs and process use, track downtime, and improve performance to be consistent and focused on the end-user experience.

N|Solid APM – Projects & Applications View

This area is mainly made up of 3 main views that use the Projects and Applications and Process Monitoring:
– Applications view
– Application summary view
– Processes view

Read more about this feature here: https://nsrc.io/ProjectApplicationsMonitoringNS

[2] Process Monitoring in N|Solid

Access deep performance insights.

The applications and associated processes are displayed in this feature of our N|Solid Console. You can visualize Event Loop Estimated Lag, Heap Used, or CPU Used, for example, and you can correlate these metrics in a planimetry. You can also select a specific process to know its general status and vulnerabilities and choose a specific graphic to visually represent the selected information.

N|Solid APM – Process Monitoring

Read more here: https://nsrc.io/ProcessMonitoringNS

[3] CPU Profiling in N|Solid

Shows what functions consume CPU% and how resources are allocated.

CPU Profiling allows you to understand where opportunities exist to improve your Node processes’ speed and load capacity. This feature shows what functions consume CPU% and how resources are allocated.

N|Solid APM – Flamegraph-CPU Profile

Read more here: https://nsrc.io/CPUProfilingNS

[4] Worker Threads in N|Solid

View In-depth metrics of each worker thread.

Worker threads are treated first class and have the same access to CPU profiles, snapshots, etc. as the main process. We are the only solution that has full support worker threads.

View In-depth metrics of each worker thread. With this feature, identify opportunities to improve the performance of CPU-intensive work.

Read more here: https://nsrc.io/WorkerThreadsNS

[5] Capture Heap Snapshots in N|Solid

Understand where and how memory is being used

Taking heap snapshots is a great way to help identify the underlying problem when faced with a memory leak or performance issue. In this way, you will be able to understand where and how memory is being used, and you will be able to quickly resolve memory leaks and performance issues.

N|Solid APM – Capture Heap Snapshots

Read more here: https://nsrc.io/HeapSnapshotsNS

[6] Memory Anomaly Detection in N|Solid

View In-depth metrics of each worker thread.

Identify Memory anomalies taken with a more accurate detection method.
– Insights and metrics are historical, before and after the incident happened.
– Get anomalies at different heap usage levels.
– Detect correlation between sets of memory-specific metrics.
– Filter results by specific processes inside your application.

N|Solid APM – Memory Anomaly Detection

Read more here: https://nsrc.io/MemoryAnomalyNS

[7] Vulnerability Scanning – NCM – in N|Solid

Know all of the potential vulnerabilities within your application.

NCM is security, compliance, and curation tool around the 3rd-Party Node.js & JavaScript package ecosystem. It provides protection against security vulnerabilities and licensing compliance issues and provides risk assessment when working with a 3rd-party ecosystem.

The N|Solid Console can be configured to perform periodic verification of all packages loaded by all N|Solid processes.

N|Solid APM – Vulnerability Scanning from N|Solid Runtime

NCM provides

Actionable insights.
Offline vulnerability scanning.
Prevent processes in an application from launching if they have vulnerabilities with “strict mode.”
NCM-CI (Service Tokens and CI Processes) customization.

__Note__: NCM can be viewed from 3 locations: full overview, per application, and per process.

Read more here: https://nsrc.io/VulnerabilityScanningNS

[8] HTTP Tracing support in N|Solid

Enables the ability to debug application latency and other issues.

HTTP tracing gathers throughput and the lifecycle of any HTTP, DNS, or other types of request.
– Debug latency issues, monitor your services, and more with the collected information.
– See in a timeline graph the density of the number of tracked spans.
– Inspect each span for more detail on the collected trace.
– Filter the results by the attributes of a span and delimit them to the time range.

N|Solid APM – HTTP Tracing Support

Read more here: https://nsrc.io/HTTPTracingNS

[9] Global Alerts & Integrations in N|Solid

Be aware of issues and vulnerabilities. Pre-configured API integrations with key 3rd party services.

You can use automation to trigger alerts over integrations, CPU profiles, or heap snapshots. Be aware of issues and vulnerabilities, Pre-configured API integrations with key 3rd party services.

So when creating the heap snapshot, for example, I will have the notification directly in Slack of my N|Solid Console’s behavior; from there, I can check it by opening the Console.

N|Solid APM – Global Alerts & Integrations – Slack Example

Read more here: https://nsrc.io/GlobalAlertsIntegrationsNS

[10] Distributed Tracing in N|Solid

Better understand the factors that affect an application’s latency.

Distributed tracing is a core component of Observability mainly used by site reliability engineers (SREs) but also by developers and is recommended in that way to obtain the greatest benefits as a team in charge of modern distributed software.

As your system scales, you’ll need to add a tracing and refine sampling capabilities, which means getting the context to understand the complexity of distributed architectures.

N|Solid APM – Distributed Tracing

Distributed Tracing provides several solutions, which include:

Monitoring system health
Latency trend and outliers
Control flow graph
Asynchronous process visualization
Debugging microservices

Read more here: https://nsrc.io/DistributedTracingNS

Still, on our roadmap, we are planning and executing features that will shake up the ecosystem in the coming months. Stay tuned! 😎

Top Ten Features In N|Solid

🧭Projects & Applications Monitoring in N|Solid – https://nsrc.io/ProjectApplicationsMonitoringNS

🌌 Process Monitoring in N|Solid – https://nsrc.io/ProcessMonitoringNS

🔍 CPU Profiling in N|Solid – https://nsrc.io/CPUProfilingNS

🕵️‍♂️ Worker Threads Monitoring in N|Solid – https://nsrc.io/WorkerThreadsNS

📸 Capture Heap Snapshots in N|Solid – https://nsrc.io/HeapSnapshotsNS

🚨 Memory Anomaly Detection in N|Solid – https://nsrc.io/MemoryAnomalyNS

🚩 Vulnerability Scanning & 3rd party Modules Certification in N|Solid – https://nsrc.io/VulnerabilityScanningNS

👣 HTTP Tracing Support in N|Solid – https://nsrc.io/HTTPTracingNS

⏰ Global Alerts & Integrations in N|Solid – https://nsrc.io/GlobalAlertsIntegrationsNS

🪄 Distributed Tracing in N|Solid – https://nsrc.io/DistributedTracingNS
…and more

Want to try N|Solid?

To check out the top 10 features and more in N|Solid, create your account in sign up or sign in, in the top right corner of our main page. More information is available here.

As always, we’re happy to hear your thoughts – feel free to get in touch with our team or reach out to us on Twitter at @nodesource.