Express.js: Logging info with global unique request ID – Node.js

When we deploy a system to production, logs are one of the most important elements when it comes to detecting any problem or anomaly. In the case of API implementations, one of the biggest problems is being able to track the flow of a single request. Since when there are many concurrent requests, the logs of all of them are mixed, making it impossible to track them unless they have a unique identifier.
Express.js does not have a unique identifier per request, nor does it implement a way of obtaining information about the request from the context at any site in the code. Unless the request object is passed on to all affected functions. This makes it very difficult to trace logs, or the code gets very dirty if we decide to pass the request object in all calls.

Is there a solution to this problem?

Yes, it will also be useful for projects that are already developed. Since it will not require us to pass objects between multiple calls or large changes in the different modules of the project, it will be possible.

 

 

Logger express.js node.js unique id

 

Introduction

During development, we often forget to enter logs into the code. This makes many parts of our code blind when it comes to finding errors on the part of system administrators.

In the case of implementing an API in Node.js, the most common is to use the Express framework. Espress.js allows us to implement APIs in a very simple way. Some of the most powerful features it offers are:

  • Implementation of functions to be executed in each request that arrives at the API
  • Establishment of a function flow for each API path treated
  • Objects request and response for each request received, which allow to obtain the information of the request as well as respond in a very simple way

 

Express.js Limitations

Express is a very simple framework that makes life much easier for us. It is also very light. And the way to achieve these objectives has been by simplifying the maximum number of functionalities that it implements, leaving the strictly necessary for its operation.

In other API implementation frameworks (not only in JavaScript), each request usually has its own unique identifier. It is also usually available in context, so that at any point of execution, the program can write a log unambiguously identifying the request. So that the requests can then be tracked, as well as using programs that extract statistics on the number of requests, to detect problems or failure of requests.

The ability to identify in each trace of log to which request belongs is very important. So in Express it is necessary to solve this problem if we want to have a system that in production does not mean headache every time there is a failure.

Passing the request object through all code functions would not be an acceptable solution for two main reasons:

– We would be dirtying our code by passing parameters to functions that have nothing to do with the functionality for which these functions were created
– The API module would be joined to the rest of the modules so that it would be almost impossible (or excessively tedious) to replace it with another interface, add another interface module or remove it from our system.

 

 

Logger express.js node.js unique id

 

Library to print unique identifier on each request

If what we want is for a library to do all the work for us, we just need to install this express-logger-unique-req-id library.

npm install express-logger-unique-req-id –save

Setup is very simple, as it uses Winston underneath. We could leave the default configuration if it works for us, or we could pass a valid configuration to Winston and apply it for us. In each log line you will add the id of the req that you will have generated in a unique way when the request arrives to express. The great thing about this library is that it solves problems with other libraries that worked with contexts but only worked with callbacks, but had problems with promises.

var express= require(‘express’);
var app = express();

var express_logger = require(‘./express-logger-unique-req-id’);

express_logger.initializeLogger(app);
let logger = express_logger.getLogger();

logger.debug(‘First message’);

With this simple example we would already have a logger object that we could use anywhere in our code. We will be able to obtain the object in all the modules we want and he will be in charge of printing the id of each request in a transparent way without having to do anything else.

If you want to do it on your own, read on…

 

Unique request identifier in Express.js

The first step to achieve our goal is to assign a unique identifier to each request and to maintain it even if there are several recurring requests.

For this we’re going to need two libraries:

  • uuid: The node-uuid library is widely used to generate unique identificators in the system. With it we will create an uuid that we will assign to each request in a unique way.
  • express-http-context: The express-http-context library helps us to store information associated with each execution thread. In Node.js as you know there are no threads as such, but it is based on asynchronous execution. The library will help us to share the namespaces among the different modules of our project and to share the information we need.

We install both libraries in our environment:

 

npm install uuid

npm install express-http-context

 

In the module where we start the server, typically the file server.js we import the libraries and create a namespace of the library express-http-context, which we will use to store the identifiers of the requests:

var uuid = require('node-uuid');
var httpContext = require('express-http-context');
Once the Express framework has been initialized, we will add a functionality with the use method that will be executed once for each incoming request. In it we will generate the unique identifier of the request and save it in the namespace that we have created and will be associated to the thread of our request.
var app = express();

app.use(httpContext.middleware);

// Run the context for each request. Assign a unique identifier to each request
app.use(function(req, res, next) {
    httpContext.set('reqId', uuid.v1());
    next();
});

 

In this code we have initialized the express module, and we have added a functionality that we want it to be executed with every request with the use method. Then we have called the set method, which is the way the library allows us to write data in the context of the thread. We pass a unique identifier (uuid.v1() ) to an entry that we have called reqId. That we will then need to recover this identificator in other modules.

 

 

Logger express.js node.js unique id

 

 

The Express Logger

The logger we usually use in Node.js is Winston, probably the most extended logger.

npm install winston

 

Typically, this library allows you to write log traces with different levels of criticality by calling a different method for each level and passing the message to them to print logger.debug(message), logger.info(message), logger.error(message) ... or by calling the log method and passing it the criticality level and the message as logger.log(level, message) parameters;

So far the usual way of working of the library. But we want that in case we are in a thread of execution of a request to the API we can write that message that we pass to the library along with the identificator of the request without having to pass the object request in every call to the logger. Is it possible? Of course it does, although we have to put on the overalls.

This is where the magic comes from. We will wrap the logger library and then retrieve the unique identifier of the request (if it is a request) and print it along with the message. We are also going to do it in such a way that the interface of the library remains the same, so that there will be no modifications to the rest of modules if we were already using the logger in multiple places. But don’t worry, it’s going to be a lot easier than it looks.

The wrapper is going to be a new module, which I have called logger.js. I used to have there the configuration of the logger module. But for this case it doesn’t affect so we won’t use it. In this module we will have to import the express-http-context library that we have already used before. From it we will use the get method, with which we will be able to recover the value of the unique identifier of the request you are trying to write in the log.

 

var httpContext = require('express-http-context');
var reqId = httpContext .get('reqId');

 

With these lines we will obtain each request identifier, although for it to work, the namespace and the identifier will have to be obtained in each logger call.

Now that we are able to retrieve the request identificator, the next step will be to overwrite the library methods respecting its current interface. So that we can call a method of our own which we will call formatMessage and which will add the request identifier to each line printed in the log.

The whole module would look like this:

 

var winston = require('winston');

var httpContext = require('express-http-context');

// Wrap Winston logger to print reqId in each log
var formatMessage = function(message) {
    var reqId = httpContext.get('reqId');
    message = reqId ? message + " reqId: " + reqId : message;
    return message;
};

var logger = {
    log: function(level, message) {
        winstonLogger.log(level, formatMessage(message));
    },
    error: function(message) {
        winstonLogger.error(formatMessage(message));
    },
    warn: function(message) {
        winstonLogger.warn(formatMessage(message));
    },
    verbose: function(message) {
        winstonLogger.verbose(formatMessage(message));
    },
    info: function(message) {
        winstonLogger.info(formatMessage(message));
    },
    debug: function(message) {
        winstonLogger.debug(formatMessage(message));
    },
    silly: function(message) {
        winstonLogger.silly(formatMessage(message));
    }
};

module.exports = logger;

 

The logger object is the one we export, and the rest of the modules will be called. I have made their methods match those of the library to respect its interface. Within each one we call the corresponding method of the library but passing the message to print by our formatMessage method that is in charge of making the magic. In other words, in formatMessage we recover the namespace and if there is a reqId entry in it, we take it out and add it to the message, otherwise we leave the message as it was.

And with this we have already managed to print the unique identifier of the request with each log line printed. The only change that would have to be made in the modules where printing logs is to import our module instead of the Winston module.

 

Here it is a very simple project working with this logger implementation:
https://github.com/davicente/express_logger

 

Would this solution work with other log libraries?

Yes, we would simply have to do the wrapping of the library methods, and we would only have to import our module instead of the log library we are using in our modules.

 

 

Logger express.js node.js unique id

 

Conclusion

Express.js lacks a method that allows us to access the context of each request without passing the request object in each call. But by using a combination of a library for storing information based on the chain of callbacks, creating unique identifiers and making a small wrapping on the logger library we can have this functionality without the need for our code to become a tangle of parameters that circulate through all calls to functions.

 

 

Related articles

https://ahorasomos.izertis.com/solidgear/mocha-javascript-unit-testing

https://ahorasomos.izertis.com/solidgear/refresh-token-with-jwt-authentication-node-js

https://ahorasomos.izertis.com/solidgear/clean-architecture-in-nodejs

 

15 thoughts on “Express.js: Logging info with global unique request ID – Node.js”

  1. I tried it in my application. Seems like when fetching namespace to get the unique id it is loosing the stored active data (unique id). I’m not using anything such as promises. Seems like an issue with the Namespace itself

    Reply
    • Hi Harish,
      make sure you are adding the uuid to each request, you have to do it in a phase of the middleware before you load your paths:

      // Run the context for each request. Assign a unique identifier to each request
      app.use(function(req, res, next) {
      myRequest.run(function() {
      myRequest.set(‘reqId’, uuid.v1());
      next();
      });
      });

      If it doesn’t work, can you share any code?? It works good to me.

      Reply
  2. I think continuation-local-storage is not working for post and put calls.
    NameSpace.active becomes null in post and put calls. while setting the logger for same.

    Reply
  3. Thanks for the article,
    I followed the same thing in my project, but it is working as expected in get request and not working for post request.
    Can u suggest me something , why it is happening in post and put.

    Reply
      • Unable to retrieve reqId from HttpContext for POST and PUT, works for GET. Do you know whats the issue.
        Currently logs come with unique identifier for GET requests not for POST or PUT.

        The following is the code in my app.js

        //define dependencies
        const express = require(“express”);
        const helmet = require(“helmet”);
        const morgan = require(“morgan”);
        const config = require(“config”);
        const instagram = require(“passport-instagram”);
        const passport = require(“passport”);
        const startupDebug = require(“debug”)(“app:startup”);
        const dbDebug = require(“debug”)(“app:db”);
        const authRouter = require(“./router/authRouter”);
        const userRouter = require(“./router/userRouter”);
        const customerRouter = require(“./router/customerRouter”);
        const cartRouter = require(“./router/cartRouter”);
        const orderRouter = require(“./router/orderRouter”);
        const reviewRouter = require(“./router/reviewRouter”);
        const providerRouter = require(“./router/providerRouter”);
        const itemRouter = require(“./router/itemRouter”);
        const superAdminRouter = require(“./router/superAdminRouter”);
        const baseRouter = require(“./router/baseRouter”);
        const goCreditRouter = require(“./router/goCreditRouter”);
        const Connection = require(“./database/connection.js”);
        const providerAppRouter = require(“./router/providerAppRouter.js”);
        const paymentRouter =require(“./router/paymentRouter.js”);
        const UserBean =require(“./businessobjects/UserBean.js”);
        const CustomerBean =require(“./businessobjects/CustomerBean.js”);
        const express_logger = require(‘express-logger-unique-req-id’);
        const app = express();

        //Setting up Logger
        express_logger.initializeLogger(app, config.get(“logger.fileConf”), config.get(“logger.consoleConf”));
        let logger = express_logger.getLogger();
        logger.info(“Begin************Application Starting***************”);
        //Env variables
        const PORT = process.env.PORT || 3001;

        //Creating Connection Pool
        logger.info(“Begin-Db Connection”);
        Connection.setupPool();
        logger.info(“End-Db Connection”);

        //middleware
        app.use(express.json()); // configure bodyParser
        app.use(helmet()); //http headers

        if (app.get(“env”) === “development”) {
        console.log(“morgan enabled”);
        app.use(morgan(“tiny”));
        }

        //use to access a resource in the root through url
        app.use(express.static(“resource”));
        //restructuring the router
        app.use(“/”, baseRouter);
        app.use(“/auth”, authRouter);
        app.use(“/users”, userRouter);
        app.use(“/customer”, customerRouter);
        app.use(“/item”, itemRouter);
        app.use(“/cart”, cartRouter);
        app.use(“/reviews”, reviewRouter);
        app.use(“/orders”, orderRouter);
        app.use(“/provider”, providerRouter);
        app.use(“/payment”, paymentRouter);
        app.use(“/goCredit”,goCreditRouter);
        //For ProviderApp
        app.use(“/providerApp”, providerAppRouter);
        //For SuperAdmin
        app.use(“/superAdmin”, superAdminRouter);
        //Error handling Should be always below all other middlewares
        //app.use(error);

        logger.info(“Begin-Instagram Authentication Configuration”);
        //Instagram authentication
        const instagramStrategy = instagram.Strategy;
        app.use(passport.initialize());
        app.use(passport.session());
        passport.serializeUser((user, done) => {
        done(null, user);
        });
        passport.deserializeUser((user, done) => {
        done(null, user);
        });
        const instaConfig = {
        clientID: “xxxx”,
        clientSecret: “xxx”,
        callbackURL: “xxx”
        //callbackURL: “xxx”
        };
        const instagramInit = async function(accessToken, refreshToken, profile, callback) {
        console.log(“Access Token ” + accessToken + ” profile ” + profile);
        console.log(`Check if User ${profile.id} is present in DB`);
        let userBean =new UserBean();
        try {
        await userBean.getbyName(profile.username);
        } catch (error) {
        //SomeLogic
        }else{
        throw error;
        }
        }
        var token = userBean.generateToken();
        let user={
        userId:userBean.userId,
        userRole:userBean.userRole,
        token:token
        }
        return callback(null, user);
        };
        passport.use(new instagramStrategy(instaConfig, instagramInit));
        passport.authenticate(“oath2”, { failWithError: true });

        app.get(“/auth/instagram”, passport.authenticate(“instagram”));
        app.get(
        “/auth/instagram/callback”,
        passport.authenticate(“instagram”, {
        //successRedirect: “/success”,
        failureRedirect: “/auth/instagram”
        }),
        (req, res) => res.redirect(‘xxxx’ + JSON.stringify(req.user)));

        logger.info(“End-Instagram Authentication Configuration”);
        app.listen(PORT, () => {
        logger.info(`app running port ${PORT}`);
        });
        logger.info(“End-************Application Starting***************”);

        Reply
  4. Thanks for the idea!

    Saw your post on Stackoverflow, however, this could become a problem if the id is being stored in a DB, as if the node instance is restarted, the id won’t be unique.

    You should also update the post to use uuid instead of node-uuin (deprecated).

    Reply
  5. Hi

    It’s just an advice, how to add request id to winston log, separately from message

    let requestIdFormat = winston.format(info => {
    let requestId = httpContext.get(‘reqId’);
    if (requestId) {
    return Object.assign({}, info, {
    reqId: requestId
    });
    }
    return info
    });

    fileConf = {
    level: ‘debug’,
    filename: ‘./logs.log’,
    handleExceptions: true,
    format: combine(
    requestIdFormat(),
    timestamp(),
    json()
    ),
    maxsize: 5242880, // 5MB
    maxFiles: 5,
    colorize: false
    };

    Reply

Leave a Comment

¿Necesitas una estimación?

Calcula ahora

Privacy Preference Center

Own cookies

__unam, gdpr 1P_JAR, DV, NID, _icl_current_language

Analytics Cookies

This cookies help us to understand how users interact with our site

_ga, _gat_UA-42883984-1, _gid, _hjIncludedInSample,

Subscription Cookies

These cookies are used to execute functions of the Web, such as not displaying the advertising banner and / or remembering the user's settings within the session.

tl_3832_3832_2 tl_5886_5886_12 tve_leads_unique

Other