node modules Archives

Node

Now that all of that left-pad nonsense is behind us, we can get back to focusing on the business of utilizing node modules, or more specifically, how to upgrade our node packages in a safe, verifiable, refactory way (as refactory as Javascript lets us get, anyway).

The Problem

Recently, I was tasked with bringing our react-router dependency up to date, which was woefully out-of-date: we were running version 0.13.3, and I believe the most current version as of this writing is 2.4.2.

In our application, our routing code is probably among the most difficult to reason about and hairiest code our frontend employs. It relies a great deal on a home-spun web history module, it accommodates legacy routes and redirects them to their proper current route location, and employs some “smart routing” to determine an object’s route depending on its ID and its context. There are also many places throughout the app where we were exposing the Router object in order to programmatically route to locations (something that isn’t available as an option in later versions of react-router).

The approach

We had recently brought in two refactoring experts to give instruction on proper refactoring of legacy code, closely following Martin Fowler’s pivotal book on the subject. Unfortunately, our instructors didn’t dive as deeply into refactoring frontend code as the frontend engineers would’ve liked, but we still had some valuable takeaways from the instruction that we could apply to our work. I would be utilizing the tools we gained from this instruction to bring this module up to date.

First things first

React-router, mercifully, has a lot of documentation around their major version upgrades. We dug into the documents to gain some understanding of the biggest changes we would have to make.

I won’t get into the specifics of the upgrades unless it’s suitable to explain the process of migrating to the new modules.

Feature flags

Javascript is unsafe. Profoundly so. Just about anything you do in the code can introduce some kind of side effect, even something as mundane as importing a module (this, in fact, happened during this process as I found a certain module modified the global state and by importing it after the new module had imported, would break the new module).

The best way to safely make changes to your code is to introduce feature flags or environmental variables or some other environment factor that provides control over the introduction of new code. In our Java application, we use a plugin called Togglz to store the values in a database and inject them into the frontend.

We also use Webpack; we could forego the Java implementation and in our Webpack configuration, we could provide the configuration we wanted. This can be done through an aliased module:

// In the webpack config
...
resolve: {
    alias: {
        featureFlags: {
            'react_013_to_1': true
        }
    }
}
...

// In a file that needs the feature flag
var featureFlags = require('featureFlags');

if (featureFlags.react_013_to_1) {
    // do something
}

There are many ways to do this — figure out the one that works best for your environment and build setup.

Two module versions at a time?

Now that we have somewhere we can turn on and off the feature in question, we have to figure out a way to import and safely use two different versions of a module at the same time. It turns out this is pretty trivial if we break some rules.

We were running 0.13.3 of react-router, but we wanted our first step to be to 1.0.3, or the highest version of version 1.x.x available. The reason is we want to end up as close to the 2.x.x version as we can possibly get so that transition is made much easier.

Since react-router is a direct dependency of our frontend entry point, there’s no real way around the fact that you can only run one version of that dependency at the same time — at least not through npm. What we can do is clone react-router from github locally, build it, and then add its libs to our application, checked into our codebase.

At this point, you might be crying foul: Under no circumstance should we be including a library as a checked in member of our code. Indeed, node_modules should be part of your .gitignore config. But the intention of that rule is to make your build process transferrable, and avoid overhead that comes with having a ton of dependency code checked in. In this case, we want to do something temporary and build-agnostic.

The plan is this:

We clone the react-router repo
We go into the cloned repo and git checkout version 1.0.3 of the module.
We npm install to get the module’s dependencies
We npm run build to create the 1.0.3 build dist of the module
We create a folder in our frontend code called module-upgrades, and inside of that, a folder called react-router-1
We copy the lib folder generated by the react-router module into the react-router-1 folder

We can now import and use two versions of react-router with the flip of a feature flag switch. The end game of this approach is that once we’ve tested the updates thoroughly, we will upgrade our app’s package.json file to now point at version “^1.0.0” of react-router, refactor references to the temporary react-router (“react-router-1”) we have checked into our app to point to the actual node module (defined in package.json), and continue on our merry way.

Migrating Components

In the case of the router, we wanted to be able to convert one route at a time and then test it. First, we came up with a plan to test it:

Navigate to route directly
Navigate to route through forward button
Navigate to route through back button
Navigate to route through legacy route (if applicable)
Navigate to route through programmatic interface

This plan will vary, or even if you decide to implement a plan. Our unit tests around this area of code were so spotty that we decided it’d be best to at least do a smoke test around the route once we converted it.

Next, we create a place to switch between routes. All of our routes were represented in a file called Routes.jsx that merely concatenated the routes into an array and exported them for consumption. It looked something like this

var Route = require('react-router').Route;

var routes = [
 ,
 ,
 ...
 ];

module.export = routes;

Since this file was being used by the old router, we thought it best to simply create a temporary new file called NewRoutes.jsx that would be a carbon copy of the old Routes.jsx, and migrate the components over piecemeal. Once we were ready to remove the feature flags, it would simply be a matter of removing the old Routes.jsx file and renaming NewRoutes.jsx, both of which are very easy to do as refactors, even in Javascript.

NewRoutes.jsx looked like this:

var Route = require('react-router').Route,
 NewRoute = require('module-upgrades/react-router');

var routes = [
 , // migrated
 , // to be migrated
 ...
 ];

module.export = routes;

During the process of migrating each route, we would load both routers, but once all routes were migrated over, we could remove the reference to the old router, and this NewRouter.jsx file would only handle new routes.

Abstracting away differences

Now that we’ve secured a path toward being able to safely test changes and refactor our dependencies, we can start the work of migrating our code.

Since the specifics of this depend heavily on your particular migration, there’s no point in delving into the specific transforms we had to do with react-router — only that we wanted to create a set of transforms that could be applied across our codebase.

In our app, we had essentially two different types of programmatic routing:

We would point to a location by “name” (a relic of react-router 0.13)
We would point to a location by name, but also provide data

Since scenario 2 is merely an extension of scenario 1, we could just extend the solution of 1 to work with 2.

Here’s an example of our old programmatic routing mechanism:

...
var RouterContainer = require('./../../routes/RouterContainer'),
...
RouterContainer.get().transitionTo(DocumentTypeToPathMap[documentType], { id: this.props.id }, { projectId: this.getProjectId() });
...

We obtained a reference to a “RouterContainer”, which held a reference to the router singleton (retrieved with the static “get” method). Then we called the method “transitionTo” on it, passing the name of the route, parameters (in this case, just an id), and query or search values (projectId)

Since the history handling of the new version of react-router was delegated out to the new history dependency, there was no longer a built-in way of passing params to a route, so we had to create a new method that could take a route and an id and concatenate them into a properly formed route. The new history dependency did, however, allow us to pass in a search string.

Now we have a transform for all cases of programmatic routing:

The above example, to work in the new world, must turn into this:

...
var NewJamaLocation = require('jama/routes/NewJamaLocation'),
 ...
NewJamaLocation.push({ pathname: NewDocumentTypeToPathMap.documentTypeToPath(documentType, this.props.id), search: '?projectId='+this.getProjectId() });

NewJamaLocation represents the new history dependency singleton. It has a push method that is analogous to transitionTo.push accepts a location object that contains a pathname and search value. We’ve abstracted out the formulation of the path to NewDocumentTypetoPathMap which has a static method that will take a documentType and an id and create a viable path from it.

However, we want to be able to safely implement this, so we’re going to do so with the conditional logic. Remember, even importing has side effects, so our end result will look like this:

var NewJamaLocation,
    RouterContainer,
    featureFlags = require('featureFlags');
...

if (featureFlags.react_013_to_1) {
    NewJamaLocation = require('jama/routes/NewJamaLocation');
} else {
    RouterContainer = require('./../../routes/RouterContainer');
}

...

if (featureFlags.react_013_to_1) {
    NewJamaLocation.push({ pathname: NewDocumentTypeToPathMap.documentTypeToPath(documentType, this.props.id), search: '?projectId='+this.getProjectId() });
} else {
    RouterContainer.get().transitionTo(DocumentTypeToPathMap[documentType], { id: this.props.id }, { projectId: this.getProjectId() });
}

...

Another way to do this would be to write the conditional switch into the RouterContainer, and create a shim function “transitionTo” that would handle the transition from the old router to the new history object. I avoided this path because I wanted us to get away undocumented methods of implementing libraries. Many libraries have very good documentation and examples. The closer we come to implementing our code the way it is documented at the source, the easier it is for newcomers to jump in our code and learn it, and the less bespoke documentation and domain knowledge we have to write or know for our code.

Copy/Paste

Now that we have a mechanical transform of a programmatic route, we can apply it everywhere we do programmatic routing. Literally keep a copy/pasted implementation that you can drop in (to avoid fat-fingered errors).

I suggest doing this as well when it comes to conditionals. During the migration work, I had a scratch pad open with the following sets of “macros”:

var NewJamaLocation,
    RouterContainer,
    featureFlags = require('featureFlags');
...

if (featureFlags.react_013_to_1) {
    NewJamaLocation = require('jama/routes/NewJamaLocation');
} else {
    RouterContainer = require('./../../routes/RouterContainer');
}

if (featureFlags.react_013_to_1) {

} else {

}

Again, the fewer times you have to explicitly type things, the fewer errors you’ll be prone to.

A note on committing

Throughout this process, you should employ a micro-commit philosophy — that is, you’ll want to commit after just about every meaningful code transformation, and especially after any “dangerous” changes. When I say “dangerous”, I mean any code changes that contain a change that isn’t a provable or safe transformation.

Finishing

Once we’ve had our code out in the wild for some time and feel it has been thoroughly tested, we can start our path toward removing the old, obsolete code. At this point, it should be as easy as removing the feature flag conditionals to always run the new code.

I recommend using IntelliJ or another well-mannered IDE that understands refactoring to do filename transforms and dependency renamings. It will catch paths you may have perhaps missed a bit better than a simple grep, since it understands import statements and such.

You can then delete your checked in module, have your module imports point to the resident node module (in this case, react-router), and update your package.json file to point to appropriate version (in this case, ^1.0.3), do an npm install, and if you’ve done everything correctly, there should be no difference whatsoever between having the feature flag on and having removed it and pointed to the newer version of the module.

Caveats

Another module upgrade we attempted to perform as safely as possible was upgrading react from 0.14.x to 15.x.x. This upgrade, however, proved impossible in our system. Due to our feature flagging mechanism, which is reliant upon a separate backend generating frontend global objects, we had no way to be able to switch between the versions. React also doesn’t allow two versions of itself to be run at the same time.

There is a possible path to a safe React upgrade if your application has a Node backend, but your server will still require a restart in order to configure the modules to pull from the different versions, so there doesn’t appear to be a way to “hot swap” between React versions.

That said, React goes out of its way to inform of deprecations and create tools for migration, so upgrading isn’t typically painful once you’ve gotten past a certain threshold.

Tag Archive for: node modules