monorepo setup with parcel 2 (2/n) - Node.js / Nest.js / Fastify / Lambda / Serverless

tldr: parcel doesn't work with nestJS, because they have differing opinions about which parts of javascript are garbage

In the previous post, I walked through converting my Create React App based client to a parcel build system.  Now I'm going to do the server.

Present state, the server builds on tsc and gets webpacked.  A bunch of this tooling is coupled with specific versions of React and Jest, as supported by create-react-app.  Jest tests are transpiled using @swc/core, because using tsc is unusably slow for this; something like 2 minutes and change to run my unit tests instead of 20 seconds. 20 seconds is frankly still not great, and is related to known issues with Jest as a test runner BUT I DIGRESS

Good news is, during my previous attempt to upgrade this pile to yarn v3, I have upgraded server's @swc/core and @swc/jest to sane versions, so I'm not expecting this to be too hard.  I'm prebuilding everything, and then Serverless uses the build output directory to do the AWS Lambda packaging and upload.  This works for both my main REST server, and also my lambdas that are fronted by my API Gateway Websocket Application.  

Knock on wood

"install parcel as a dev dependency"

cd packages/server
yarn add --dev parcel

"add source for parcel in package.json"

Things are a little more confusing now.  I've got multiple entry points here.  Previously, I was running tsc -b src which would give me all my .js files.  There's numerous entry points here, all called out in my serverlessHTTPS.yml and serverlessWSS.yml files:

# serverlessHTTPS.yml
built/lambda.handler

# serverlessWSS.yml
built/connections/connectionHandlers.connectHandler
built/connections/connectionHandlers.sendAuthTicketHandler
built/connections/connectionHandlers.disconnectHandler
built/connections/connectionHandlers.defaultHandler
built/chat/chatHandler.sendChatHandler

gooooogling.... https://dev.to/terrierscript/build-aws-lambda-function-with-typescript-only-use-parcel-bundler-426a

Looks like something like this should work:

$ yarn parcel build handler.ts --target=node --global handler -o index.js --bundle-node-modules --no-source-maps

... but I think I can just do it by setting all three files as sources in package.json?  Reading through this https://parceljs.org/features/targets/ I can see that parcel is in fact * not * zero config, but it seems like it gives us what we want.  The claim is you can do stuff like this:

{
  "targets": {
    "frontend": {
      "source": "app/index.html"
    },
    "backend": {
      "source": "api/index.js"
    }
  }
}

And this:

{
  "targets": {
    "default": {
      "distDir": "./output"
    }
  }
}

Sooooo....

{
# ...
  "targets": {
    "httpsHandler": {
      "source": "src/lambda.ts",
      "distDir": "built"
    },
    "connectionsHandler": {
      "source": "src/connections/connectionHandlers.ts",
      "distDir": "built/connections"
    },
    "chatHandler": {
      "source": "src/chat/chatHandler.ts",
      "distDir": "built/chat"
    }
  },
  "scripts": {
    "build": "parcel build",
# ...
  },
# ... 
}

and then

yarn build
yarn run v1.22.17
$ parcel build
🚨 Build failed.

@parcel/core: Failed to resolve '@nestjs/microservices/microservices-module' from './node_modules/@nestjs/core/nest-application.js'

  /home/pg/repos/ohhell/packages/server/node_modules/@nestjs/core/nest-application.js:19:128
    18 | const { SocketModule } = optional_require_1.optionalRequire('@nestjs/websockets/socket-module', () => require('@nestjs/websocke
  > 19 | uire('@nestjs/microservices/microservices-module'));
  >    |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    20 | /**
    21 |  * @publicApi

@parcel/resolver-default: Cannot find module @nestjs/microservices
💡 Did you mean '@nestjs/core'?

I guess I'll add that as a dependency? ... and @nestjs/websockets/socket-module ? and @nestjs/platform-express ? and fastify-static ? and point-of-view ?

This is really weird.  I am fairly confident that all of this is going to get tree shaken out of the build.  Why does it want all this shit?  My code isn't even using the express adapter.

Oh neat, a thread in NestJS github from 2019 expressing consternation that someone would even use parcel on a nest project.
https://github.com/nestjs/nest/issues/3224

Also, bundling won't work with many other Node.js libraries (like mongodb or pg for postgres), so if you really want to bundle your code, you should have good knowledge & understanding on what's possible and what's not with these tools. Bundling front-end code is much easier because you never have to either access the file system dynamically or load native binaries. There are several workarounds to this issue, one of them is included in the NestJS CLI (when you use nest build), another one was shared here #1706 (comment). Using parcel doesn't make sense and I don't see any reason why we should keep tis issue opened.

I feel seen, and not in a good way.

Why am I bundling this indeed.  Let's review.  I am trying to use the same build tool across monorepo modules.  I definitely need a transpiler for my typescript code.

Basically, Nest in some cases is using require() calls lazily. For instance, if you don't use either RedisClient or RedisServer, the require('redis') expression won't be evaluated = the package won't have to be installed. This allows us to avoid creating 10 packages for every existing transport strategy just to put 2 classes in there. Also, if someone doesn't use redis strategy, he won't be forced to install redis.

Oh riiiiight, javascript is * garbage *.  Is this effort doomed?  

What I'm running into here is that Nest has a conditional runtime import strategy that means its code has dependencies undeclared in the npm packages.  Parcel is noticing this and reporting it as an error.  Sounds like NestJS CLI has its own build process. ... I don't really want to deal with that.  Well, what if I just keep importing all this stuff....?

yarn add class-transformer
yarn add @nestjs/platform-socket.io
yarn add class-validator
yarn add cache-manager
yarn add kafkajs # <---- that's a bad one
yarn add mqtt
yarn add @grpc/grpc-js
yarn add nats
yarn add redis # <----- lol
yarn add amqp-connection-manager
yarn add amqplib

And that's when the previous warning about a bad rxjs peer dependency morphed into this: @parcel/core: node_modules/rxjs/_esm5/index.js does not export 'lastValueFrom'

Also! There's an interesting warning that's been coming up this whole time during build but gets subsumed by the actual errors:

@parcel/transformer-js: Mutating process.env is not supported

  /home/pg/repos/ohhell/packages/server/node_modules/dotenv/lib/main.js:105:9
    104 |       if (!Object.prototype.hasOwnProperty.call(process.env, key)) {
  > 105 |         process.env[key] = parsed[key]
  >     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    106 |       } else if (debug) {
    107 |         log(`"${key}" is already defined in \`process.env\` and will not be overwritten`)

Ugh.  Maybe I should just start over in Deno, or Kotlin or something.

This is all so sketchy.  The parcel docs clearly indicate that you can use this on backend code, yet here we are with an incredibly common Node.js library raising flags because it mutates global process state.  Who is the villain here?  What have we done?  How the hell has this been working at all?

To review, I was outputting to built via tsc -b src. I was then running a deploy script in CICD that included this:

serverless deploy --verbose -c ./serverlessWSS.yml
serverless deploy --verbose -c ./serverlessHTTPS.yml

I believe serverless is doing babel and webpack behind the scenes.  Maybe not though?  I'm not using the webpack plugin. ... looking at AWS Console, my function code size is 19.6 megabytes.  That's ... bad.  That explains some cold start problems I've run into actually.

What are my options?

• see if I can fix the rxjs dependency issue, and also keep all 16 packages
• see if parcel has a less complain-y / correct mode
• rip out NestJS, and maybe dotenv
• use NestJS cli to bundle?
• see if there's a more compatible build mode with Parcel
• try building with swc directly
• add webpack plugin for serverless
• ... add parcel plugin for serverless?
• Give up, and just build with tsc without bundling

ugh.  I feel like if I can't get this to build with swc or parcel, I should get rid of Nest.  I'm not opposed to writing init code that wires my service classes together.  It's not even really doing that much for me in the controller layer.  If I'm being totally honest, this is all making me feel double-plus-ungood about building large scale node.js codebases.
... trying a few more things, jiggling dependencies... and I land here:

@parcel/optimizer-terser: Unexpected character '@'

  /home/pg/repos/ohhell/packages/server/src/chat/ChatService.ts:16:5
    15 |   constructor(
  > 16 |     @Inject('STORAGE') storage: Storage,
  >    |     ^ Unexpected character '@'
    17 |     @Inject('BROADCASTER') broadcaster: Broadcaster
    18 |   ) {

  💡 It's likely that Terser doesn't support this syntax yet.

Ok, so we're just not supporting decorators at all.  That's kind of wild. ... So yeah, this isn't going to work.

What was that about dotenv?

Let's go back to this error:

@parcel/transformer-js: Mutating process.env is not supported

  /home/pg/repos/ohhell/packages/server/node_modules/dotenv/lib/main.js:105:9
    104 |       if (!Object.prototype.hasOwnProperty.call(process.env, key)) {
  > 105 |         process.env[key] = parsed[key]
  >     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    106 |       } else if (debug) {
    107 |         log(`"${key}" is already defined in \`process.env\` and will not be overwritten`)

Yeah, so parcel has definitely drawn a line in the sand about not pulling in the entire ENV into node and making it accessible within the application.  They feel strongly that this is a security problem, because anyone in your npm dependency graph has access to it.  Instead, you can specify only these files for import into node:

.env
.env.{$NODE_ENV}
.env.{$NODE_ENV}.local

Then it pushes those into process.env as usual.  
That is... not how I want my environment variables to work.  I want to specify which env file to pull in based on a single environment variable, then go load that using dotenv.  First of all, $NODE_ENV is by convention restricted to develop, test, and production, but it's explicitly a bad practice to tie those to deployment stages.

There's good reason why I'm injecting this stuff into environment variables.  For staters, accessing secrets from a lambda using AWS Secret manager or systems configuration manager runs into throttling limits pretty quick.

So now what?  I'm coming away from this with a bad taste in my mouth about both nestjs and parcel, at least on the backend.

To be continued....