Update TypeScript using ClojureScript and nbb

Just for kicks, what if you could process TypeScript files using ClojureScript and have the power of a REPL at your fingertips while figuring out what you want to do?

Painting the picture

A unique scenario put me on the path of trying to find a better way than "search and replace", because we all know how well that turns out on large code bases. Also, the project where it's being used is of such a scale that it's not feasible to update by hand, as it may have to be done more than once.

I'm not the biggest TypeScript fan because I've been scarred by unintended complexity in code and not to even mention the config hell that is bound to manifest in Node.js projects.

But here we are, mixing tech that sparks joy (Clojure, ClojureScript), with tech that we can't get away from (TypeScript).

The problem at hand is not so obvious, but a journey around figuring out differences, similarities, inner workings and recipes around CommonJS and EcmaScript Modules (ESM) sent me down the rabbit hole and finding improved guidelines that will improve code posture in a large project.

There is too much to cover in this post, but one particular part points to using imports in your TypeScript with or without .js extensions. It is just weird to code TypeScript without any extension in your import declarations. But along the way, I discovered some challenges with Node.js and NOT using .js in imports.

When running a script, it would complain about not finding a module:

Error [ERR_MODULE_NOT_FOUND]: Cannot find module \
  '/.../abc-model/dist/somenamespace/SomeFile' \
  imported from /.../abc-model/dist/app.js

Note that this code is the result of compiling TypeScript using tsc only. No bundler involved. When the tsc output is processed by a bundler, it will likely work without issue. However, the aim is to get rid of complexity of multiple workspaces that each produce bundles, and let the workspaces only produce output generated by tsc and defer bundling involvement until it's needed.

Trying to figure out how to resolve this particular problem, a StackOverflow post crossed my path: Appending .js extension on relative import statements during Typescript compilation (ES6 modules)

Further investigations pointed that the module specifier in an import statement is a runtime location, so using a .js extension in your TypeScript file feels weird, but it is perfectly valid. This even happens in the typescript API itself.

Peek at src/typescript/_namespaces/ts.ts, for example:

export * from "../../compiler/_namespaces/ts.js";
export * from "../../jsTyping/_namespaces/ts.js";
export * from "../../services/_namespaces/ts.js";
export * from "../../server/_namespaces/ts.js";
import * as server from "./ts.server.js";
export { server };

So going with this acceptable methodology, my next aim is to update over 100 files, but find and replace won't cut it. So let's see what TypeScript has to offer.

TypeScript API

Having some nice experiences in the Clojure world includes being able to make sense from your source file by reading and processing it. Maybe something like that exists for TypeScript... Lo and behold, the TypeScript API. It took several tries to wrap my head around it: https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API

There are a couple of great examples of how to process and/or analyze a TypeScript source file. I needed only a very limited subset.

I only need to be able to get something from the source that tells me it is an import statement, and then I can do something with it.

The gist of what I needed is:

• Get the elements I'm interested in, and see what I can do with them
• If any updates are needed, figure out what they are
• Apply them to the source, but don't mess with the original source until I'm happy
• Once "rendered" updates appear to be in order, apply the changes to the source and write to disk.

nbb

Of course all of what I'm trying to do would be possible in only TypeScript and tsc output, but where is the fun in changing a file, modifying and running a script, again and again and again?

Setting up a whole ClojureScript environment and project would be overkill. So having heard about nbb and playing around with it before, I thought it would be a great opportunity to see whether my new knowledge around Node.js, tsc, CommonJS, EcmaScript Modules and the TypeScript API can be used in nbb!

Not babashka. Node.js babashka!?

Ad-hoc CLJS scripting on Node.js.

Babashka is another great project I can seriously recommend.

Bringing it together

In summary, what I needed to do is such that:

• in:

  // source
  import { A, B } from './my/ns/file'
  

  • the moduleSpecifier is './my/ns/file'

• and the moduleSpecifier must be updated so that the import looks like:

  // target
  import { A, B } from './my/ns/file.js'
  

Fiddling around in VSCode figuring out what's available in TypeScript, and running samples in the Calva REPL, the solution came down to a naive solution of:

• read source file,
• find ImportDeclarations and
• return a collection of updates to apply
• render the applied updates (for debugging/testing, before committing changes to the actual file)
• apply the rendition of the updated file

And roughly in Clojure(Script), it looks like:

(-> (path/join (js/process.cwd) file) ;; (file) => SourceFile
     determine-updates                ;; (SourceFile) => {:file :updates}
     render-updates                   ;; ({:file :updates}) => {:file :updates :rendered}
     apply-rendered-updates!)         ;; ({:file :updates :rendered}) => nil

In the code above, the -> is called the thread-first macro, meaning that for any subsequent expressions, the result of the previous is passed in as the first argument to the current function.

The functions' input and output is also carefully considered so that it will work in the -> macro.

Full source:

(ns update-import-module-specifiers
  (:require
   ["fs" :as fs]
   ["path" :as path]
   ["typescript" :as ts]
   [clojure.string :as str]))


(defn syntax-kind
  "Determine passed in `node`'s `SyntaxKind`.
   
   Node.kind is a numeric value.
   Used to look up type from `SyntaxKind` mapping (type: {n: string})
   
   Returns as a Clojure keyword"
  [node]
  (keyword (aget ts/SyntaxKind (.-kind node))))


(defn source-file-from
  "Reads `typescript` `SourceFile` using `typescript` API.
   
   Returns `SourceFile`"
  [file-path]
  (let [filename (path/basename file-path)]
    (ts/createSourceFile 
      filename
      (.toString (fs/readFileSync file-path))
      ts/ScriptTarget.ES2020 true)))


(defn update-module-specifier?
  "Predicate of local module-specifier ending without `.js`"
  [module-specifier]
  (let [text (-> (.getText module-specifier)
                 (str/replace #"'" "")
                 (str/replace #"\"" ""))]
    (and (str/starts-with? text ".")
         (not (str/ends-with? text ".js")))))

(defn get-import-declaration-update 
  [statement]
  (when-not (= :ImportDeclaration (syntax-kind statement))
    (throw (js/Error. "Invalid syntax kind. Expected :ImportDeclaration")))
  
  (let [module-specifier (.-moduleSpecifier statement)]
    {(syntax-kind statement)
     {:update/op
      (when (update-module-specifier? module-specifier)
        (let [text (-> (.getText module-specifier)
                       (str/replace #"'" "")
                       (str/replace #"\"" ""))
              from (str "from " (.getText module-specifier))
              to (str "from '" text ".js'")]
          {:from from :to to
           :op (fn [s] (str/replace s from to))}))}}))


(defn determine-updates
  "Determines any updates for the given `file`.
   
   returns {:file :updates}"
  [file]
  (let [source-file (source-file-from file)
        supported-updates
        {:ImportDeclaration #'get-import-declaration-update}]
    (try
      {:file file
       :updates
       (->> (.-statements source-file)
            (map (fn [statement]
                   (when-let [get-update ((syntax-kind statement) supported-updates)]
                     (get-update statement))))
            (filter identity)
            (into []))}
      (catch js/Error e
        (println "error processing" file)
        (throw e)))))


(defn render-updates 
  "Takes `determined-updates` which is result of passing `file` to `determine-updates`.
   
   returns {:file :updates :rendered}"
  [{:keys [file updates] :as _determined-updates}]
  (let [source-file (source-file-from file)]
    {:file file
     :updates updates
     :rendered
     (reduce
      (fn [acc next-update]
        (cond (not (-> next-update :ImportDeclaration :update/op))
              acc
              :else
              (let [{{:update/keys [op]} :ImportDeclaration} next-update]
                ((:op op) acc))))
      (.getFullText source-file)
      updates)}))

(defn apply-rendered-updates!
  [{:keys [file updates rendered]}]
  (cond (seq updates)
        (fs/writeFileSync file rendered)

        :else
        (println :NO_UPDATE (path/basename file))))

(defn -main
  "run: in {project}:
   
   # fish shell:
      npx nbb ~/workspace/other/sandbox/nbb/update_import_module_specifiers.cljs (find src -name '*.ts')
   "
  []

  (let [{:keys [files cwd]}
        {:files (drop 3 js/process.argv) ;drop nodejs stuff for files to process
         :cwd (js/process.cwd)}]
    (doseq [file files]
      (-> (path/join cwd file)
          determine-updates
          render-updates
          apply-rendered-updates!))))

(-main)

This seems like overkill, and it probably is, but I've learned plenty about nbb's capabilities and ClojureScript interop in the context of Node.js. Many light-bulb moments in all the tech involved.