Tokei: Quickly Count Different Metrics in Your Codebase

Once you have completed your assignment on a big software project (with or without a team), have you ever thought about how much code in different programming languages has been used in the project?

If the project is hosted on GitHub, you might catch a glimpse of the percentage of code used in different programming languages, but not everyone is going to put their codebase on GitHub.

What if there was a standalone, free, and open-source CLI utility that could easily identify and display the number of programming languages, files, total lines within those files, code comments, and blank lines in a well-structured manner?

Tokei is the tool I’m referring to, and what’s interesting is that it’s cross-platform, making it easy to install on Linux, Windows, and macOS.

• Cheat: Create a Cheatsheet for Your Favorite Command in Linux

If you’re wondering why it’s important to know the number of lines of code in a project or the various programming languages used, you’re not alone. After conducting some research, I’ve discovered the following reasons why understanding all the previously discussed details might be necessary:

• It assists developers (or project managers) in understanding the codebase’s composition and the distribution of code across different programming languages.
• If the developer (or project manager) needs to rewrite a specific part of the program in a different language for compatibility, performance, or feature enhancement reasons, they could approach this task strategically.
• Knowing the programming language also allows developers to quantify their contributions to the codebase.
• It also assists in compliance with licensing requirements and open-source software obligations by providing visibility into the use of third-party libraries and components.

There are many reasons why one might want to know the amount of code written in different programming languages on a particular software project, and for that purpose, they can use free tools like Tokei.

Tutorial Details

Features of Tokei

The following is a list of standout features offered by Tokei:

• It’s written in the Rust programming language, so speed isn’t a concern, as it can easily count millions of lines of code in seconds.
• The developer has clearly mentioned that this tool is very accurate and provides accurate code statistics, as it can easily handle multi-line comments, nested comments, and does not count comments that are in strings.
• The support includes 150 languages and their various extensions, as defined in the languages.json file.
• It can present the output in multiple formats, such as JSON, YAML, and CBOR, allowing developers to easily store and reuse it for comparison.
• Tokei also acts as a library, so one can easily integrate it with other projects.

• jnv: An Interactive JSON Viewer and jq Filter Editor for Linux

How to Install Tokei

The installation of Tokei is quite easy, as it is mostly available for installation from the default package manager in most Linux distributions with one of the following commands:

# Alpine Linux (since 3.13)
$ apk add tokei

# Arch Linux
$ pacman -S tokei

# Cargo
$ cargo install tokei

# Conda
$ conda install -c conda-forge tokei

# Fedora
$ sudo dnf install tokei

# FreeBSD
$ pkg install tokei

# NetBSD
$ pkgin install tokei

# Nix/NixOS
$ nix-env -i tokei

# OpenSUSE
$ sudo zypper install tokei

# Void Linux
$ sudo xbps-install tokei

However, if you are using a distribution other than the supported one mentioned above, you can install the Brew package manager and use it to install, which will also work for macOS.

$ brew install tokei

Lastly, if you’re using Windows, you can easily install it with the following command:

# Winget
$ winget install XAMPPRocky.tokei

# Scoop
$ scoop install tokei

• Zellij: Modern Drop-in Replacement for Tmux Command on Linux

How to Use Tokei

Once the installation is complete, the “tokei” command becomes accessible; you can navigate to your project directory and specify one or more project folders to get an estimation of the number of files, programming languages used, and other previously discussed metrics.

# It will only report the code in proj1/ and all subfolders.
$ tokei proj1/

# It will only report the code in proj1/, proj2/, proj3/ and all subfolders.
$ tokei proj1/ proj2/ proj3/

For demonstration, I have an “Algorithms” directory containing multiple files written in Python, so to get estimates of various metrics, we can use the following command:

$ tokei Algorithms/

Output:

As you can see, it has given us the number of programming languages used, the number of files, lines of code, comments, and blank spaces.

In certain situations, you’ll want to ignore specific files; for instance, in the output, we have the “Shell” and “Plain Text” results that I wish to ignore. Therefore, we can use either the “-e” or “--exclude” option to exclude these additional files.

$ tokei Algorithms/ -e *.sh -e *.txt

Output:

• Bat is a Modern Drop-in Replacement for Cat Command on Linux

The default behavior of Tokei is to only provide the output for the total languages, but you can use the “--files” option to output individual file statistics.

$ tokei --files

Output:

Tokei sorts the output alphabetically by language name by default, but you can use different columns like “blanks“, “code“, “comments“, and “lines” for sorting the output using the “--sort” option.

$ tokei --sort code

Output:

• Chezmoi: Manage Your Dotfiles Across Multiple Linux Systems

Finally, to present the output in different supported formats, such as JSON, YAML, and CBOR, you can use the “--output” option, followed by the supported format name in lowercase.

$ tokei --output yaml

Output:

That’s it; here we’ve discussed the important aspects of Tokei, but if you want to dig deeper, you can check its help page by typing the “tokei --help” command.

Final Word

Tokei is an interesting tool that could be useful for those who are very particular about inspecting the use of programming languages, lines, comments, and blanks in their codebase. If you are one of them, then do let me know in the comment section, including your use case.

Till then, peace!