JSON is everywhere on the Internet. Servers spend a lot of time parsing it. We need a fresh approach. The simdjson library uses commonly available SIMD instructions and microparallel algorithms to parse JSON 2.5x faster than anything else out there.

ncdu (NCurses Disk Usage) is a disk utility for Unix systems. Its name refers to its similar purpose to the du utility, but ncdu uses a text-based user interface under the [n]curses programming library.[2] Users can navigate the list using the arrow keys and delete files that are taking up too much space by pressing the 'd' key. Version 1.09 and later can export the file listing in JSON format.

Ecoji encodes data as 1024 emojis, its base1024 with an emoji character set. As a bonus, includes code to decode emojis to original data.

Using a terminal from 1976 as home automation hub, because why not?

Pricing constraints brought further limitations. The ZX Spectrum had no dedicated sprite generation ICs. This was in contrast to other emerging micro-computers of the period, such as the Commodore 64. The Spectrum’s Z80 CPU and ULA would do all the graphical heavy lifting. The ULA or Un-committed Logic Array is similar in function, but not in implementation to today’s CPLD’s where the work of many TTL logic chips is combined into one IC. The Spectrum’s minimal graphics processing hardware and constrained 16k of RAM led to Altwasser’s’ invention of an extremely efficient and somewhat esoteric display system.

The conceived design was unique, allowing Sinclair Research and Richard Altwasser to patent the implementation. The Spectrum set aside 6912 Bytes of RAM to be used for a display file and an attribute array. The display file holds 256×192 pixel data in 32×24 characters blocks, taking 6144 bytes of RAM, leaving 768 bytes for the attribute array. The attribute array stores colour information for each character block. One byte per block holds foreground and background colours, plus a blink value for on or off. This is an excellent space saving solution for displaying coloured text, though slightly limiting in advanced graphics usage.

Shuf is a Linux and Unix command-line utility that puts its input text in random order to generate output consisting of random permutations of the input. In simple terms, it just shuffles the input of either text file or standard input passed through the command line.

A new way to see and navigate directory trees

Delta provides language syntax-highlighting, within-line insertion/deletion detection, and restructured diff output for git on the command line.

A cat(1) clone with syntax highlighting and Git integration.

bat supports syntax highlighting for a large number of programming and markup languages
Git integration
Show non-printable characters
Automatic paging
File concatenation

I've now learned that grep can, halfway through grepping in a file, think
the file is suddenly binary and stop returning results.

xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files. Commands should be simple, fast and composable:

Simple tasks should be easy.
Performance trade offs should be exposed in the CLI interface.
Composition should not come at the expense of performance.

Ncdu is a command line tool to view and analyse disk space usage on linux. It can drill down into directories and report space used by individual directories. This way it is very easy to track down space consuming files/directories. It actually allows the user to do this much faster than even a gui file manager. On the server ofcourse gui tools are not present.

SSHHeatmap

Generates a heatmap of IPs that made failed SSH login attempts on linux systems, using /var/log/auth.log to get failed attempts. Uses the ipinfo.io library to fetch the IP address coordinates, and folium to generate the heatmap

The xpipe command reads input from stdin and splits it by the given number of bytes, lines, or if matching the given pattern. It then invokes the given utility repeatedly, feeding it the generated data chunks as input.

You can think of it as a Unix love-child of the split(1), tee(1), and xargs(1) commands.

It's usefulness might best be illustrated by an example. Suppose you have a file 'certs.pem' containing a number of x509 certificates in PEM format, and you wish to extract e.g., the subject and validity dates from each.

The openssl s_client(1) utility can only accept a single certificate at a time, so you'll have to first split the input into individual files containing exactly one cert, then repeatedly run the s_client(1) command against each file.

And, let's be honest, you probably have to google how to use sed(1) or awk(1) to extract subsequent blocks from a flip-flop pattern.

xpipe(1) can do the job for you in a single command:

Resource monitor that shows usage and stats for processor, memory, disks, network and processes.

Some commenters requested that we use our restored vintage 1930 Model 15 Teletype as a terminal for Linux. Hooking up a 5-bit Baudot mechanical contraption to a modern OS, even one that is terminal friendly, is not without some challenges: adapting to the non-standard high voltage 60 mA current loop, interfacing ASCII to the much smaller and different Baudot encoding, working in all caps, dealing with Baudot FIGS and LTRS modes, and making sure the computer doesn't overrun the pokey 45.5 bauds connection. But hey, Unix was developed on (much more modern 8-bit) teletypes, so that should still work, shouldn't it?

The first of McIlroy's dicta is often paraphrased as "do one thing and do it well", which is shortened from "Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new 'features.'"

McIlroy's example of this dictum is:

Surprising to outsiders is the fact that UNIX compilers produce no listings: printing can be done better and more flexibly by a separate program.

If you open up a manpage for ls on mac, you’ll see that it starts with

ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]

That is, the one-letter flags to ls include every lowercase letter except for {jvyz}, 14 uppercase letters, plus @ and 1. That’s 22 + 14 + 2 = 38 single-character options alone.

One of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc.

When working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool.

Once in a while a new program really surprises me. Reminiscing a while
ago, I came up with a list of eye-opening Unix gems. Only a couple of
these programs are indispensable or much used. What singles them out is
their originality. I cannot imagine myself inventing any of them.

Meld is a visual diff tool that makes it easier to compare and merge changes in files, directories, Git repos, and more.