If you’re trying to learn Docker you will first have to master its various terminal commands. This guide aims to help you get started with basic docker commands.

systemd has become a mainstay for the Linux world, but one of the things that still seems to stick around is cron jobs. It’s understandable, as cron is a tool that we have been using for a long time. Change is hard, but I think systemd Timers make the change well worth it. Here are a few reasons why…

A large majority of computer systems have some state and are likely to depend on a storage system. My knowledge on databases accumulated over time, but along the way our design mistakes caused data loss and outages. In data-heavy systems, databases are at the core of system design goals and tradeoffs. Even though it is impossible to ignore how databases work, the problems that application developers foresee and experience will often be just the tip of the iceberg. In this series, I’m sharing a few insights I specifically found useful for developers who are not specialized in this domain.

I've now learned that grep can, halfway through grepping in a file, think
the file is suddenly binary and stop returning results.

xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files. Commands should be simple, fast and composable:

Simple tasks should be easy.
Performance trade offs should be exposed in the CLI interface.
Composition should not come at the expense of performance.

On any given day, we handle around 15% of daily retail trading volume across all stock exchanges in India. Billions of requests generated in the process are handled by a suite of systems we have built in-house. Also, we are very particular on self-hosting as many dependencies as possible, everything from CRMs to large databases, Kafka clusters, mail servers etc.

To aid these primary systems, there are a large number of ancillary workloads that run, covering everything from real-time trades, document processing, KYC, and account opening, legal and compliance, complex, large scale P&L and number crunching, and a wide range of backoffice workloads. The systems are spread across a hybrid setup; physical racks across two different data centres (where exchange leased lines terminate) and AWS. All of this means that we have a lot of dynamic workloads and dissimilar systems and environments, bare metal to Kubernetes clusters, to be monitored independently.

The first and second open source migration waves were periods of rapid expansion for companies that rose up to provide commercial assurances for Linux and the open source databases, like Red Hat, MongoDB, and Cloudera. Or platforms that made it easier to host open source workloads in a reliable, consistent, and flexible manner via the cloud, like Amazon Web Services, Google Cloud, and Microsoft Azure.

This trend will continue in the third wave of open source migration, as organizations interested in reducing cost without sacrificing development speed will look to migrate more of their applications to open source. They’ll need a new breed of vendor—akin to Red Hat or AWS—to provide the commercial assurances they need to do it safely.

I’ve been writing about running Docker on Raspberry Pi for five years now and things have got a lot easier than when I started back in the day. There’s now no need to patch the kernel, use a bespoke OS, or even build Go and Docker from scratch.

Ncdu is a command line tool to view and analyse disk space usage on linux. It can drill down into directories and report space used by individual directories. This way it is very easy to track down space consuming files/directories. It actually allows the user to do this much faster than even a gui file manager. On the server ofcourse gui tools are not present.

The decision in 2017 to move back to a monolith considered all the trade-offs, including being comfortable with losing the benefits of microservices. The resulting architecture, named Centrifuge, is able to handle billions of messages per day sent to dozens of public APIs. There is now a single code repository, and all destination workers use the same version of the shared library. The larger worker is better able to handle spikes in load. Adding new destinations no longer adds operational overhead, and deployments only take minutes. Most important for the business, they were able to start building new products again. The team felt all these benefits were worth the reduced modularity, environmental isolation, and visibility that came for free with microservices.

SSHHeatmap

Generates a heatmap of IPs that made failed SSH login attempts on linux systems, using /var/log/auth.log to get failed attempts. Uses the ipinfo.io library to fetch the IP address coordinates, and folium to generate the heatmap

The xpipe command reads input from stdin and splits it by the given number of bytes, lines, or if matching the given pattern. It then invokes the given utility repeatedly, feeding it the generated data chunks as input.

You can think of it as a Unix love-child of the split(1), tee(1), and xargs(1) commands.

It's usefulness might best be illustrated by an example. Suppose you have a file 'certs.pem' containing a number of x509 certificates in PEM format, and you wish to extract e.g., the subject and validity dates from each.

The openssl s_client(1) utility can only accept a single certificate at a time, so you'll have to first split the input into individual files containing exactly one cert, then repeatedly run the s_client(1) command against each file.

And, let's be honest, you probably have to google how to use sed(1) or awk(1) to extract subsequent blocks from a flip-flop pattern.

xpipe(1) can do the job for you in a single command:

In this letter, the minister agrees to the principle of Free Software by default ("Open Source by default") for procurement, which can be considered a parallel to the 'comply or explain' policy that is already in effect for the adoption of open standards. The minister also agrees to the government actively developing and publishing Free Software.

As part two (see previous attempt) of my ongoing series in ‘computational necromancy,’ I’ve spent the last year and a half or so constructing my own 1/10-scale, binary-compatible, cycle-accurate Cray-1. This project falls purely into the “because I can!” category – I was poking around the internet one day looking for a Cray emulator and came up dry, so I decided to do something about it. Luckily, the Cray-1 hardware reference manual turned out to be useful enough that implementing most of this was pretty straightforward. The Cray-1 is one of those iconic machines that just makes you say “Now that‘s a super computer!” Sure, your iPhone is 10X faster, and it’s completely useless to own one, but admit it . . you really want one, don’t you?

Play Fortress battles from the comfort of your home

"Protesting is a non-essential activity." That's what the police said.

Humans are complex beings, and we are capable of holding more than two thoughts in our minds at once. That's how I'm able to recognize that these protestors were, at best, the unwitting patsies of a bigoted death cult — while also acknowledging that a police department deeming protest as "non-essential" is, at best, dumb, and at worst, utterly horrifying.

Resource monitor that shows usage and stats for processor, memory, disks, network and processes.

What Should the US Federal Government and the States Do to Fight the Coronavirus

Die Novelle soll am Dienstag im Nationalrat beschlossen werden. Die Änderung des Epidemiegesetzes, die ohne Ankündigung im Gesundheitsausschuss von ÖVP und Grünen beschlossen wurde, schlug zuletzt hohen Wellen. Die Opposition kritisierte, dass die Bestimmung, die künftig erlaubt, Veranstaltungen auf „bestimmte Personengruppen“ zu beschränken, bedeute: Andere Personengruppen, etwa CoV-Risikogruppen und Personen, die keine App installiert haben, werden ausgeschlossen.