~/tutorials/021-a-chain-four-stages-deep.md

021: A chain four stages deep

Two stages was a pipe. Four stages is a pipeline, and pipelines fail in the middle where nobody is looking. This lesson builds one honestly, stage by stage, and then takes it apart when the answer looks wrong.

Build it one stage at a time

Stage one, the raw list. Every news file, names and titles and sizes:

ls -l ~/news

Stage two, keep only the rows that mention this month:

ls -l ~/news | grep 2026-09

Stage three, put it in order:

ls -l ~/news | grep 2026-09 | sort

Stage four, keep the smallest end of the order:

ls -l ~/news | grep 2026-09 | sort | head -3

Ten rows went in at stage two on the count of names, and three came out the end.  But the chain above greps the long listing, and a long row can mention the month in two places: the stamp on the left, and the file's name at the far end. The stamp says when this copy was placed on the disk, not when the story happened. Whether the stamp can match at all belongs to how your ls renders it: the default spells the month (Sep 21 17:18, and for old files Jan 15 2026), so a year pattern like 2026-09 can only ever match a name, and the long chain answers exactly what the short one does. Ask for the stamp in ISO (--time-style=long-iso) and the stamp column starts speaking the pattern's language: on the day this was written the same chain answered eleven, because August's one item rode in on its September placement stamp. A filter counts what the column means, not what you meant by it. That eleventh row is owned twice over, by the rendering flag and by the wall clock: move either one and the rider falls off. A count that a flag or a calendar can change is a property, not a number.

Which stage ate your result

Now the failure drill, which is the actual lesson. Suppose stage four had returned nothing. You have four suspects and one method: read the chain from the middle out, because a stage can only eat what the stage before it produced.

ls -l ~/news | grep 2026-09 | wc -l

Ten on the default ls, and eleven under the ISO stamp at the time of writing, for the reason above. The first two stages are innocent, so the fault is in three or four. sort emits every row it receives, and sorting cannot delete rows, so stage three is innocent by construction. That leaves head -3, which cannot return nothing from a column of rows. When every stage is innocent and the answer is still empty, the suspect is not in the chain at all: it is the pattern. grep 2026-09 matched nothing because the month rolled over, or the word was misspelled, and the stage that ate your result is the one you typed blind.

The method generalises: count at the joints. wc -l between any two stages tells you where the data died, and a pipeline debugged from the joints inward takes minutes. One debugged blind from the outside takes evenings.

A chain that answers a real question

Which of this month's items is the shortest read? Same chain, different tail:

ls -l ~/news | grep 2026-09 | sort | head -1

Read what sort actually did before you trust the answer. It orders whole lines as text, left to right, and every column before the timestamp settles first: permissions, owner, group, then the size field, which stands left of the date. On this tree the earlier columns never differ, so size is the first key the sort actually uses -- compared as text, where 1479 sorts before 1980, but 999 would sort after 1000. The date only gets a vote between rows whose size fields match to the character. The head -1 above is the true smallest file only because every size here runs to four digits. Say which column your sort keyed on before you quote its winner: a chain that sorts by size and a chain that sorts by date agree only when the digits cooperate, and the reader who cannot say which one they ran is the reader who reports the wrong file. That is what reading a pipeline is for.

Try it

Each of these has one answer, and the shell gives it to you. Check yourself.

Run the four stages as written and read the three survivors. What are they, in one sentence each?
The chain with wc -l replacing head answered eleven under the ISO stamp at the time of writing, ten on the default ls. Rebuild the chain to answer "how many items arrived in August" and run it. One at the time of writing.
Break the chain on purpose: change 2026-09 to a month with no items and run all four stages. Which stage reported the emptiness, and which one silently passed it on?
Build a different four-stage chain over dig l1ackers.com that ends in a count of mail servers. Four is still the count, and the chain is yours to choose.

Counts go stale as the news grows. The chain is the current truth, and it tells you so every time you run it.