~/tutorials/054-what-a-scanner-can-know.md

054: What a scanner can know

The last three lessons built the instrument. This one begins the scanner trilogy, three lessons on what automation can and cannot honestly tell you about a web application, starting with the strongest case. What a scanner can know is what the target states about itself: its server software, its versions, its headers, its files that answer, its pages that exist. A scanner is a machine asking a very large number of very small questions, and the answers it can trust are only ever the ones the target itself supplied.

What this lesson is for

Calibration. Scanners are how most people meet web testing tooling, and the two failure modes are symmetrical: believing the scanner's output is the truth of the site, or dismissing it as noise because it is often wrong. Neither survives understanding what the scanner's knowledge is made of. Every fact a scanner reports came from a response the target sent, and the lesson of this tier's first five lessons, read the shipment, not the rendering, is exactly the discipline a scanner applies at machine speed.

What the reader meets

Two scanners, one target, real receipts. The target is the lab box from lesson 033, and the whole seat, target and scanners together, stands up from the apparatus page the web lab in a few commands. The addresses in this lesson's receipts, like 172.31.13.2, are that page's private docker addresses: minted at build time, printed by one command on the page, and different numbers on your machine. The first scanner is whatweb, a fingerprinter: one request, one read, everything the target volunteers in one response. The second is nikto, the classic web scanner: thousands of requests, a checklist census of what answers and what is missing.

Both run from the scanner seat of that page, the kaliscan container: a Kali image on the lab wire, installed once with apt-get install -y nikto whatweb, holding every scanner the tier quotes. On your machine every command below is prefixed docker exec kaliscan, and the address is your own build's, printed by the page's inspect command. In the receipts the prefix is dropped for width, which changes nothing about where the words run.

whatweb first, one line, aimed at the lab login page:

$ whatweb -a 3 http://172.31.13.2/login.php

http://172.31.13.2/login.php [200 OK] Apache[2.4.25], Cookies[PHPSESSID,security],
Country[RESERVED][ZZ], DVWA, HTTPServer[Debian Linux][Apache/2.4.25 (Debian)],
IP[172.31.13.2], PHP, PasswordField[password],
Title[Login :: Damn Vulnerable Web Application (DVWA) v1.10 *Development*]

Read that line as a stack of receipts, because every token in it is one. Apache and its version: the Server header, quoted verbatim, which you read by hand in lesson 047. The cookies: Set-Cookie lines from the response, the same lines lesson 049's census hunts for. PHP: the cookie name and header shapes carry the signature. DVWA and the full version, down to the development flag: the page title, volunteered in the HTML. The password field: the form's markup, shipped to anyone who asks. Not one of these facts required an exploit, a login, or even a second request. The target stated all of them, to a stranger, in one exchange, and whatweb did was collect the statements into a line.

That is the deepest truth about what scanners can know: the best case is perfect recall of the target's own testimony. It is the 049 census, the not-shown made systematic, with a signature library doing the reading.

The instrument

nikto now, the checklist census, and its receipt from the same lab:

$ nikto -h http://172.31.13.2/

+ Server: Apache/2.4.25 (Debian)
+ [013587] /: Suggested security header missing: content-security-policy
+ [013587] /: Suggested security header missing: strict-transport-security
+ [600050] Apache/2.4.25 appears to be outdated (current is at least 2.4.68)
+ [750500] /config/: Directory indexing found. See: CWE-548
+ [007224] /.gitignore: .gitignore file found. It is possible to grasp the directory structure
+ [006333] /login.php: Admin login page/section found
+ 8033 requests: 0 errors and 14 items reported on the remote host

The last line is the one to slow down on, because it is the scanner's own honest receipt about itself: eight thousand and thirty three requests produced fourteen findings, in a run of seconds. Fourteen findings from eight thousand questions is not a failure rate. It is the shape of scanning: nearly all questions are answered no, and the value is in the fourteen, provided you know what kind of yes each one is.

The three kinds of yes, which is the heart of this lesson. Read each finding and ask where its knowledge came from.

The target said so. The Server header, the version, the title, the directory indexes. These are facts, stated by the target, and the scanner's recall of them is trustworthy to exactly the degree the target is honest. A hardened site can lie in its headers, and a scanner will faithfully report the lie.

The target answered differently than expected. The missing headers: nikto knows what a well-configured site sends, noticed these absent, and reported the gap. This is real knowledge, but it is knowledge about a standard, not about this site's behavior. A missing CSP header is a true observation and a defensible finding, and the next lesson is where the limits of acting on it are taught.

The scanner guessed and something answered. The .gitignore and the config directory: nikto asked for paths from a list of interesting names, and the site answered 200. This is the 052 lesson wearing a wordlist, discovery by guessing, and every hit is exactly as meaningful as the fuzzer lesson will teach: a file exists. What it means, whether it is development leftover on a lab box or a live exposure on a production one, is a judgment the scanner does not make and you must.

Three kinds of yes, one discipline: for every finding, ask which kind it is, because the finding's trustworthiness is set by its source, not its severity label.

Try it

Run whatweb at aggression level 3 against your lab box, and then, by hand, reproduce three of its claims: the Server header with curl -I, the title with grep, one cookie with the 049 census line. You have just verified the scanner against your own eyes, which is the only calibration that sticks.
Run nikto against the same box and sort its findings into the three kinds of yes: said-so, answered-differently, guessed-and-answered. The sort is the skill. The scanner prints severity. The kind is what tells you whether to trust it, and no scanner prints that column.
Change one thing on the target and re-run both. If your lab box lets you edit its pages, change the title or remove a directory index. Watch whatweb and nikto report the new world, and note how fast machine-speed observation tracks a one-line change. That speed is the argument for scanners, and you just receipted it.

Reveal the answer

The scanner questions card, the reading discipline for any scanner output, with this lesson's receipts as the worked column:

For each finding, one line in your notes:
  WHAT stated it?      target header  /  difference from standard  /  a guess answered
  WHO verified it?     the scanner alone  /  you by hand
  WHAT KIND of yes?    said-so  /  answered-differently  /  guessed-and-answered

Worked, three findings from this lesson's nikto run:
  Apache/2.4.25 outdated   said-so, from the Server header, verifiable by hand in
                           one curl, and trustworthy exactly as far as the header is honest
  CSP header missing       answered-differently, true against a standard, an
                           observation about configuration, not about behavior
  /.gitignore found        guessed-and-answered, exists because a 200 came back,
                           meaning assigned by you, not the scanner

The card's law, worth writing once and keeping: a scanner does not find vulnerabilities. It finds statements, gaps, and answers, and attaches labels. The vulnerability, if there is one, is an inference a human makes from the finding's kind. Run this discipline over any scanner report, nikto, ZAP's, a paid cloud platform's, and the report transforms: the same list of rows becomes three short columns of different trustworthiness, and you suddenly know which rows deserve your afternoon.

And the speed receipt, the reason scanners earn their place in a workflow at all. This lesson's nikto run: eight thousand requests, sixteen seconds, fourteen findings. A human doing the same census by hand, one curl per path, is most of a day for the same fourteen rows. The scanner does not out-think you. It out-patients you, and the parts of testing that are patience are exactly the parts worth delegating, provided the reading discipline stays yours.

What this lesson does not claim

Everything a scanner cannot know is the next lesson's whole subject, but one boundary belongs here because it bounds even the best case. A scanner knows what the target stated, and a target can state falsely: the Server header can lie, the version can be patched behind the banner, the title can be anything. The scanner's knowledge of the said-so kind is perfect recall of testimony, not proof of the world behind it. And the scanner knows nothing at all about the things stated nowhere: the business logic, the data, the users, the code. It asked eight thousand questions and all of them were about the doorway. What the doorway opens onto is why the human is in the room.