Link: https://gitlab.com/pdfgrep/pdfgrep/-/work_items/14

(Apologies for publicly posting a link into Cloudflare’s walled-garden, but archive.org comes up empty.)

The linked bug report shows that astonishing but deliberate regular expression treatment was reported 9 years ago. Still today I spent time trying to work out why my regular expression was not matching only to find that the dev hi-jacked the meaning of ^ and $ for another purpose. Man page still neglects to warn users.

It makes sense that pdfs don't really have 'lines' in the same way.

If I say match a b, without the concept of a line you would be unable to even work out whether a comes immediately before a space which is immediately before b with nothing else in between. Lines are indispensible for determining sequence. Whether a pattern is a b or ^b, the tool necessarily must determine what, if anything, is to the left of b. The dev took a shortcut and skipped the effort of analysing vertical significance between tokens. The pdftotext tool proves that text lines can be determined from a searchable PDF.

In any case, docs that lie or omit something important are always a bug. Man page says: “pdfgrep tries to be mostly compatible with GNU grep with some PDF-specific distinctions and additional options. Most notably, -n prints page instead of line numbers.” No mention of changing the meaning of ^ and $.

You must just not have encountered a PDF that renders the page in a very strange way. The exact situation mentioned by the developer as an example in that report of a two-column PDF merging both columns into one line is very common. I don't have such a PDF at hand but in my experience this is particularly common for scanned documents that have OCR'd text content. I'm positive that pdftotext would trip up on those too, because what else could it do than show the text as it is logically laid out in the file?

If the functionality was different, people would just be complaining about a different incongruency.

You must just not have encountered a PDF that renders the page in a very strange way.

A scanned document is trivially rendered. So you must be referring to pure vector PDFs. With pure vector PDFs, the PDF generator is responsible for the contents. Indeed the PDF standard is about as shitty as the HTML standard and the quality of results are a product of the quality of the input. But in the case at hand, we are not really talking about managing poor quality input.

how pdftotext handles multi-column docs

The exact situation mentioned by the developer as an example in that report of a two-column PDF merging both columns into one line is very common. I don’t have such a PDF at hand but in my experience this is particularly common for scanned documents that have OCR’d text content. I’m positive that pdftotext would trip up on those too, because what else could it do than show the text as it is logically laid out in the file?

The pdftotext tool has a feature to handle 2 column docs. I often have docs with the left column in one language and the right column in another. Using pdftotext, we can specify a crop box that only extracts text landing in the left or right half of the page. In the absence of that feature, the left half of a line would be language 1 and the right half would be language 2 (merged). I don’t imagine that any user would expect ^ to match the beginning of the right-hand column. But given the docs as they are, users would expect the ^ to match the beginning of the line of the left-hand column.

the rule of least astonishment

If the functionality was different, people would just be complaining about a different incongruency.

Maybe but there is a principle to manage this: “the rule of least astonishment”. That is, software should be implemented to minimise astonishment. There could still be complaints if a problem is complex and difficult, but when the rule of least astonishment is followed it yields the least amount of complaints, which is not necessarily zero complaints. The case at hand seems to neglect the rule of least astonishment.

Yeah, it does add some confusion. It should be documented, or maybe the use of ^ and $ should be explicitly refused except by special flag, or something.