Expectations come from the djot syntax reference and IR.hs rather than
from what the parser currently produces, so a failure is evidence of a
bug instead of a record of one. 55 properties: 32 inline (emphasis,
verbatim, math, raw, links, images, references, autolinks, spans,
attributes, footnotes, symbols, escapes, smart punctuation) and 23 block
(headings, quotes, breaks, code fences, divs, raw blocks, every ordered
marker style and delimiter, tight vs loose items, nesting, task lists,
definition lists, tables, definitions, block attributes, round trip).
Two rules are deliberately unasserted and noted in the module header:
headings of seven or more hashes, and the general verbatim space
stripping rule, of which only the unambiguous backtick case is tested.
Three deviations the suite found, fixed in Djot.hs:
- an unclosed verbatim span failed the construct back into plain text.
The reference says it "extends to the end of the text"; manyTill
cannot say that on its own, it needs the eof alternative.
- a task list checkbox was recognised without the space the reference
requires after it, so "1. [x](5)" ate a link and turned it into a
checked item containing a list starting at 5.
- checkboxes were recognised on ordered items, but the reference gives
them to bullet items only.
Test.Gen.Document is now shared by both suites. Syntax grew
quoteContent and listTypeFor for the two places markdown and djot
disagree about the resulting IR, and two djot facts became generator
invariants rather than assertions: adjacent markup sharing a delimiter
is ambiguous, so runs separate it, and a blank line between two lists
makes one loose list rather than two blocks, so documents never place
lists back to back.
The test suite is no longer markdown only, so it is just "tests" now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The suite is now one tasty binary over shared modules, so the Djot
tests can sit next to the Markdown ones without another copy of the
harness:
tests/Main.hs the TestTree
tests/Test/Harness.hs running a parser inside a property
tests/Test/Gen.hs generators, shrinkers, escaping
tests/Test/Gen/Document.hs whole document generation
tests/Markdown/Parse.hs the Markdown properties
Generated values moved from `pick` to `Arbitrary` instances on
newtypes. `pick` embeds its value with `forAll (return a)` and cannot
shrink; going through Arbitrary gets shrinking back at the cost of
hand written shrinkers, which have to preserve the invariants their
generator established.
IR now derives Eq, and the properties assert against a complete
expected tree rather than pattern matching. The old patterns bound
fresh names that shadowed the generated ones, so they only ever
checked the shape of the tree and never its content.
Test.Gen.Document generates a description of a document which renders
to source and derives its own expected tree, so the two stay in step
while shrinking. Rendering goes through a Syntax record so Djot can
reuse the spec and the expectations. Escaping is a generator level
concern: an Escaper says which characters are special and how (or
whether) they can be written literally.
Two properties fail, both on parser bugs rather than test bugs:
code_block fencedCodeBlock inverts its language test, so a
fence naming a language reports Nothing. Djot.hs
gets this right at src/Djot.hs:354.
document_round_trip a nested list with more than one item makes the
whole list fail to parse and fall back to a
paragraph. In listBlock's child parser the second
alternative has no `try`, so it fails having
consumed the first item's indentation. The
existing nesting tests only ever used one nested
item.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The test suite now runs on tasty + tasty-quickcheck. Translation is
mechanical: `property`/`forAll`/`lift` become
`monadicIO`/`pick`/`run`, `success`/`fail` become `pure ()`/`stop`
with a counterexample, and Hedgehog's ranged generators are replaced
by a `linear` helper that scales with QuickCheck's size parameter.
Shrinking is dropped since QuickCheck can't shrink these while keeping
the generated text within the ranges the assertions assume.
The expected-value patterns were also stale relative to the IR
(`LI`'s `child` field is gone, blocks carry a trailing `Attrs`, list
types carry a `style`), so they've been updated to match. The
structural checks are otherwise unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fills out src/Djot.hs, which was a skeleton of `error "todo"` stubs.
Three layers, so container prefixes and indentation never have to be
threaded through the parser state:
- line primitives (rawLine/peekLine/lineWhen/dedent): block parsers peek
at the next line, decide whether they want it, then consume it
- container blocks strip their prefix off the lines they own and re-parse
the result with subParse, a rank-2 helper that runs a polymorphic parser
at Identity and shifts error offsets back into the outer input
- inline content is a second pass over the text a block collected
Blocks: headings, paragraphs, block quotes, fenced code, raw blocks, divs,
tables (alignments, `^` captions, escaped pipes), footnote and reference
definitions, block attributes, thematic breaks, and lists -- bullet,
ordered (decimal/alpha/roman, `.`/`)`/`(n)`), task, and definition. Tight
lists unwrap their paragraphs to Transparent.
Inlines: emphasis and its braced forms, verbatim, raw inline, math, links,
reference links, images, autolinks, spans, footnote references, symbols,
escapes, hard breaks, attribute sets, and smart punctuation.
Two spots where the IR came up short:
- there is no reference image, so for `![alt][ref]` the label lands in
the url field
- unordered lists get style = Nothing rather than the bullet character,
which HTML.hs would otherwise emit as an invalid type="-"
Also includes the psb.cabal fix dropping OverloadedRecordUpdate, without
which Psb.Main does not build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lists and task lists are a PITA to handle, I may need to handle them
myself in the future but that'll probably become clear via testing
rather than simply doing it as is because the parsing logic is annoying
This commit gets bundling to the point where it can be used now
TODO:
- Get ESBuild Stdout/stderr to not pop up in psb output
- figure out how to split off map files so I'm not shunting them into
script tags
