Literate Programming 7

Prev:WEB 6 Top:WEB 0 Next:WEB 8 @*WEB 7 - Quality of Life Improvements.

Jon Breuer - September 10, 2024. @*__table_of_contents__. @* Existing features.

@ @^ New Features:@> @* Desired features. @ I used WEB4 to write documentation for my Joy interpreter, and I hit a few rough points I'd like to improve. @*Including previous code. Of this wishlist, including web4's source is the priority. I'll have to hack my changes into it to get started. @ In asdf, I'll insert a new handler for @@# which will be my include/pragma/define tag. My extensions will lie under that. @= void parse_web_then_tangle_and_weave(...) { ... while(charIndex < fileContents.length) { dchar ch = fileContents[charIndex]; if(ch == '@@') { if(chNext == '@@') { // Escaped At } else if(chNext == 'p') { // Program Start } else if(chNext == '>') { // End Code Section } else if(chNext == '<') { // Start Code Section } else if(chNext == '*') { // Headers } else if (chNext == '#') { // A pragma section will instruct the parser. @ } else { // Anything else is documentation. } } else { // Anything else is documentation. } } ... } @> @ I'm imagining the pragmas to look like @@ #include @@ > or @@ #define new type @@ >. Read the entire pragma command header, then determine what to do with it. @= @ @ @ The existing |slurp_section| function is already built to read the whole section. I'll use that. @= SBlock[] pragmaBlocks = slurp_section(fileContents, charIndex, lineNumber, inputFilename, false, ESectionType.IDENTIFIER); assert(pragmaBlocks.length == 1); string command = pragmaBlocks[0].content; @ Include is the important one. @= if(command.startsWith("include ")) { @ @ @ @ @ } @ I'll treat the rest of the identifier as the filename. no quotes to avoid. @= string includeFilename = command[8..$]; @ This is the same command I use in |main|() to read the primary input file. @= string includeFileContents; try { includeFileContents = cast(string) std.file.read(includeFilename); } catch(Exception err) { writefln("Error: %s", err.msg); return []; } @ Use the existing |parse_web| function. @= string parsedIncludeDisplay= ""; string parsedIncludeCode= ""; SSection[] includedSections = parse_web( parsedIncludeDisplay, parsedIncludeCode, includeFileContents, includeFilename); @ I'm torn, but I don't want all the discussion from the previous file. The writer should include a hyperlink back to the previous document, but since I'm parsing the source, I don't know what the published name will be. @= fileSections ~= SSection("Included " ~ includeFilename, ESectionType.PARAGRAPH, [SBlock(ESectionType.PARAGRAPH, lineNumber, "Included " ~ includeFilename)]); @ The whole point. Insert the code here. @= //writefln("%d sections parsed from %s", includedSections.length, includeFilename); //writefln("%s", includedSections[0..15]); //SSection[] filteredSections = //std.array.array(includedSections.filter!(section=>section.type == ESectionType.CODE)); //writefln("%d filtered sections.", includedSections.length); //fileSections ~= filteredSections; auto filtered = includedSections.filter!(section=>section.type == ESectionType.CODE); foreach(SSection section; filtered) { fileSections ~= section; } @ Skip code included from literate_programming_4_source
@= import std.array; @= // This is new code for Web7. @ Huh. I printed the internal representation and the types aren't what I expect.
    SSection("", PARAGRAPH, [
        SBlock(PARAGRAPH, 18, "

"), SBlock(CODE, 16, "I really want to redefine existing sections and dynamically add content into group sections.\r\n\r\n")])

@*Escaping @s. Did some debugging. The problem isn't in the parser continuing to read the second at as a command, but in leaving stray angle brackets <content> in an HTML file. Because while my code is escaped, my content isn't. Hmm. This is a deeper design issue. I want @<Token@> to render and <b> to bold. @ The program is doing the right thing. Can I add a warning message that I might be doing the wrong thing? @@<tag> won't render correctly because HTML thinks it's a tag. < hello there> @= SSection[] parse_web(ref string outputDisplayContents, ref string outputCodeContents, string fileContents, string inputFilename) { ... while(charIndex < fileContents.length) { dchar ch = fileContents[charIndex]; if(ch == '@@') { dchar chNext = charIndex < fileContents.length - 1 ? fileContents[charIndex + 1] : 0; if(chNext == '@@') { // It's just an escaped at. Continue parsing. @ Let's insert an extra check for raw less than that may be interpreted as HTML tags. @= if(charIndex < fileContents.length - 1 && fileContents[charIndex] == '<' && fileContents[charIndex + 1] != ' ') { // I think the writer wrote at, at, lessthan to try and escape the at, but the lessthan will be problematic in HTML. writefln("Warning: Using %s%stag in an HTML file may have unintended effects. Near line %d in %s.", CHAR_AT, '<', lineNumber, inputFilename); } @ And we need the same fix inside slurp_section. @= if(index < contents.length - 1 && contents[index+1] == '<' && contents[index + 2] != ' ') { // I think the writer wrote at, at, lessthan to try and escape the at, but the lessthan will be problematic in HTML. writefln("Warning: Using %s%stag in an HTML file may have unintended effects. Near line %d in %s.", CHAR_AT, '<', lineNumber, inputFilename); } @ Let's test this : @@<an error@@>? - It works.
Warning: Using @@<tag in an HTML file may have unintended effects.  Near line 207
@*Custom Formatters. This implementation can do bold text, italic text, and bold italic terms. I need an inline code style and maybe another. I want to define a style and be able to apply it. @
@@#define A=<b><i>content</i></b>
...
@@ This is a paragraph with @@Asome styling@@>.
@ would yield : This is a paragraph with some styling. @ In |New Display Work in parse_web_then_tangle_and_weave| is this: @= foreach(SSection section; fileSections) { if(section.type != ESectionType.CODE) { string paragraphReducer(string output, SBlock block) { if(block.type == ESectionType.INDEX_TERM) { return output ~ "" ~ block.content.strip("|") ~ ""; } else if(block.type == ESectionType.BOLD) { return output ~ "" ~ block.content ~ ""; } else if(block.type == ESectionType.PRE) { return output ~ "" ~ block.content ~ ""; @ I'll hack WEB4 so I can improve those tags. @ Before BOLD and PRE were separate types, I'm going to fold them into just CUSTOM. @
= CUSTOM, @ The SBlock type now can have the type be CUSTOM and the exact formatter specified here. @= string customFormat; @ SFormat is a new type which will define the tags. @= struct SFormat{ string pre; string post; }; @ @= SFormat[string] Custom_formats; @ To preserve backward compatibility, bold and italic are defined as custom formatters. @= Custom_formats["^"]=SFormat("", ""); Custom_formats["."]=SFormat("", ""); @ The existing |Detect a new text style| has hard-coded values: @= } else if(contents[index + 1] == '^' || contents[index + 1] == '.') { ESectionType styleType = contents[index + 1] == '^' ? ESectionType.BOLD : ESectionType.PRE; @ and our new version will detect a registered formatter and use that. @= } else if("" ~ contents[index + 1] in Custom_formats) { ESectionType styleType = ESectionType.CUSTOM; string customFormat = "" ~ contents[index + 1]; @ And note the format on the block of text. @= results ~= SBlock(styleType, lineNumber, textBlock, customFormat); @ When styling the text, look up the formatter and apply it. @