@yozora/tokenizer-paragraph
A sequence of non-blank lines that cannot be interpreted as other kinds of blocks forms a paragraph. The contents of the paragraph are the result of parsing the paragraph’s raw content as inlines. The paragraph’s raw content is formed by concatenating the lines and removing initial and final whitespace.
- See github flavor markdown spec for details.
- See Live Examples for an intuitive impression.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/tokenizer-paragraph
yarn add @yozora/tokenizer-paragraph
pnpm add @yozora/tokenizer-paragraph
Usage
@yozora/tokenizer-paragraph has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm,
so you can use YozoraParser
/ GfmExParser
/ GfmParser
directly.
- Basic Usage
- YozoraParser
- GfmParser
- GfmExParser
@yozora/tokenizer-paragraph cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.
import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
const parser = new DefaultYastParser()
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
.useTokenizer(new ParagraphTokenizer())
// parse source markdown content
parser.parse(`
aaa
bbb
`)
import YozoraParser from '@yozora/parser'
const parser = new YozoraParser()
// parse source markdown content
parser.parse(`
aaa
bbb
`)
import GfmParser from '@yozora/parser-gfm'
const parser = new GfmParser()
// parse source markdown content
parser.parse(`
aaa
bbb
`)
import GfmExParser from '@yozora/parser-gfm-ex'
const parser = new GfmExParser()
// parse source markdown content
parser.parse(`
aaa
bbb
`)
Options
Name | Type | Required | Default |
---|---|---|---|
name | string | false | "@yozora/tokenizer-paragraph" |
priority | number | false | TokenizerPriority.FALLBACK |
-
name
: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase. -
priority
: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in thematch-block
stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.
Types
@yozora/tokenizer-paragraph produce Paragraph type nodes. See @yozora/ast for full base types.
import type { YastParent } from '@yozora/ast'
export const ParagraphType = 'paragraph'
export type ParagraphType = typeof ParagraphType
/**
* Paragraph represents a unit of discourse dealing with a particular
* point or idea.
* @see https://github.com/syntax-tree/mdast#paragraph
* @see https://github.github.com/gfm/#paragraphs
*/
export type Paragraph = YastParent<ParagraphType>
Live Examples
-
Basic.
-
Paragraphs can contain multiple lines, but no blank lines.
-
Multiple blank lines between paragraph have no effect.
-
Leading spaces are skipped.
-
Lines after the first may be indented any amount, since indented code blocks cannot interrupt paragraphs.
-
However, the first line may be indented at most three spaces, or an indented code block will be triggered.
-
Final spaces are stripped before inline parsing, so a paragraph that ends with two or more spaces will not end with a hard line break.