@yozora/core-parser
@yozora/core-parser provides a plug-in system that can load tokenizer for collaborative processing in the lifecycles.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/core-parser
yarn add @yozora/core-parser
pnpm add @yozora/core-parser
Usage
@yozora/core-parser provides a default YastParser
implementation
DefaultYastParser.
import { DefaultYastParser } from '@yozora/core-parser'
import EmphasisTokenizer from '@yozora/tokenizer-emphasis'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
const parser = new DefaultYastParser({ shouldReservePosition: true })
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
// add custom tokenizers through `useTokenizer()`
.useTokenizer(new EmphasisTokenizer())
// Parse content
parser.parse('source content')
// Specify the content range
parser.parse('source content', 0, 10)
// Remove the positions from the YastNode.
parser.parse('source content', undefined, undefined, false)
// Reserve the positions
parser.parse('source content', undefined, undefined, true)
Options
Name | Required | Default | Type |
---|---|---|---|
blockFallbackTokenizer | false | null | BlockFallbackTokenizer | null |
inlineFallbackTokenizer | false | null | InlineFallbackTokenizer | null |
lazinessTypes | false | See below | string[] |
shouldReservePosition | false | false | boolean |
-
blockFallbackTokenizer
: Fallback tokenizer on processing block structure phase. -
inlineFallbackTokenizer
: Fallback tokenizer on processing inline structure phase. -
lazinessTypes
: Types of YastNode which could has laziness contents.- Default: result of
[PhrasingContentType].concat(blockFallbackTokenizer.uniqueTypes)
.
- Default: result of
-
shouldReservePosition
: Whether it is necessary to reserve the position of the YastNode parsed.
Lifecycle
match-block
See TokenizerMatchBlockHook for Details.
Name | Required | Type |
---|---|---|
isContainerBlock | true | boolean |
interruptableTypes | true | string[] |
eatOpener | true | <Function> |
eatAndInterruptPreviousSibling | false | <Function> |
eatContinuationText | false | <Function> |
eatLazyContinuationText | false | <Function> |
onClose | false | <Function> |
extractPhrasingContentLines | false | <Function> |
buildBlockState | false | <Function> |
-
Examples
- @yozora/tokenizer-admonition
- @yozora/tokenizer-blockquote
- @yozora/tokenizer-fenced-code
- @yozora/tokenizer-heading
- @yozora/tokenizer-html-block
- @yozora/tokenizer-indented-code
- @yozora/tokenizer-definition
- @yozora/tokenizer-list-item
- @yozora/tokenizer-math
- @yozora/tokenizer-paragraph
- @yozora/tokenizer-setext-heading
- @yozora/tokenizer-table
- @yozora/tokenizer-thematic-break
post-match-block
See TokenizerPostMatchBlockHook for Details.
Name | Required | Type |
---|---|---|
transformMatch | true | <Function> |
-
Examples
parse-block
See TokenizerParseBlockHook for Details.
Name | Required | Type |
---|---|---|
parseBlock | true | <Function> |
parseMeta | false | <Function> |
-
Examples
- @yozora/tokenizer-admonition
- @yozora/tokenizer-blockquote
- @yozora/tokenizer-fenced-code
- @yozora/tokenizer-heading
- @yozora/tokenizer-html-block
- @yozora/tokenizer-indented-code
- @yozora/tokenizer-definition
- @yozora/tokenizer-list-item
- @yozora/tokenizer-list
- @yozora/tokenizer-math
- @yozora/tokenizer-paragraph
- @yozora/tokenizer-setext-heading
- @yozora/tokenizer-table
- @yozora/tokenizer-thematic-break
match-inline
See TokenizerMatchInlineHook for Details.
Name | Required | Type |
---|---|---|
priority | number | <Function> |
delimiterGroup | string | <Function> |
findDelimiter | true | <Function> |
isDelimiterPair | false | <Function> |
processDelimiterPair | false | <Function> |
processFullDelimiter | false | <Function> |
-
Examples
- @yozora/tokenizer-autolink
- @yozora/tokenizer-autolink-extension
- @yozora/tokenizer-break
- @yozora/tokenizer-delete
- @yozora/tokenizer-emphasis
- @yozora/tokenizer-html-inline
- @yozora/tokenizer-image
- @yozora/tokenizer-image-reference
- @yozora/tokenizer-inline-code
- @yozora/tokenizer-inline-math
- @yozora/tokenizer-link
- @yozora/tokenizer-link-reference
- @yozora/tokenizer-text
parse-inline
See TokenizerParseInlineHook for Details.
Name | Required | Type |
---|---|---|
processToken | true | <Function> |
-
Examples
- @yozora/tokenizer-autolink
- @yozora/tokenizer-autolink-extension
- @yozora/tokenizer-break
- @yozora/tokenizer-delete
- @yozora/tokenizer-emphasis
- @yozora/tokenizer-html-inline
- @yozora/tokenizer-image
- @yozora/tokenizer-image-reference
- @yozora/tokenizer-inline-code
- @yozora/tokenizer-inline-math
- @yozora/tokenizer-link
- @yozora/tokenizer-link-reference
- @yozora/tokenizer-text