Skip to main content
Version: 1.x.x

@yozora/core-parser

Npm VersionNpm DownloadNpm LicenseModule formats: cjs, esmNode.js VersionCode Style: prettier

@yozora/core-parser provides a plug-in system that can load tokenizer for collaborative processing in the lifecycles.

Install

npm install --save @yozora/core-parser

Usage

@yozora/core-parser provides a default YastParser implementation DefaultYastParser.

import { DefaultYastParser } from '@yozora/core-parser'
import EmphasisTokenizer from '@yozora/tokenizer-emphasis'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'

const parser = new DefaultYastParser({ shouldReservePosition: true })
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())

// add custom tokenizers through `useTokenizer()`
.useTokenizer(new EmphasisTokenizer())

// Parse content
parser.parse('source content')

// Specify the content range
parser.parse('source content', 0, 10)

// Remove the positions from the YastNode.
parser.parse('source content', undefined, undefined, false)

// Reserve the positions
parser.parse('source content', undefined, undefined, true)

Options

NameRequiredDefaultType
blockFallbackTokenizerfalsenullBlockFallbackTokenizer | null
inlineFallbackTokenizerfalsenullInlineFallbackTokenizer | null
lazinessTypesfalseSee belowstring[]
shouldReservePositionfalsefalseboolean
  • blockFallbackTokenizer: Fallback tokenizer on processing block structure phase.

  • inlineFallbackTokenizer: Fallback tokenizer on processing inline structure phase.

  • lazinessTypes: Types of YastNode which could has laziness contents.

    • Default: result of [PhrasingContentType].concat(blockFallbackTokenizer.uniqueTypes).
  • shouldReservePosition: Whether it is necessary to reserve the position of the YastNode parsed.

Lifecycle

match-block

See TokenizerMatchBlockHook for Details.

NameRequiredType
isContainerBlocktrueboolean
interruptableTypestruestring[]
eatOpenertrue<Function>
eatAndInterruptPreviousSiblingfalse<Function>
eatContinuationTextfalse<Function>
eatLazyContinuationTextfalse<Function>
onClosefalse<Function>
extractPhrasingContentLinesfalse<Function>
buildBlockStatefalse<Function>

post-match-block

See TokenizerPostMatchBlockHook for Details.

NameRequiredType
transformMatchtrue<Function>

parse-block

See TokenizerParseBlockHook for Details.

NameRequiredType
parseBlocktrue<Function>
parseMetafalse<Function>

match-inline

See TokenizerMatchInlineHook for Details.

NameRequiredType
prioritynumber<Function>
delimiterGroupstring<Function>
findDelimitertrue<Function>
isDelimiterPairfalse<Function>
processDelimiterPairfalse<Function>
processFullDelimiterfalse<Function>

parse-inline

See TokenizerParseInlineHook for Details.

NameRequiredType
processTokentrue<Function>