@yozora/tokenizer-list-item
A list marker is a bullet list marker or an ordered list marker.
A bullet list marker is a -
, +
, or *
character.
An ordered list markeris a sequence of arabic
digits (0-9
), followed by either a .
character or a )
character. (The
reason for the length limit is that with 10 digits we start seeing integer
overflows in some browsers.)
The following rules define list items:
- Basic case. If a sequence of lines constitute a sequence of blocks starting with a non-whitespace character, and is a list marker of width followed by spaces, then the result of prepending and the following spaces to the first line of , and indenting subsequent lines of by spaces, is a list item with Bs as its contents. The type of the list item (bullet or ordered) is determined by the type of its list marker. If the list item is ordered, then it is also assigned a start number, based on the ordered list marker.
Exceptions:
-
When the first list item in a list interrupts a paragraph—that is, when it starts on a line that would otherwise count as paragraph continuation text—then
- (a) the lines Ls must not begin with a blank line, and
- (b) if the list item is ordered, the start number must be .
-
If any line is a thematic break then that line is not a list item.
-
Item starting with indented code. If a sequence of lines constitute a sequence of blocks starting with an indented code block, and is a list marker of width followed by one space, then the result of prepending and the following space to the first line of , and indenting subsequent lines of by spaces, is a list item with as its contents. If a line is empty, then it need not be indented. The type of the list item (bullet or ordered) is determined by the type of its list marker. If the list item is ordered, then it is also assigned a start number, based on the ordered list marker.
-
Item starting with a blank line. If a sequence of lines starting with a single blank line constitute a (possibly empty) sequence of blocks , not separated from each other by more than one blank line, and is a list marker of width , then the result of prepending to the first line of , and indenting subsequent lines of by spaces, is a list item with as its contents. If a line is empty, then it need not be indented. The type of the list item (bullet or ordered) is determined by the type of its list marker. If the list item is ordered, then it is also assigned a start number, based on the ordered list marker.
-
Indentation. If a sequence of lines constitutes a list item according to rule #1, #2, or #3, then the result of indenting each line of by spaces (the same for each line) also constitutes a list item with the same contents and attributes. If a line is empty, then it need not be indented.
-
Laziness. If a string of lines constitute a list item with contents , then the result of deleting some or all of the indentation from one or more lines in which the next non-whitespace character after the indentation is paragraph continuation text is a list item with the same contents and attributes. The unindented lines are called lazy continuation lines.
- See github flavor markdown spec for details.
- See Live Examples for an intuitive impression.
Install
- npm
- Yarn
- pnpm
npm install --save @yozora/tokenizer-list-item
yarn add @yozora/tokenizer-list-item
pnpm add @yozora/tokenizer-list-item
Usage
@yozora/tokenizer-list-item has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm,
so you can use YozoraParser
/ GfmExParser
/ GfmParser
directly.
- Basic Usage
- YozoraParser
- GfmParser
- GfmExParser
@yozora/tokenizer-list-item cannot be used alone, it needs to be registered in YastParser as a plug-in for use.
import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import ListItemTokenizer from '@yozora/tokenizer-list-item'
const parser = new DefaultYastParser()
.useBlockFallbackTokenizer(new ParagraphTokenizer())
.useInlineFallbackTokenizer(new TextTokenizer())
.useTokenizer(new ListItemTokenizer({
emptyItemCouldNotInterruptedTypes: [ParagraphType],
enableTaskListItem: false,
}))
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
import YozoraParser from '@yozora/parser'
const parser = new YozoraParser()
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
import GfmParser from '@yozora/parser-gfm'
const parser = new GfmParser()
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
import GfmExParser from '@yozora/parser-gfm-ex'
const parser = new GfmExParser()
// parse source markdown content
parser.parse(`
- a
- b
- c
- d
- e
- f
- g
---
- [ ] This is a TODO item.
- [-] This is a processing TODO item.
- [x] This is a finished TODO item.
---
1. This is an ordered list item
a. This is an another type of ordered list item
`)
Options
Name | Type | Required | Default |
---|---|---|---|
name | string | false | "@yozora/tokenizer-list-item" |
priority | number | false | TokenizerPriority.CONTAINING_BLOCK |
emptyItemCouldNotInterruptedTypes | string[] | false | [ParagraphType, PhrasingContentType] |
enableTaskListItem | boolean | false | false |
-
name
: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase. -
priority
: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in thematch-block
stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.` -
emptyItemCouldNotInterruptedTypes
: Specify an array of YastNode types that could not be interrupted by this Tokenizer if the current list-item is empty. -
enableTaskListItem
: Should enable task list item (extension).
Types
@yozora/tokenizer-list-item produce ListItem type nodes. See @yozora/ast for full base types.
import type { YastParent } from '@yozora/ast'
export const ListItemType = 'listItem'
export type ListItemType = typeof ListItemType
/**
* Status of a task list item.
* @see https://github.github.com/gfm/#task-list-items-extension-
*/
export enum TaskStatus {
/**
* To do, not yet started.
*/
TODO = 'todo',
/**
* In progress.
*/
DOING = 'doing',
/**
* Completed.
*/
DONE = 'done',
}
/**
* ListItem represents an item in a List.
* @see https://github.com/syntax-tree/mdast#listitem
* @see https://github.github.com/gfm/#list-items
*/
export interface ListItem extends YastParent<ListItemType> {
/**
* Status of a todo task.
*/
status?: TaskStatus
}
Live Examples
-
Basic.
-
Item Starting with indented code.
-
Item Starting with a blank line.
-
Indentation.
-
Lazyniess.
-
Indentation in sublist.
-
Task list item (extension).
Related
- @yozora/ast
- @yozora/parser
- @yozora/parser-gfm
- @yozora/parser-gfm-ex
- @yozora/tokenizer-list
- @yozora/react-list
- @yozora/react-list-item
- @yozora/react-markdown
- Sourcecode
- ListItem | Yozora AST
- List items | Github Flavor Markdown Spec
- Task list items (extension) | Github Flavor Markdown Spec
- ListItem | Mdast