@yozora/tokenizer-link-reference

github flavor markdown spec

There are three kinds of reference links: full, collapsed, and shortcut.

Full reference link

A full reference link consists of a link text immediately followed by a link label that matches a link reference definition elsewhere in the document.

A link label begins with a left bracket ([) and ends with the first right bracket (]) that is not backslash-escaped. Between these brackets there must be at least one non-whitespace character. Unescaped square bracket characters are not allowed inside the opening and closing square brackets of link labels. A link label can have at most $999$ characters inside the square brackets.

One label matches another just in case their normalized forms are equal. To normalize a label, strip off the opening and closing brackets, perform the Unicode case fold, strip leading and trailing whitespace and collapse consecutive internal whitespace to a single space. If there are multiple matching reference link definitions, the one that comes first in the document is used. (It is desirable in such cases to emit a warning.)

The link’s URI and title are provided by the matching link reference definition.

Collapsed reference link

A collapsed reference link consists of a link label that matches a link reference definition elsewhere in the document, followed by the string []. The contents of the first link label are parsed as inlines, which are used as the link’s text. The link’s URI and title are provided by the matching reference link definition. Thus, [foo][] is equivalent to [foo][foo].

Shortcut reference link

A shortcut reference link consists of a link label that matches a link reference definition elsewhere in the document and is not followed by [] or a link label. The contents of the first link label are parsed as inlines, which are used as the link’s text. The link’s URI and title are provided by the matching [link reference definition][gfm-link-definiiton]. Thus, [foo] is equivalent to [foo][].

• See github flavor markdown spec for details.
• See Live Examples for an intuitive impression.

Install

npm

npm install --save @yozora/tokenizer-link-reference

Yarn

yarn add @yozora/tokenizer-link-reference

pnpm

pnpm add @yozora/tokenizer-link-reference

Usage

tip

@yozora/tokenizer-link-reference has been integrated into @yozora/parser / @yozora/parser-gfm-ex / @yozora/parser-gfm, so you can use YozoraParser / GfmExParser / GfmParser directly.

Basic Usage

@yozora/tokenizer-link-reference cannot be used alone, it needs to be registered in YastParser as a plugin-in before it can be used.

import { DefaultYastParser } from '@yozora/core-parser'
import ParagraphTokenizer from '@yozora/tokenizer-paragraph'
import TextTokenizer from '@yozora/tokenizer-text'
import LinkReferenceTokenizer from '@yozora/tokenizer-link-reference'

const parser = new DefaultYastParser()
  .useBlockFallbackTokenizer(new ParagraphTokenizer())
  .useInlineFallbackTokenizer(new TextTokenizer())
  .useTokenizer(new LinkReferenceTokenizer())

// parse source markdown content
parser.parse(`
[foo][bar]

[bar]: /url "title"
`)

YozoraParser

import YozoraParser from '@yozora/parser'

const parser = new YozoraParser()

// parse source markdown content
parser.parse(`
[foo][bar]

[bar]: /url "title"
`)

GfmParser

import GfmParser from '@yozora/parser-gfm'

const parser = new GfmParser()

// parse source markdown content
parser.parse(`
[foo][bar]

[bar]: /url "title"
`)

GfmExParser

import GfmExParser from '@yozora/parser-gfm-ex'

const parser = new GfmExParser()

// parse source markdown content
parser.parse(`
[foo][bar]

[bar]: /url "title"
`)

Options

Name	Type	Required	Default
name	string	false	"@yozora/tokenizer-link-reference"
priority	number	false	TokenizerPriority.LINKS

• name: The unique name of the tokenizer, used to bind the token it generates, to determine the tokenizer that should be called in each life cycle of the token in the entire matching / parsing phase.

• priority: Priority of the tokenizer, determine the order of processing, high priority priority execution. interruptable. In addition, in the match-block stage, a high-priority tokenizer can interrupt the matching process of a low-priority tokenizer.

  Exception: Delimiters of type full are always processed before other type delimiters.

Types

@yozora/tokenizer-link-reference produce LinkReference type nodes. See @yozora/ast for full base types.

import type {
YastAssociation,
YastParent,
YastReference,
} from '@yozora/ast'

export const LinkReferenceType = 'linkReference'
export type LinkReferenceType = typeof LinkReferenceType

/**
* LinkReference represents a hyperlink through association, or its original
* source if there is no association.
* @see https://github.com/syntax-tree/mdast#linkreference
* @see https://github.github.com/gfm/#reference-link
*/
export interface LinkReference extends
YastParent<LinkReferenceType>,
YastAssociation,
YastReference {}

Live Examples

Full reference link

• Basic.

  #535

  yozora

    

  pretty-json

    

  [foo][bar] [bar]: /url "title"

  1
  2
  3
  [foo][bar]
  
  [bar]: /url "title"

• The link text may contain balanced brackets, but not unbalanced ones, unless they are escaped.

  #536

  yozora

    

  pretty-json

    

  [link [foo [bar]]][ref] [ref]: /uri

  1
  2
  3
  [link [foo [bar]]][ref]
  
  [ref]: /uri

• The link text may contain inline content.

  #538

  yozora

    

  pretty-json

    

  [link *foo **bar** `#`*][ref] [ref]: /uri

  1
  2
  3
  [link *foo **bar** `#`*][ref]
  
  [ref]: /uri

• However, links may not contain other links, at any level of nesting.

  #540

  yozora

    

  pretty-json

    

  [foo [bar](/uri)][ref] [ref]: /uri

  1
  2
  3
  [foo [bar](/uri)][ref]
  
  [ref]: /uri

• The following cases illustrate the precedence of link text grouping over emphasis grouping.

  #542

  yozora

    

  pretty-json

    

  *[foo*][ref] [ref]: /uri

  1
  2
  3
  *[foo*][ref]
  
  [ref]: /uri

• These cases illustrate the precedence of HTML tags, code spans, and autolinks over link grouping.

  #544

  yozora

    

  pretty-json

    

  [foo <bar attr="][ref]"> [ref]: /uri

  1
  2
  3
  [foo <bar attr="][ref]">
  
  [ref]: /uri

• Matching is case-insensitive.

  #547

  yozora

    

  pretty-json

    

  [foo][BaR] [bar]: /url "title"

  1
  2
  3
  [foo][BaR]
  
  [bar]: /url "title"

• Unicode case fold is used.

  #548

  yozora

    

  pretty-json

    

  [ẞ] [SS]: /url

  1
  2
  3
  [ẞ]
  
  [SS]: /url

• Consecutive internal whitespace is treated as one space for purposes of determining matching.

  #549

  yozora

    

  pretty-json

    

  [Foo bar]: /url [Baz][Foo bar]

  1
  2
  3
  4
  [Foo
    bar]: /url
  
  [Baz][Foo bar]

• No whitespace is allowed between the link text and the link label.

  #550

  yozora

    

  pretty-json

    

  [foo] [bar] [bar]: /url "title"

  1
  2
  3
  [foo] [bar]
  
  [bar]: /url "title"

• When there are multiple matching link reference definitions, the first is used.

  #552

  yozora

    

  pretty-json

    

  [foo]: /url1 [foo]: /url2 [bar][foo]

  1
  2
  3
  4
  5
  [foo]: /url1
  
  [foo]: /url2
  
  [bar][foo]

• Note that matching is performed on normalized strings, not parsed inline content. So the following does not match, even though the labels define equivalent inline content.

  #553

  yozora

    

  pretty-json

    

  [bar][foo\!] [foo!]: /url

  1
  2
  3
  [bar][foo\!]
  
  [foo!]: /url

• Link labels cannot contain brackets, unless they are backslash-escaped.

  #554

  yozora

    

  pretty-json

    

  [foo][ref[] [ref[]: /uri

  1
  2
  3
  [foo][ref[]
  
  [ref[]: /uri

• Note that in this example ] is not backslash-escaped.

  #558

  yozora

    

  pretty-json

    

  [bar\\]: /uri [bar\\]

  1
  2
  3
  [bar\\]: /uri
  
  [bar\\]

• A link label must contain at least one non-whitespace character.

  #559

  yozora

    

  pretty-json

    

  [] []: /uri

  1
  2
  3
  []
  
  []: /uri

Collapsed reference link

• Basic.

  #561

  yozora

    

  pretty-json

    

  [foo][] [foo]: /url "title"

  1
  2
  3
  [foo][]
  
  [foo]: /url "title"

• The link labels are case-insensitive.

  #563

  yozora

    

  pretty-json

    

  [Foo][] [foo]: /url "title"

  1
  2
  3
  [Foo][]
  
  [foo]: /url "title"

• As with full reference links, whitespace is not allowed between the two sets of brackets.

  #564

  yozora

    

  pretty-json

    

  [foo] [] [foo]: /url "title"

  1
  2
  3
  4
  [foo] 
  []
  
  [foo]: /url "title"

Shortcut reference link

• Basic.

  #565

  yozora

    

  pretty-json

    

  [foo] [foo]: /url "title"

  1
  2
  3
  [foo]
  
  [foo]: /url "title"

• The link labels are case-insensitive.

  #569

  yozora

    

  pretty-json

    

  [Foo] [foo]: /url "title"

  1
  2
  3
  [Foo]
  
  [foo]: /url "title"

• A space after the link text should be preserved.

  #570

  yozora

    

  pretty-json

    

  [foo] bar [foo]: /url

  1
  2
  3
  [foo] bar
  
  [foo]: /url

• If you just want bracketed text, you can backslash-escape the opening bracket to avoid links.

  #571

  yozora

    

  pretty-json

    

  \[foo] [foo]: /url "title"

  1
  2
  3
  \[foo]
  
  [foo]: /url "title"

• Note that this is a link, because a link label ends with the first following closing bracket.

  #572

  yozora

    

  pretty-json

    

  [foo*]: /url *[foo*]

  1
  2
  3
  [foo*]: /url
  
  *[foo*]

• Full and compact references take precedence over shortcut references.

  #573

  yozora

    

  pretty-json

    

  [foo][bar] [foo]: /url1 [bar]: /url2

  1
  2
  3
  4
  [foo][bar]
  
  [foo]: /url1
  [bar]: /url2

• Inline links also take precedence.

  #575

  yozora

    

  pretty-json

    

  [foo]() [foo]: /url1

  1
  2
  3
  [foo]()
  
  [foo]: /url1

• In the following case [bar][baz] is parsed as a reference, [foo] as normal text.

  #577

  yozora

    

  pretty-json

    

  [foo][bar][baz] [baz]: /url

  1
  2
  3
  [foo][bar][baz]
  
  [baz]: /url

• Here, though, [foo][bar] is parsed as a reference, since [bar] is defined.

  #578

  yozora

    

  pretty-json

    

  [foo][bar][baz] [baz]: /url1 [bar]: /url2

  1
  2
  3
  4
  [foo][bar][baz]
  
  [baz]: /url1
  [bar]: /url2

• Here [foo] is not parsed as a shortcut reference, because it is followed by a link label (even though [bar] is not defined).

  #579

  yozora

    

  pretty-json

    

  [foo][bar][baz] [baz]: /url1 [foo]: /url2

  1
  2
  3
  4
  [foo][bar][baz]
  
  [baz]: /url1
  [foo]: /url2

Related

• @yozora/ast
• @yozora/parser
• @yozora/parser-gfm
• @yozora/parser-gfm-ex
• @yozora/tokenizer-autolink
• @yozora/tokenizer-autolink-extension
• @yozora/tokenizer-definition
• @yozora/tokenizer-link
• @yozora/react-link
• @yozora/react-markdown
• Sourcecode
• LinkReference | Yozora AST
• Reference links | Github Flavor Markdown Spec
• LinkReference | Mdast