cotonic.tokenizer Cotonic

The tokenizer transforms text to html tokens. The tokenizer is used by the user interface composer in order to call the incremental-dom api. It uses incremental-dom to do in-place diffing of the dom-tree.
Note: The tokenizer does not parse or validate the html. It just tokenizes the input.

tokens cotonic.tokenizer.tokens(text)

Transforms a string with html tags into a list of tokens. The tokens are objects of the form: {type: "type", [args]}, with type being one of:

open

Represents an open tag. Contains the attribute tag which is set to the tagname of the element, and the attribute attributes which is a list of attributes of the element.

close

Represents a close tag. Contains the attribute tag which is set to the tagname of the close element.

void

Represents a void element. The attribute tag is set to the tagname of the void element.

text

Represents a test element. The attribute data is set to the text data.

doctype

Represents a doctype element. The attribute attributes is set to the attributes of the element.

pi

Represents a processing instruction element. The attribute tag is set to the tagname of the processing instruction. The attribute arguments contains the list of attributes.

comment

Represents a comment element. The attribute data contains the text in the comment element.

cotonic.tokenizer.tokens("<div class='example'>Tokenizing<br /> is cool</div>");
=> [{type: "open", tag: "div", attributes: ["class", "example"]},
    {type: "text", data: "Tokenizing"},
    {type: "void", tag: "br", attributes: []},
    {type: "text", data: " is cool"},
    {type: "close", tag: "div"}] 

charref cotonic.tokenizer.charref(text)

Transforms a html charref into a character.

cotonic.tokenizer.charref("#128540");
=> "😜"
cotonic.tokenizer.charref("amp");
=> "&"