Browse Source

Update ParserHtml

custom-code-parser
Artur Arseniev 4 months ago
parent
commit
e9f09ca811
  1. 137
      docs/guides/Custom-HTML-parser.md
  2. 35
      docs/modules/Components.md
  3. 441
      packages/core/src/parser/model/ParserHtml.ts
  4. 12
      packages/core/src/parser/model/SyntheticElement.ts
  5. 19
      packages/core/src/parser/types.ts

137
docs/guides/Custom-HTML-parser.md

@ -0,0 +1,137 @@
---
title: Use Custom HTML Parser
---
# Use Custom HTML Parser
If your GrapesJS integration needs to parse HTML in environments where DOM APIs are not available, you can register a custom HTML code parser and let GrapesJS compile the returned parsed nodes into components.
This is useful for:
- server-side or worker-based HTML imports
- integrations that already have their own HTML parser
- cases where component recognition should not depend on browser DOM nodes
## Register a parser
Code parsers are managed by the `Parser` module.
```js
const { Parser } = editor;
Parser.addParserCode(
'my-parser',
(input) => {
return [
{
nodeType: 1,
tagName: 'section',
attributes: { class: 'hero' },
childNodes: [{ nodeType: 3, textContent: 'Hello world' }],
},
];
}
);
```
The parser function must always return an array of parsed nodes.
## Select a parser
You can select the active parser globally:
```js
Parser.parserCode = 'my-parser';
const result = Parser.parseHtml('<section>Hello world</section>');
```
Or for a single call:
```js
const result = Parser.parseHtml('<section>Hello world</section>', {
parserCode: 'my-parser',
});
```
Passing `parserCode: ''` forces the built-in DOM parser path for that call.
## Parsed nodes
Custom parsers return nodes shaped like this:
```ts
interface ParsedNode {
nodeType?: number;
tagName?: string;
namespaceURI?: string;
attributes?: Record<string, string>;
childNodes?: ParsedNode[];
textContent?: string;
}
```
Supported node types in the current implementation are:
- `1` for elements
- `3` for text nodes
- `8` for comments
- `9` for documents
- `11` for document fragments
When `asDocument: true` is used, GrapesJS normalizes the parser output to a document-like root so `root`, `head`, and `body` can still be compiled.
## Component recognition
For headless parsing, component types can implement `isParsedNode`:
```js
editor.Components.addType('my-component', {
isParsedNode(node, opts) {
if (node.tagName === 'my-component') {
return { type: 'my-component' };
}
},
});
```
When `parserCode` is active, `isParsedNode` is preferred over `isComponent`.
## Legacy `isComponent` fallback
Existing components that only implement `isComponent` continue to work with `parserCode`.
GrapesJS passes a read-only synthetic element that exposes the most common DOM-like properties:
- `nodeType`
- `tagName`
- `nodeName`
- `namespaceURI`
- `textContent`
- `nodeValue`
- `parentNode`
- `childNodes`
- `children`
- `getAttribute`
- `hasAttribute`
If you need more DOM-like helpers, extend the base synthetic element:
```js
editor.Parser.config.customSyntheticElement = (SyntheticElement) =>
class MySyntheticElement extends SyntheticElement {
get foo() {
return this.getAttribute('data-foo') || '';
}
};
```
## Registry helpers
You can inspect and manage the registry at runtime:
```js
const parser = Parser.getParserCode('my-parser');
const removed = Parser.removeParserCode('my-parser');
const registry = Parser.parsersCode;
```
Removing the selected parser clears `Parser.parserCode`.

35
docs/modules/Components.md

@ -93,6 +93,8 @@ As we mentioned before, when you pass an HTML string as a component to the edito
<img :src="$withBase('/component-type-stack.svg')" class="img-ctr"> <img :src="$withBase('/component-type-stack.svg')" class="img-ctr">
When you use a custom HTML code parser via `Parser.parserCode`, component recognition can also rely on `isParsedNode`, which receives the normalized parsed node instead of a DOM element.
::: tip ::: tip
If you're importing big string chunks of HTML code you might want to improve the performances by skipping the parsing and the component recognition steps by passing directly Component Definition objects or using the JSX syntax. If you're importing big string chunks of HTML code you might want to improve the performances by skipping the parsing and the component recognition steps by passing directly Component Definition objects or using the JSX syntax.
Read [here](#setup-jsx-syntax) about how to setup JSX syntax parser Read [here](#setup-jsx-syntax) about how to setup JSX syntax parser
@ -339,6 +341,39 @@ editor.addComponents('<some-element data-gjs-type="some-component">...');
If you define the Component Type without using `isComponent`, the only way for the editor to see that component will be with an explicitly declared type (via an object `{ type: '...' }` or using `data-gjs-type`). If you define the Component Type without using `isComponent`, the only way for the editor to see that component will be with an explicitly declared type (via an object `{ type: '...' }` or using `data-gjs-type`).
### isParsedNode
If your HTML is parsed through a custom code parser, you can avoid DOM dependencies completely by using `isParsedNode`.
```js
editor.Components.addType('my-input-type', {
isParsedNode: (node) => {
if (node.tagName === 'input') {
return {
type: 'my-input-type',
};
}
},
// ...
});
```
The method receives a normalized parsed node and can return the same kind of values accepted by `isComponent`.
If both `isParsedNode` and `isComponent` are provided, `isParsedNode` has priority while `parserCode` is active.
Existing `isComponent` definitions continue to work in headless parsing too. In that case GrapesJS provides a read-only synthetic element with common DOM-like properties such as `tagName`, `childNodes`, `children`, `getAttribute`, and `textContent`.
If one of your legacy checks needs extra helpers, extend the synthetic element globally:
```js
editor.Parser.config.customSyntheticElement = (SyntheticElement) =>
class MySyntheticElement extends SyntheticElement {
get foo() {
return this.getAttribute('data-foo') || '';
}
};
```
### Model ### Model
Now that we got how `isComponent` works we can start to explore the `model` property. Now that we got how `isComponent` works we can start to explore the `model` property.

441
packages/core/src/parser/model/ParserHtml.ts

@ -2,14 +2,35 @@ import { each, isArray, isFunction, isUndefined, result } from 'underscore';
import { ObjectAny, ObjectStrings } from '../../common'; import { ObjectAny, ObjectStrings } from '../../common';
import { ComponentDefinitionDefined, ComponentStackItem } from '../../dom_components/model/types'; import { ComponentDefinitionDefined, ComponentStackItem } from '../../dom_components/model/types';
import EditorModel from '../../editor/model/Editor'; import EditorModel from '../../editor/model/Editor';
import { HTMLParseResult, HTMLParserOptions, ParseNodeOptions, ParserConfig } from '../config/config';
import BrowserParserHtml from './BrowserParserHtml';
import { doctypeToString, processDataGjsAttributeHyphen } from '../../utils/dom'; import { doctypeToString, processDataGjsAttributeHyphen } from '../../utils/dom';
import { isDef } from '../../utils/mixins'; import { isDef } from '../../utils/mixins';
import { ParserEvents } from '../types'; import { HTMLParserOptions, ParseNodeOptions, ParserConfig } from '../config/config';
import {
HTMLParseResult,
ParsedElementNode,
ParsedNode,
ParsedNodeMeta,
ParsedNodeNamespace,
ParsedNodeType,
ParserEvents,
SyntheticElementCtor,
} from '../types';
import BrowserParserHtml from './BrowserParserHtml';
import { getSyntheticElementCtor } from './SyntheticElement';
const modelAttrStart = 'data-gjs-'; const modelAttrStart = 'data-gjs-';
interface ParserHtmlInternalOptions extends ParseNodeOptions {
__parsedMode?: boolean;
__syntheticElementCtor?: SyntheticElementCtor;
}
const hasOwn = (obj: object, key: string) => Object.prototype.hasOwnProperty.call(obj, key);
const getNodeChildNodes = (node: ParsedNodeMeta) => node.childNodes || [];
const getNodeTagName = (node: ParsedNodeMeta) => `${node.tagName || ''}`.toLowerCase();
const getSourceNode = (node: ParsedNodeMeta) => node.__domNode || node;
const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boolean } = {}) => { const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boolean } = {}) => {
return { return {
compTypes: [] as ComponentStackItem[], compTypes: [] as ComponentStackItem[],
@ -50,7 +71,7 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
shouldConvertAttributeValue( shouldConvertAttributeValue(
attribute: string, attribute: string,
value: string | boolean, value: string | boolean,
node: HTMLElement, node: HTMLElement | ParsedElementNode,
convertAttributeValues: HTMLParserOptions['convertAttributeValues'], convertAttributeValues: HTMLParserOptions['convertAttributeValues'],
) { ) {
if (!convertAttributeValues) { if (!convertAttributeValues) {
@ -156,31 +177,30 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
}, },
parseNodeAttr( parseNodeAttr(
node: HTMLElement, node: ParsedNodeMeta,
modelResult?: ComponentDefinitionDefined, modelResult?: ComponentDefinitionDefined,
opts: HTMLParserOptions = config.optionsHtml || {}, opts: HTMLParserOptions = config.optionsHtml || {},
) { ) {
const model = modelResult || {}; const model = modelResult || {};
const attrs = node.attributes || []; const attrs = node.attributes || {};
const attrsLen = attrs.length;
const convertHyphens = !!opts.convertDataGjsAttributesHyphens; const convertHyphens = !!opts.convertDataGjsAttributesHyphens;
const { convertAttributeValues } = opts; const { convertAttributeValues } = opts;
const defaults = const defaults =
(convertHyphens && !!model.type && result(em?.Components.getType(model.type)?.model.prototype, 'defaults')) || (convertHyphens && !!model.type && result(em?.Components.getType(model.type)?.model.prototype, 'defaults')) ||
{}; {};
const sourceNode = getSourceNode(node) as HTMLElement | ParsedElementNode;
for (let i = 0; i < attrsLen; i++) {
let nodeName = attrs[i].nodeName; each(attrs, (attrValue, attrName) => {
let nodeValue: any = attrs[i].nodeValue!; let nodeValue: any = attrValue;
if (nodeName == 'style') { if (attrName == 'style') {
model.style = this.parseStyle(nodeValue); model.style = this.parseStyle(`${nodeValue}`);
} else if (nodeName == 'class') { } else if (attrName == 'class') {
model.classes = this.parseClass(nodeValue); model.classes = this.parseClass(`${nodeValue}`);
} else if (nodeName == 'contenteditable') { } else if (attrName == 'contenteditable') {
continue; return;
} else if (nodeName.indexOf(this.modelAttrStart) === 0) { } else if (attrName.indexOf(this.modelAttrStart) === 0) {
const propsResult = this.getPropAttribute(nodeName, nodeValue); const propsResult = this.getPropAttribute(attrName, `${nodeValue}`);
let resolvedName = propsResult.name; let resolvedName = propsResult.name;
if (convertHyphens && !(resolvedName in defaults)) { if (convertHyphens && !(resolvedName in defaults)) {
const transformed = processDataGjsAttributeHyphen(resolvedName); const transformed = processDataGjsAttributeHyphen(resolvedName);
@ -189,12 +209,14 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
model[resolvedName] = propsResult.value; model[resolvedName] = propsResult.value;
} else { } else {
// @ts-ignore Check for attributes from props (eg. required, disabled) if (
if (nodeValue === '' && node[nodeName] === true) { nodeValue === '' &&
((node.__domNode as any)?.[attrName] === true || node.__boolAttributes?.includes(attrName))
) {
nodeValue = true; nodeValue = true;
} }
if (this.shouldConvertAttributeValue(nodeName, nodeValue, node, convertAttributeValues)) { if (this.shouldConvertAttributeValue(attrName, nodeValue, sourceNode, convertAttributeValues)) {
nodeValue = this.parseAttributeValue(nodeValue); nodeValue = this.parseAttributeValue(nodeValue);
} }
@ -202,19 +224,19 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
model.attributes = {}; model.attributes = {};
} }
model.attributes[nodeName] = nodeValue; model.attributes[attrName] = nodeValue;
} }
} });
return model; return model;
}, },
detectNode(node: HTMLElement, opts: ParseNodeOptions = {}) { detectNode(node: ParsedNodeMeta, opts: ParserHtmlInternalOptions = {}) {
const { compTypes } = this; const { compTypes } = this;
let result: ComponentDefinitionDefined = {}; let result: ComponentDefinitionDefined = {};
if (compTypes) { if (compTypes) {
const type = node.getAttribute?.(`${this.modelAttrStart}type`); const type = node.attributes?.[`${this.modelAttrStart}type`];
// If the type is already defined, use it // If the type is already defined, use it
if (type) { if (type) {
@ -223,7 +245,16 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
// Find the component type // Find the component type
for (let i = 0; i < compTypes.length; i++) { for (let i = 0; i < compTypes.length; i++) {
const compType = compTypes[i]; const compType = compTypes[i];
let obj = compType.model.isComponent(node, opts); const { model } = compType;
let obj;
if (opts.__parsedMode) {
obj = model.isParsedNode
? model.isParsedNode(node, opts)
: model.isComponent(this.__getSyntheticNode(node, opts) as any, opts);
} else {
obj = model.isComponent(getSourceNode(node), opts);
}
if (obj) { if (obj) {
if (typeof obj !== 'object') { if (typeof obj !== 'object') {
@ -239,21 +270,21 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
return result; return result;
}, },
parseNode(node: HTMLElement, opts: ParseNodeOptions = {}) { parseNode(node: ParsedNodeMeta, opts: ParserHtmlInternalOptions = {}) {
const nodes = (node as HTMLTemplateElement).content?.childNodes || node.childNodes; const nodes = getNodeChildNodes(node);
const nodesLen = nodes.length; const nodesLen = nodes.length;
let model = this.detectNode(node, opts); let model = this.detectNode(node, opts);
if (!model.tagName && model.tagName !== '') { if (!model.tagName && model.tagName !== '') {
const tag = node.tagName || ''; const tag = node.tagName || '';
const ns = node.namespaceURI || ''; const ns = node.namespaceURI || '';
model.tagName = tag && ns === 'http://www.w3.org/1999/xhtml' ? tag.toLowerCase() : tag; model.tagName = tag && ns === ParsedNodeNamespace.html ? tag.toLowerCase() : tag;
} }
model = this.parseNodeAttr(node, model, opts); model = this.parseNodeAttr(node, model, opts);
// Check for custom void elements (valid in XML) // Check for custom void elements (valid in XML)
if (!nodesLen && `${node.outerHTML}`.slice(-2) === '/>') { if (!nodesLen && node.__selfClosing) {
model.void = true; model.void = true;
} }
@ -264,11 +295,11 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
// If there is only one child and it's a TEXTNODE // If there is only one child and it's a TEXTNODE
// just make it content of the current node // just make it content of the current node
if (nodesLen === 1 && firstChild.nodeType === 3) { if (nodesLen === 1 && firstChild.nodeType === ParsedNodeType.text) {
!model.type && (model.type = 'text'); !model.type && (model.type = 'text');
model.components = { model.components = {
type: 'textnode', type: 'textnode',
content: firstChild.nodeValue, content: firstChild.textContent,
}; };
} else { } else {
model.components = this.parseNodes(node, { model.components = this.parseNodes(node, {
@ -313,13 +344,13 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
* @param {HTMLElement} el DOM element to traverse * @param {HTMLElement} el DOM element to traverse
* @return {Array<Object>} * @return {Array<Object>}
*/ */
parseNodes(el: HTMLElement, opts: ParseNodeOptions = {}) { parseNodes(el: ParsedNodeMeta, opts: ParserHtmlInternalOptions = {}) {
const result: ComponentDefinitionDefined[] = []; const result: ComponentDefinitionDefined[] = [];
const nodes = (el as HTMLTemplateElement).content?.childNodes || el.childNodes; const nodes = getNodeChildNodes(el);
const nodesLen = nodes.length; const nodesLen = nodes.length;
for (let i = 0; i < nodesLen; i++) { for (let i = 0; i < nodesLen; i++) {
const node = nodes[i] as HTMLElement; const node = nodes[i];
const nodePrev = result[result.length - 1]; const nodePrev = result[result.length - 1];
const model = this.parseNode(node, opts); const model = this.parseNode(node, opts);
@ -333,10 +364,10 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
// Try to keep meaningful whitespaces when possible (#5984) // Try to keep meaningful whitespaces when possible (#5984)
// Ref: https://github.com/GrapesJS/grapesjs/pull/5719#discussion_r1518531999 // Ref: https://github.com/GrapesJS/grapesjs/pull/5719#discussion_r1518531999
if (!opts.keepEmptyTextNodes) { if (!opts.keepEmptyTextNodes) {
const content = node.nodeValue || ''; const content = node.textContent || '';
const isFirstOrLast = i === 0 || i === nodesLen - 1; const isFirstOrLast = i === 0 || i === nodesLen - 1;
const hasNewLive = content.includes('\n'); const hasNewLine = content.includes('\n');
if (content != ' ' && !content.trim() && (isFirstOrLast || hasNewLive)) { if (content != ' ' && !content.trim() && (isFirstOrLast || hasNewLine)) {
continue; continue;
} }
} }
@ -363,14 +394,15 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
const conf = em?.get('Config') || {}; const conf = em?.get('Config') || {};
const Parser = em?.Parser; const Parser = em?.Parser;
const res: HTMLParseResult = { html: [] }; const res: HTMLParseResult = { html: [] };
const parserCode = this.__getParserCodeId(opts);
const preOptions = { const preOptions = {
...config.optionsHtml, ...config.optionsHtml,
// @ts-ignore Support previous `configParser.htmlType` option htmlType: config.optionsHtml?.htmlType || (config as any).htmlType,
htmlType: config.optionsHtml?.htmlType || config.htmlType,
...opts, ...opts,
}; };
const options = { const options = {
...preOptions, ...preOptions,
parserCode,
asDocument: this.__checkAsDocument(str, preOptions), asDocument: this.__checkAsDocument(str, preOptions),
}; };
const cf = { ...config, ...options }; const cf = { ...config, ...options };
@ -378,42 +410,28 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
const inputOptions = { input: isFunction(preParser) ? preParser(str, { editor: em?.getEditor()! }) : str }; const inputOptions = { input: isFunction(preParser) ? preParser(str, { editor: em?.getEditor()! }) : str };
Parser?.__emitEvent(ParserEvents.htmlBefore, inputOptions); Parser?.__emitEvent(ParserEvents.htmlBefore, inputOptions);
const { input } = inputOptions; const { input } = inputOptions;
const parseRes = isFunction(cf.parserHtml) ? cf.parserHtml(input, options) : BrowserParserHtml(input, options); const parseRes = this.__parseInput(input, options, cf, parserCode);
let root = parseRes as HTMLElement; const root = parseRes.root;
const docEl = parseRes as Document; const parserConfig = Parser?.getConfig() || config;
const parseOptions: ParserHtmlInternalOptions = {
if (asDocument) { ...cf,
root = docEl.documentElement; __parsedMode: parseRes.isParsedMode,
res.doctype = doctypeToString(docEl.doctype); __syntheticElementCtor: getSyntheticElementCtor(parserConfig.customSyntheticElement),
} };
const scripts = root.querySelectorAll('script');
let i = scripts.length;
// Support previous `configMain.allowScripts` option
const allowScripts = !isUndefined(conf.allowScripts) ? conf.allowScripts : options.allowScripts; const allowScripts = !isUndefined(conf.allowScripts) ? conf.allowScripts : options.allowScripts;
// Remove script tags
if (!allowScripts) { if (!allowScripts) {
while (i--) scripts[i].parentNode?.removeChild(scripts[i]); this.__removeElementNodes(root, 'script');
} }
// Remove unsafe attributes
if (!options.allowUnsafeAttr || !options.allowUnsafeAttrValue) { if (!options.allowUnsafeAttr || !options.allowUnsafeAttrValue) {
this.__sanitizeNode(root, options); this.__sanitizeNode(root, options);
} }
// Detach style tags and parse them
if (parserCss) { if (parserCss) {
const styles = root.querySelectorAll('style'); const styleNodes = this.__removeElementNodes(root, 'style');
let j = styles.length; const styleStr = styleNodes.map((node) => this.__getNodeTextContent(node)).join('');
let styleStr = '';
while (j--) {
styleStr = styles[j].innerHTML + styleStr;
styles[j].parentNode?.removeChild(styles[j]);
}
if (styleStr) res.css = parserCss.parse(styleStr); if (styleStr) res.css = parserCss.parse(styleStr);
} }
@ -421,12 +439,19 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
let resHtml: HTMLParseResult['html'] = []; let resHtml: HTMLParseResult['html'] = [];
if (asDocument) { if (asDocument) {
res.head = this.parseNode(docEl.head, cf); const docNode: ParsedNodeMeta =
res.root = this.parseNodeAttr(root, undefined, cf); root.nodeType === ParsedNodeType.document ? root : this.__normalizeDocumentRoot(getNodeChildNodes(root));
resHtml = this.parseNode(docEl.body, cf); const htmlNode = this.__findChildElement(docNode, 'html');
const headNode = htmlNode && this.__findChildElement(htmlNode, 'head');
const bodyNode = (htmlNode && this.__findChildElement(htmlNode, 'body')) || this.__createElementNode('body');
res.doctype = docNode.__doctype;
headNode && (res.head = this.parseNode(headNode, parseOptions));
htmlNode && (res.root = this.parseNodeAttr(htmlNode, undefined, parseOptions));
resHtml = this.parseNode(bodyNode, parseOptions);
} else { } else {
const result = this.parseNodes(root, cf); const result = this.parseNodes(root, parseOptions);
// I have to keep it otherwise it breaks the DomComponents.addComponent (returns always array) // Need this otherwise it breaks the DomComponents.addComponent (returns always array)
resHtml = result.length === 1 && !cf.returnArray ? result[0] : result; resHtml = result.length === 1 && !cf.returnArray ? result[0] : result;
} }
@ -436,18 +461,268 @@ const ParserHtml = (em?: EditorModel, config: ParserConfig & { returnArray?: boo
return res; return res;
}, },
__sanitizeNode(node: HTMLElement, opts: HTMLParserOptions) { __getSyntheticNode(node: ParsedNodeMeta, opts: ParserHtmlInternalOptions) {
const attrs = node.attributes || []; const parserConfig = em?.Parser?.getConfig() || config;
const nodes = node.childNodes || []; const SyntheticElement =
const toRemove: string[] = []; opts.__syntheticElementCtor || getSyntheticElementCtor(parserConfig.customSyntheticElement);
each(attrs, (attr) => { return new SyntheticElement(node);
const name = attr.nodeName || ''; },
const value = attr.nodeValue || '';
!opts.allowUnsafeAttr && name.startsWith('on') && toRemove.push(name); __parseInput(input: string, options: HTMLParserOptions, cf: ParserConfig, parserCode: string) {
!opts.allowUnsafeAttrValue && value.startsWith('javascript:') && toRemove.push(name); const codeParser = parserCode ? em?.Parser?.getParserCode(parserCode) : undefined;
const { asDocument } = options;
if (parserCode) {
if (!codeParser) throw new Error(`Parser code "${parserCode}" not found`);
const parsedNode = codeParser.parse(input, { editor: em?.getEditor()!, options });
const parsedNodes = isArray(parsedNode) ? parsedNode : [parsedNode];
return {
root: asDocument ? this.__normalizeDocumentRoot(parsedNodes) : this.__createFragmentRoot(parsedNodes),
isParsedMode: true,
};
}
const parseRes = isFunction(cf.parserHtml) ? cf.parserHtml(input, options) : BrowserParserHtml(input, options);
return {
root: asDocument
? this.__domDocumentToParsedNode(parseRes as Document)
: this.__domRootToFragment(parseRes as HTMLElement),
isParsedMode: false,
};
},
__getParserCodeId(options: HTMLParserOptions) {
if (hasOwn(options, 'parserCode')) return options.parserCode || '';
return em?.Parser?.parserCode || config.parserCode || '';
},
__cloneParsedNode(node: ParsedNode): ParsedNodeMeta {
const cloned: ParsedNodeMeta = {
nodeType: node.nodeType,
};
node.tagName && (cloned.tagName = node.tagName);
node.namespaceURI && (cloned.namespaceURI = node.namespaceURI);
node.textContent !== undefined && (cloned.textContent = node.textContent);
node.attributes && (cloned.attributes = { ...node.attributes });
node.childNodes && (cloned.childNodes = node.childNodes.map((child) => this.__cloneParsedNode(child)));
const meta = node as ParsedNodeMeta & { doctype?: string; selfClosing?: boolean };
meta.__doctype && (cloned.__doctype = meta.__doctype);
meta.doctype && (cloned.__doctype = meta.doctype);
meta.__selfClosing && (cloned.__selfClosing = meta.__selfClosing);
meta.selfClosing && (cloned.__selfClosing = meta.selfClosing);
meta.__boolAttributes && (cloned.__boolAttributes = [...meta.__boolAttributes]);
return cloned;
},
__domRootToFragment(root: HTMLElement) {
return {
nodeType: ParsedNodeType.fragment,
__domNode: root,
childNodes: this.__getDomChildNodes(root).map((node) => this.__domToParsedNode(node)),
} as ParsedNodeMeta;
},
__domDocumentToParsedNode(doc: Document): ParsedNodeMeta {
return {
nodeType: ParsedNodeType.document,
__domNode: doc,
__doctype: doctypeToString(doc.doctype),
childNodes: doc.documentElement ? [this.__domToParsedNode(doc.documentElement)] : [],
};
},
__domToParsedNode(node: Node): ParsedNodeMeta {
if (node.nodeType === ParsedNodeType.document) {
return this.__domDocumentToParsedNode(node as Document);
}
const parsedNode: ParsedNodeMeta = {
nodeType: node.nodeType,
__domNode: node,
};
if (node.nodeType === ParsedNodeType.text || node.nodeType === ParsedNodeType.comment) {
parsedNode.textContent = node.textContent ?? '';
}
if (node.nodeType === ParsedNodeType.element) {
const el = node as HTMLElement;
parsedNode.tagName = el.tagName || '';
parsedNode.namespaceURI = el.namespaceURI || undefined;
const attrs = el.attributes || [];
if (attrs.length) {
parsedNode.attributes = {};
}
const boolAttributes: string[] = [];
for (let i = 0; i < attrs.length; i++) {
const attr = attrs[i];
parsedNode.attributes![attr.nodeName] = attr.nodeValue || '';
if (attr.nodeValue === '' && (el as any)[attr.nodeName] === true) {
boolAttributes.push(attr.nodeName);
}
}
boolAttributes.length && (parsedNode.__boolAttributes = boolAttributes);
const childNodes = this.__getDomChildNodes(el);
childNodes.length && (parsedNode.childNodes = childNodes.map((child) => this.__domToParsedNode(child)));
parsedNode.__selfClosing = `${el.outerHTML || ''}`.slice(-2) === '/>';
}
return parsedNode;
},
__getDomChildNodes(node: Node) {
const template = node as HTMLTemplateElement;
const childNodes = template.content?.childNodes || node.childNodes || [];
return Array.from(childNodes);
},
__createElementNode(tagName: string, childNodes: ParsedNodeMeta[] = []) {
return {
nodeType: ParsedNodeType.element,
tagName,
namespaceURI: ParsedNodeNamespace.html,
childNodes,
} as ParsedNodeMeta;
},
__createFragmentRoot(nodes: ParsedNode[]) {
return {
nodeType: ParsedNodeType.fragment,
childNodes: nodes.map((node) => this.__cloneParsedNode(node)),
} as ParsedNodeMeta;
},
__normalizeDocumentRoot(nodes: ParsedNode[]) {
const flatNodes = nodes
.map((node) => this.__cloneParsedNode(node))
.flatMap((node) => (node.nodeType === ParsedNodeType.fragment ? getNodeChildNodes(node) : [node]));
const documentNode = flatNodes.find((node) => node.nodeType === ParsedNodeType.document);
if (documentNode) {
return documentNode;
}
const htmlNode = flatNodes.find(
(node) => node.nodeType === ParsedNodeType.element && getNodeTagName(node) === 'html',
);
const documentRoot: ParsedNodeMeta = {
nodeType: ParsedNodeType.document,
childNodes: [],
};
if (htmlNode) {
const extraNodes = flatNodes.filter((node) => node !== htmlNode);
if (extraNodes.length) {
const bodyNode = this.__findChildElement(htmlNode, 'body') || this.__appendChildElement(htmlNode, 'body');
bodyNode.childNodes = [...getNodeChildNodes(bodyNode), ...extraNodes];
}
documentRoot.childNodes = [htmlNode];
return documentRoot;
}
const remaining = [...flatNodes];
const headIndex = remaining.findIndex(
(node) => node.nodeType === ParsedNodeType.element && getNodeTagName(node) === 'head',
);
const bodyIndex = remaining.findIndex(
(node) => node.nodeType === ParsedNodeType.element && getNodeTagName(node) === 'body',
);
const headNode = headIndex >= 0 ? remaining.splice(headIndex, 1)[0] : undefined;
const bodyNode =
bodyIndex >= 0
? remaining.splice(bodyIndex > headIndex && headIndex >= 0 ? bodyIndex - 1 : bodyIndex, 1)[0]
: undefined;
const htmlRoot = this.__createElementNode('html');
const htmlChildren: ParsedNodeMeta[] = [];
headNode && htmlChildren.push(headNode);
const normalizedBody = bodyNode || this.__createElementNode('body');
normalizedBody.childNodes = [...getNodeChildNodes(normalizedBody), ...remaining];
htmlChildren.push(normalizedBody);
htmlRoot.childNodes = htmlChildren;
documentRoot.childNodes = [htmlRoot];
return documentRoot;
},
__appendChildElement(node: ParsedNodeMeta, tagName: string) {
const child = this.__createElementNode(tagName);
node.childNodes = [...getNodeChildNodes(node), child];
return child;
},
__findChildElement(node: ParsedNodeMeta, tagName: string) {
return getNodeChildNodes(node).find((child) => getNodeTagName(child) === tagName);
},
__removeElementNodes(root: ParsedNodeMeta, tagName: string) {
const removed: ParsedNodeMeta[] = [];
const remove = (node: ParsedNodeMeta) => {
if (!node.childNodes?.length) return;
const nextNodes: ParsedNodeMeta[] = [];
node.childNodes.forEach((child) => {
if (getNodeTagName(child) === tagName) {
removed.push(child);
return;
}
remove(child);
nextNodes.push(child);
});
node.childNodes = nextNodes;
};
remove(root);
return removed;
},
__getNodeTextContent(node: ParsedNodeMeta): string {
if (node.nodeType === ParsedNodeType.text || node.nodeType === ParsedNodeType.comment) {
return node.textContent ?? '';
}
if (node.textContent && !node.childNodes?.length) {
return node.textContent;
}
return getNodeChildNodes(node)
.map((child) => this.__getNodeTextContent(child))
.join('');
},
__sanitizeNode(node: ParsedNodeMeta, opts: HTMLParserOptions) {
const attrs = node.attributes || {};
const cleanAttrs: Record<string, string> = {};
each(attrs, (value, name) => {
const attrValue = `${value}`;
const isUnsafeAttr = !opts.allowUnsafeAttr && name.startsWith('on');
const isUnsafeValue = !opts.allowUnsafeAttrValue && attrValue.startsWith('javascript:');
if (!isUnsafeAttr && !isUnsafeValue) {
cleanAttrs[name] = attrValue;
}
}); });
toRemove.map((name) => node.removeAttribute(name));
each(nodes, (node) => this.__sanitizeNode(node as HTMLElement, opts)); if (Object.keys(cleanAttrs).length) {
node.attributes = cleanAttrs;
} else {
delete node.attributes;
}
each(getNodeChildNodes(node), (child) => this.__sanitizeNode(child, opts));
}, },
__checkAsDocument(str: string, opts: HTMLParserOptions) { __checkAsDocument(str: string, opts: HTMLParserOptions) {

12
packages/core/src/parser/model/SyntheticElement.ts

@ -1,14 +1,14 @@
import { ParsedNode, SyntheticElementCtor } from '../types'; import { ParsedNode, ParsedNodeNamespace, ParsedNodeType, SyntheticElementCtor } from '../types';
const HTML_NAMESPACE = 'http://www.w3.org/1999/xhtml';
const getTagName = (node: ParsedNode) => { const getTagName = (node: ParsedNode) => {
const tagName = node.tagName || ''; const tagName = node.tagName || '';
return tagName && (!node.namespaceURI || node.namespaceURI === HTML_NAMESPACE) ? tagName.toUpperCase() : tagName; return tagName && (!node.namespaceURI || node.namespaceURI === ParsedNodeNamespace.html)
? tagName.toUpperCase()
: tagName;
}; };
const getNodeTextContent = (node: ParsedNode): string => { const getNodeTextContent = (node: ParsedNode): string => {
if (node.nodeType === 3 || node.nodeType === 8) { if (node.nodeType === ParsedNodeType.text || node.nodeType === ParsedNodeType.comment) {
return node.textContent ?? ''; return node.textContent ?? '';
} }
@ -80,7 +80,7 @@ export class SyntheticElement {
} }
get nodeValue() { get nodeValue() {
return this.nodeType === 3 || this.nodeType === 8 ? this.textContent : null; return this.nodeType === ParsedNodeType.text || this.nodeType === ParsedNodeType.comment ? this.textContent : null;
} }
get className() { get className() {

19
packages/core/src/parser/types.ts

@ -20,6 +20,25 @@ export interface ParsedNode {
textContent?: string; textContent?: string;
} }
export interface ParsedNodeMeta extends ParsedNode {
__boolAttributes?: string[];
__doctype?: string;
__domNode?: any;
__selfClosing?: boolean;
}
export enum ParsedNodeType {
element = 1,
text = 3,
comment = 8,
document = 9,
fragment = 11,
}
export enum ParsedNodeNamespace {
html = 'http://www.w3.org/1999/xhtml',
}
export type ParsedElementNode = ParsedNode & { tagName: string }; export type ParsedElementNode = ParsedNode & { tagName: string };
export interface CustomParserCodeContext { export interface CustomParserCodeContext {

Loading…
Cancel
Save