数据仓库服务 GAUSSDB(DWS)-解析器测试

时间：2024-11-28 17:38:55

解析器测试

函数ts_parse可以直接测试文本搜索解析器。

     
          ts_parse(parser_name text, document text,
         OUT tokid integer, OUT token text) returns setof record

ts_parse解析指定的document并返回一系列的记录，一条记录代表一个解析生成的token。每条记录包括标识token类型的tokid，及token文本。比如：

     
          SELECT * FROM ts_parse('default', '123 - a number');
 tokid | token
-------+--------
    22 | 123
    12 |
    12 | -
     1 | a
    12 |
     1 | number
(6 rows)

     
          ts_token_type(parser_name text, OUT tokid integer,
              OUT alias text, OUT description text) returns setof record

ts_token_type返回一个表，这个表描述了指定解析器可以识别的每种token类型。对于每个token类型，表中给出了整数类型的tokid--用于解析器标记对应的token类型；alias——命名分词器命令中的token类型；及简单描述。比如：

     
      
        
        
          SELECT * FROM ts_token_type('default');
 tokid |      alias      |               description                
-------+-----------------+------------------------------------------
| asciiword       | Word, all ASCII
| word            | Word, all letters
| numword         | Word, letters and digits
| email           | Email address
| url             | URL
| host            | Host
| sfloat          | Scientific notation
| version         | Version number
| hword_numpart   | Hyphenated word part, letters and digits
| hword_part      | Hyphenated word part, all letters
| hword_asciipart | Hyphenated word part, all ASCII
| blank           | Space symbols
| tag             | XML tag
| protocol        | Protocol head
| numhword        | Hyphenated word, letters and digits
| asciihword      | Hyphenated word, all ASCII
| hword           | Hyphenated word, all letters
| url_path        | URL path
| file            | File or path name
| float           | Decimal notation
| int             | Signed integer
| uint            | Unsigned integer
| entity          | XML entity
(23 rows)

         

       

     
    