Corpus tools developed by members of BFSU Corpus Research Group
Concordancers and query tools (语料库检索工具)
- BFSU PowerConc 1.0 beta 21c: A freeware concordancer for Windows (1.8MB).
- BFSU PowerConc 1.0 beta25b.
- BFSU CQPweb online concordancer (download CQPweb tutorial here.)
- BFSU ParaConc 1.2.2: A freeware parallel concordancer (4.6MB)
- Colligator 2.0: A colligation query and analysis tool (1.4MB)
- SearchSubtitle: A programme for video based time-aligned subtitle concordancing (Chinese user interface). The tool was designed by Wenzhong Li and programmed by Zhaoyang Han (533KB).
- PatCount 1.0: PatCount is the abbreviated form of 'pattern counting'. It is a query tool of counting the frequency of lexical, syntactic, and discoursal features in texts. The result of the tool is shown and can be exported as 'feature(s) x text(s)' matrices, which is most suitable for follow-up advanced (inferential) statistical analyses. Regular expressions are fully supported in the tool. Microsoft .Net framework is required before you run the tool. The tool was designed by Maocheng Liang and Wenxin Xiong and programmed by Wenxin Xiong (3.6MB).
Annotation tools (语料库标注工具)
- TreeTagger for Windows 3.0 (multilingual edition) (47.5MB)
- TreeTagger for Windows 3.0 Lite (English tagging only) (14.5MB)
- TreeTagger for Windows 2.0 (multilingual edition) (17.6MB)
- BFSU Stanford POS Tagger (two tagging models) (13.3MB)
- BFSU Stanford POS Tagger Lite (one tagging model, left3words) (4.1MB)
- BFSU Stanford Parser (4.8MB)
- BFSU Qualitative Coder 1.2 (728KB). The tool assists manual annotation based on a user-defined category template (e.g. taxonomy of speech acts). A semi-automatic feature is also enabled given some heuristic knowledge, e.g. a thesaurus, or recurrent lexico-grammar on the basis of preliminary human annotation.
- YACSI Chinese tokeniser and POS tagger version 0.96 (8.4MB). Version 0.96 is a more stable release, your system time has to be set to a date earlier than 2012 though.
- YACSI Chinese tokeniser and POS tagger version 1.0 (a Windows GUI of ICTCLAS2015 with a Chinese user interface and an English user interface and supports input texts of both ANSI/GBK and UTF-8 encodings. A bug has been spotted that the tool stops working in the middle of some texts. Please swap to version 0.96 if this happens in your work.) (9.6 MB)
- KAT tool (or KWIC-based Annotation Tool): The tool enables users to automatically group the patterns of a search term based on a couple of human identified patterns. A text file is needed for the annotation tool to proceed. The text file or scheme file contains lables (one lable per line) to which users would like to apply to the patterns identified out of conconcordance lines.
- DeTagging Tool (414KB): The tool helps to strip off tags of four different formats (namely, underscore, forward slash, angle brackets and square brackets) in annotated texts.
Statistical tools for corpus analyis (语料库统计工具)
- Chi-square and Log Likelihood Calculator (1.1MB)
- Log Likelihood Calculation Excel Spreadsheet (40KB) (see also Paul Rayson's log likelihood calculator and Excel Spreadsheet)
- Fisher's Exact Test p-value Calculation Excel Spreadsheet (118KB) with chi-square test score, effect size metrics of relative risk and odds ratio scores.
Specialised corpus tools (语料库分析专用工具)
- BFSU Collocator (835KB) is a search-based collocation extration tool which yields MI, MI3, T-score, Z-score, Log-Log and Log likelihood scores of collocational strength. The tool works will raw and CLAWS-tagged PoS English texts, and does not work for texts of Chinese or other languages.
- BFSU English Sentence Segmenter 1.1 (447KB)
- Concordance Randomizer 1.2 (531KB)
- The Edinburgh Associative Thesaurus query tool (EAT) (3.9MB)
- Keywords Plus 1.0 (1.87 MB) (an earlier release of Keywords Plus tool in which the resulting keywords are linked to their original concordance lines. This feature has not been retained in version 2.)
- Keywords Plus 2.0 (5.67 MB) (a free keyword generation tool based on the comparison of two corpora or wordlists. The tool is helpful of creating Chinese and English keyword lists, key ngram lists and key POSgram lists.)
- Pattern Builder (7.2 MB) is an aid for those who are not familiar with regular expressions in searching PoS-tagged English texts.
- Readability Analyzer 1.0 (1.1MB): A tool which yields Readability indices, type/token ratio (TTR), standandarised type/token ratio (STTR), lemmatised TTR, lemmatised STTR, average word length, average sentence length, etc.
- Readability Analyzer 1.1. This version fixes the bug that some users can't save the results. A known issue for the current build is that the functionalities from Reading Ease to AWL do not work properly.
- Sub-corpus Creator (2.9MB): Sub-corpora can be extracted based on the text strings contained in filenames of texts or in-text metadata markup.
- Text Cleaning Library for PowerGREP (5.5KB)
- TextSmith Tools (6.1MB): This tool showcases a methodological innovation of a genre-informed phraseological profile across the discourse segments. TextSmith segments texts by equal proportion, based on the users’ own intuitive estimation of the sections the imported texts might contain.
Data driven learning tools and resources (数据驱动学习工具)
- BFSU Sentence Collector is a pedagogically motivated concordancing tool which allows users to refine search results according to sentence length and lexical difficulty. The results of the tool are displayed in complete sentences instead of the KWIC mode. To customise your own textual data for text collection. Please first of all segment the English texts on your own hard drive with BFSU Sentence Segmenter 1.0, and then mark up the unknown/new words based on a base word list with BFSU NewWords Marker 1.0 and save the data as an *.idx file into the index folder of BFSU Sentence Collector.
Useful tools and resources that were not developed by BFSU CRG members
- Batch Encoding Converter (Chinese interface) (985KB)
- Chinese character to romanisation pinyin conversion tool (汉字转拼音工具 Chinese interface) (1.3MB)
- Filename Tool (文件名工具 Chinese interface) (1.1 MB)
- Flesh PC: Standalone tool for calculating Flesch Reading ease, Flesch-Kincaid Grade Level and other descriptive text measures. (4.7MB)
- Duometer is a command-line tool that allows to efficiently identify near-duplicate pairs of documents in large collections of texts. It works on all platforms with Java runtime installed.
- Multidimensional Analysis Tagger 1.1 by Andrea Nini (official site here) (9.7MB). Read the MAT v. 1.1 Manual.
- Multidimensional Analysis Tagger 1.2 by Andrea Nini (official site here) (9.96MB). Read the MAT v. 1.2 Manual.
- Multidimensional Analysis Tagger 1.3 by Andrea Nini (official site here) (9.8MB). Read the MAT v. 1.3 Manual.
- PDF2TXT conversion tool (PDF转TXT格式软件, Chinese interface) (2.1MB).
- Qt Readability: The tool provides values for Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning-Fog Readability Index, SMOG, and Coleman-Liau Readability Index as well as statistics of words, sentences, and syllables used in the five formulae. (5.4MB)
- Renamer and Others (Chinese interface) (1.2MB)
- Super Batch Renamer (批量改名工具 English interface freeware) (316KB)
- UTFCastExpress: Batch UTF-8 conversion from ANSI (993KB)
- WCopyfind: A programme for detecting duplicated files or plagiarism (文本查重工具 1.9MB)
- Wordaizer: From text to word cloud with a twist (词云工具 8.5MB)
- WordCreator: With the WordCreator, you can create random words or whole texts from letters, syllables, single words or sentence fragments which can in addition be weighted by probabilities. In addition, many additional functions, such as counting of characters, bi- and trigrams or real syllables, are integrated. Official site. (740KB)
- WordSmith Tools V4.0 is freely downloadable now.
- Writefull client. This tool provides concordance, near-synonym comparison and other features based on Google Books, Google Scholars, Google News, and Google Web. (32.6 MB)
- Zipfian Curve Excel Spreadsheet by (dzhigner) Zheng Ding of Luoyang Teachers' College (920KB)