Recognition
With recognition, Docuflair reads out the content of a scanned document automatically — much as a person reads a printed sheet. Text (by OCR, optical character recognition) and barcodes can be found on the pages. The information recognised can then be used as the file name or for sorting, for example.
Permission required
To access this function you need the following permission:
- Show recognitions (
AdminUI_WorkflowManagement_ListRecognitions)
You grant permissions through groups — see permissions – overview.
Recognition searches the pages of a scanned document for information, using text and image recognition techniques. A page can be searched for the following information:
- Barcodes (1D and 2D)
- Text
- Form types
- Machine-readable travel documents (MRTDs)
Docuflair lets you define any number of recognitions, and the area searched can be restricted when searching for text and barcodes. Note that the processing time may increase with every recognition, particularly where text is searched for. A click on Create search action sets up a new recognition.
So that several recognitions can be told apart, every search result has to be assigned to a variable name. You can draw on that variable name in later processing steps, in the output and the saving, and use it to check conditions or to write metadata.
- Variables in Docuflair serve to manage data. Think of a variable as a container whose content can change.
- To picture it, imagine a parcel. The parcel itself is the variable. The content of the parcel can be changed at will — in the case of Docuflair, the parcel (the variable) is filled with the passages of text or the barcodes recognised. The content of a variable is called its “value”.
- A typical example of variables in programming is user information. Names, addresses and telephone numbers are stored there, for instance.
A variable name valid in Docuflair begins with a letter and may continue with any alphanumeric characters and digits, up to a maximum length of 32 characters. Special characters and umlauts are not permitted. Upper and lower case are distinguished.
Valid variable names are, for example:
- myBarcode
- MyBarcode
- My1Barcode
- MyBarcode1
- barcode
- barcode2020
Invalid names are, for example:
- myVariableHasAFarTooLongNameAndWillNotWork33
- 33noDigitAtTheStart
- No/SpecialCharacters
- NorAny-Minus
- .inadmissibleBecauseOfTheDotAndUmlauts
The search for barcodes and text allows filtering through regular expressions. Regular expressions let you filter out information that is not needed or not wanted. A requirement of that kind could arise from the following situations:
- Delivery notes are to be processed and a barcode read out for that purpose. Awkwardly, though, several barcodes are detected, because the logistics firm prints barcodes as well. Regular expressions let you filter and admit only barcodes of a particular length or with a particular prefix.
- An incoming document from the internal post is to be checked for whether it is an invoice or a reminder. Regular expressions let you filter out every other word that is not relevant for the processing that follows.
Regular expressions are above all a kind of filter criterion for text: the regular expression in question is matched against the text in the form of a pattern. That makes it possible to search for every word beginning with D and ending in R, for example, without having to give the letters in between explicitly.
The following rules apply to building regular expressions:
| Expression | Description |
|---|---|
| . (dot) | Exactly ONE character of any kind. It can be an A or a 9 or a space and so on. But some character has to come. Exactly one. |
| - (minus) | Either as a range, [A-Z] for example – a capital letter A to Z, OR, where it stands at the beginning or the end of the REGEXP, simply a minus (the same as \-) |
| So [A-Z] means every capital letter from A to Z, [-A-Z] means the minus sign AND every capital letter from A to Z, and [A-Z-] likewise means every capital letter from A to Z and the minus sign. | |
| Where the minus sign does not stand at the beginning, it has to be marked with \. | |
| \ (backslash) | Cancels the meta functions of meta characters — when searching for a dot in the text, for instance. Since the dot as such has a meta function (a dot stands in for exactly one character of any kind, whatever it is), a search for . would bring every possible result. Where a dot really is to be searched for, the dot has to be written as . |
| ^ | Where it stands inside square brackets, it negates their content: [^d-z] means no lower-case letters between d and z. Otherwise it is the start of a string or the start of a line (^a therefore means that a has to stand at the beginning). |
| $ | The end of a string or the end of a line: on “toller”, “r\(” matches but an “a\)” does not. |
| + | The character before the plus has to occur at least once and at most x times: a+ for a, aa, aaa… |
| * | The character before the asterisk has to occur from 0 to x times: a* stands for nothing (0 occurrences) through to aaaaaaaa…, x as often as you like |
| ? | The character before the ? has to occur 0 or 1 times: “h..?se” finds ‘hose’ but also ‘house’ |
| {x,y} | The frequency of the occurrence: x to y times. \d{2,4) means 2 to 4 digits, {2,} means at least twice, {3} means exactly three times. |
| [] | Exactly one character out of those standing between the brackets: [abc] searches for occurrences of a, b or c (so not the string “abc”, but exactly one letter a or b or c. ONE letter is called for). |
| () | Round brackets for grouping expressions, ^(abc for example |
| \d | Exactly one digit [0-9] (d as in digit) |
| \D | A character that is not a digit, so [^\d] |
| \w | Exactly one letter, digit or underscore, so [a-zA-Z_0-9] (and possibly further letters, umlauts for example) |
| \W | A character that is neither a letter nor a digit nor an underscore, so [^\w] |
| \s | Whitespace; usually the class of the control characters \f, \n, \r, \t and \v |
| \S | A character that is not whitespace [^\s] |
| \b | An empty string at the start or the end of a word (recognises the start of a word): ‘\bthis or that’ finds ‘this or that’ but not ‘nothing like this or that’ |
| \B | An empty string that forms neither the start nor the end of a word |
| \< | An empty string at the start of a word |
| \> | An empty string at the end of a word |
| \n | A line break (in the Unix format) |
| \t | A tab |
Docuflair supports various modes for how regular expressions are processed:
| Mode | Description |
|---|---|
| None | The search for regular expressions is not carried out. The content recognised is assigned to the target variable unfiltered. |
| Ignore case | Searches for regular expressions but ignores differences between upper and lower case |
| Multiline | Carries out a check through regular expressions even where a line break is present in the text recognised |
| Explicit capture | Excludes the capture of unnamed groups. The only captures valid are explicitly named or numbered groups of the form (?\ |
| Singleline | Single-line mode, in which the dot (.) matches every character (instead of every character except a line break). |
| Ignore whitespace in the pattern | Excludes superfluous whitespace from the pattern and enables comments after a number sign (#). |
| Right to left | Changes the direction of the search. The search moves from right to left instead of from left to right. |
| Ignore cultural differences between languages | Ignores cultural differences in the language. |
Searching for barcodes
Section titled “Searching for barcodes”Pattern recognition is used in the search for barcodes, since scanned documents are normally present only as image data. Pattern recognition may increase the processing time slightly, because algorithms for image enhancement and recognition are run in several internal working steps.
The term “barcode” stands below for every 1D and 2D barcode, so it covers QR codes, Aztec codes and so on as well.
Having clicked Create search action, choose Barcode / QR code as the search type and give the name of the variable in the variable name field — MyBarcode, for example. As described above, the value of the barcode recognised is deposited in that variable and can be reused for the processing and output that follow. Where several barcodes are found, all of them are stored in that variable.
In the search subcategory list you determine whether every barcode type is to be searched for or only particular 1D or 2D barcodes. The list of the barcode types supported is here.
With the Search the whole page option switched on, barcodes are looked for everywhere. Switch the option off and you can restrict the area and search only in the header or the footer, for example. Since Docuflair lets you define any number of recognitions, two or more recognitions can be configured — where both the footer and the header are needed — each with a different area and a unique variable name.
The regular expression mode option determines whether and how filtering through regular expressions (see above) takes place. It allows passages of text that are not wanted or not needed to be removed before the result is assigned to the variable. In the regular expression field, enter the pattern the text is to be filtered with.
Searching for text
Section titled “Searching for text”OCR (optical character recognition) is used in the search for text, since scanned documents are normally present only as image data. Text recognition can be very time-consuming, because algorithms for image enhancement and recognition are run in several internal working steps. For high-quality results, a good OCR system has to be able to adjust the most important parameters of every single algorithm, even where only a few characteristics of the images are known in advance. Sometimes configuring the preprocessing by hand is the only way to improve the quality of the recognition. To this day there are no OCR systems that come close to a human being’s recognition performance, even though the proportion of characters recognised wrongly or not at all has meanwhile been optimised down to a fraction of a per cent.
Having clicked Create search action, choose Text as the search type and give the name of the variable in the variable name field — MyText, for example. As described above, the text recognised is deposited in that variable and can be reused for the processing and output that follow.
With the Search the whole page option switched on, recognisable text is looked for everywhere. Switch the option off and you can restrict the area and search only in the header or the footer, for example. Since Docuflair lets you define any number of recognitions, two or more recognitions can be configured — where both the footer and the header are needed — each with a different area and a unique variable name.
With text background colour you search for every passage of text that has a coloured background — one marked with a coloured highlighter, for instance.
The regular expression mode option determines whether and how filtering through regular expressions (see above) takes place. It allows passages of text that are not wanted or not needed to be removed before the result is assigned to the variable. In the regular expression field, enter the pattern the text is to be filtered with.
The OCR language determines the character set and the dictionaries used for the text recognition. That detail is required because it makes the OCR deliver better results — English manages without umlauts, for example, whereas other special characters can occur in French.
With the OCR character set whitelist you restrict the set of characters the OCR can recognise. Where a document contains only digits, for instance, enter “0123456789” in this field. The OCR then attempts to assign every letter found to one of those characters.
The OCR character set blacklist defines characters the OCR may on no account recognise. Where a document never contains mathematical operators, for instance, enter “+-*/” in this field. The OCR then attempts to assign every letter found to some other character that is not in that list.
Searching for forms
Section titled “Searching for forms”
With the forms search type you search for form types taught earlier. Where a form taught is recognised by similarity search, the name of the form is assigned to the variable.
No further options are required for form recognition, because every relevant zone containing text or barcodes can already be defined while teaching Docuflair new forms.
The form recognition mode
Section titled “The form recognition mode”The form recognition mode determines how Docuflair searches for matching forms:
| Mode | Description |
|---|---|
| Use a static form | A particular form is assigned permanently. No automatic recognition — the form selected is applied to the document directly. The fastest processing. |
| Recognise the form automatically | Docuflair checks every form available and recognises automatically which fits best (the default). |
| Search only for forms with conditions | Only forms that have conditions defined are drawn on for the recognition. Faster than fully automatic recognition. |
Searching for MRTDs
Section titled “Searching for MRTDs”With the machine-readable travel documents (MRTD) search type you recognise and read out the machine-readable zone (MRZ) on passports, identity cards and other identity documents. The MRZ contains standardised information such as the given name, the surname, the document number, the nationality and the date of birth, in accordance with ISO/IEC 7501-1.
Having clicked Create search action, choose Machine-readable travel documents (MRTD) as the search type and give the name of the variable in the variable name field. The MRZ data recognised is broken down into individual fields (the given name and the surname, for example) automatically and assigned to the variable.
Unlike the other search types, MRTD recognition requires no further configuration options — the whole page is searched for the machine-readable zone automatically, and the data it contains is recognised and parsed by a specialised OCR engine.