Features

See also

TextPipe FAQ
TextPipe screen shots
Export filter to clipboard
Export filter to web page
Custom web front-end demo
Audit logs sample
Data cleansing sample
Encoding conversions to and from Unicode
ASCII/ANSI/Code page conversions

Key Features - Summary

• Fix text or binary files of ANY size. Unlike competitor products, TextPipe does not attempt to read the entire file into memory first, so it can process files larger than the amount of memory you have.
• Process unlimited files and folders (with exclusion lists and subfolder depth settings), or the Windows clipboard
• Unmatched search and replace power. Read more
• EasyPattern text matching and perl patterns
• Can be scheduled for automated lights-out processing, with audit logs tracking every step
• Integrates with Windows Explorer and various third-party tools
• Developer API - can be scripted via a COM interface by other programs
• 145 Unicode, ANSI, ASCII and single byte and multi-byte conversion functions, and 151 code page conversions
• Roll your own custom filters using industry standard JScript and VBScript
• Restrict changes to lines, columns, fields, XML and HTML elements and more, eg just lines 10..30, columns 1-3, CSV fields 4 and 7, Tab-delimited fields 8..10, replacement text etc
• Wizard to assemble complex filters
• Filters can be exported to the clipboard for documentation purposes and for revision control in external applications. The export includes the hierarchical filter structure, file lists and comments.
• Filters can also be exported to the command line, to JScript, VBScript or a web page. This is very handy for automating processing jobs, and for designing customized front ends for TextPipe, making it easier for less technical users.
• Uses no temporary files (except for large sorts), so you save massive amounts of hard disk space
• Make multiple changes to multiple files in just ONE pass. Successive transforms can be applied in a user-defined order, such as multiple search-replace operations in a row, or an Extract Email Addresses filter followed by a Sort and Remove Duplicates filter etc
• Create and modify XML/CSV/Tab/Insert Script database extracts from OLE DB/ODBC databases
• No limit on line length. Unlike other products, lines are NOT limited to a fixed length
• Test and debug filters without affecting key data
• Five part tutorial introduces key concepts
• 100+ Transforms
• Library of more than 200 example filters you can link to
• Ultra-High Speed
• Click here for more technical details.

Key Features - Details

TextPipe can operate on files, the clipboard, or a trial run area with sample data, which is very handy for testing filters. TextPipe provides context menus (right-click menus) on every field to Cut/Copy/Paste/Clear/Select All, escape special characters, insert common patterns, find and replace, and save to and load from a file.

Filters can be inserted inside each other, or they can be linked at runtime to other filters, allowing common modules to be reused.

Processing Options

Input options:

• Skip binary files (those with less than 80% readable characters). The user can also be prompted whether to process a binary file or not
• Prompt before processing each file
• Prompt before processing read-only files
• Delete the input files after processing them
• Include or exclude subfolders (0-9 levels or All)
• Input from clipboard
• Input from Trial Run area
• Generate random input of a given size

Output options:

• Test mode where output goes to a new file rather than overwriting the original file
• Backup mode where original files are saved
• Only output modified files
• Retain the original file's time and date
• Append to existing output files
• Change output file's extension
• Change output file's folder
• Clipboard output (all output is merged to the clipboard)
• Merge all output into one file

Logging:

• Logging can overwrite or append to an existing file
• Logging can be disabled if there is less than a given number of lines

Text Manipulation Filters

TextPipe provides a huge range of filters for manipulating text. TextPipe's unmatched power comes from it's unique architecture and its tremendous flexibility in combining and arranging these filters to suit each task.

Filters to convert text:

• Convert between Unix, PC, Mac and Mainframe end of line and fixed width record formats. Conversion can detect incoming line feeds, and insert new characters such as line feeds between fixed-length records. Invalid End of Line characters can be automatically removed
• Convert unprintable IBM drawing characters to + and | and -
• Convert tabs to spaces or spaces to tabs
• Convert character case to UPPERCASE, lowercase, tOGGLE cASE, Title Case, Sentence case, rANdoM cASE
• Convert character collating sequence from ASCII to EBCDIC or EBCDIC to ASCII. Expand EBCDIC packed or zoned decimal, compress to EBCDIC packed or zoned decimal. Useful for handling mainframe files
• Convert ASCII (Windows OEM) to ANSI and ANSI to ASCII. Useful with Windows and non-English languages
• Convert CSV to Tab-delimited, CSV to XML
• Convert Tab-delimited to CSV, Tab-delimited to XML
• Convert text to a word list
• Convert text to a Hex or Decimal dump (very useful for finding control characters)
• Convert Word documents to text
• Perform mathematical functions on characters (useful for custom ciphers)

Unicode functions:

• Conversions to and from 144 different Unicode formats, as well as ANSI/ASCII and a multitude of other formats
• Convert Unicode to ANSI and ANSI to Unicode
• Convert Unicode to ASCII and ASCII to Unicode
• Swap UTF-16 word order, swap UTF-32 word order
• Remove Byte Order Mark (BOM)
• Make Big Endian, make Little Endian
• Compose, Decompose and Normalize functions (NFC, NFD, NFKD, NFKC)

Filters to add text:

• Add line numbers, with control over the start number and increment, format, numbering of blank lines etc
• Add left margin to the start of each line (e.g. to add reply quotes to email). The text can optionally come from an external file
• Add right margin to the end of each line
• Add header text to the start of each file
• Add footer text to the end of each file
• Add a column of text to every line at a specified column position
• Add bytes to a file at a given position
• Join two files side by side (repeating or non-repeating)
• Word wrap text at a specified column width
• Pad lines out with spaces to a specified column width (useful for generating fixed width records)
• Truncate lines at a specified column width (useful for generating fixed width records)
• Centre text in a specified column width
• Right Justify text in a specified column width
• Force lines to a specified column width (useful for generating fixed width records)

Filters to remove text:

• Remove blanks from start of line
• Remove blanks from end of line
• Remove blank lines
• Remove multiple whitespace to a single space (useful for processing web pages)
• Remove HTML, XML and SGML codes
• Remove or resolve backspaces
• Remove a line range
• Remove lines matching or not matching a pattern
• Remove lines from a list
• Remove lines from a list of patterns
• Remove duplicate lines
• Remove lines or characters from the start or end of the file (like Unix head/tail)
• Remove columns of text from each line
• Remove delimited fields (CSV, Tab, Pipe etc)
• Remove binary characters
• Remove ANSI escape sequences
• Remove all (used for its side effects)

Filters for email:

• Extract email address - for mailing lists, etc
• Extract URLs and links (for web site spidering)
• Remove email headers
• ROT13 encoding/decoding
• Hex encode/decode
• MIME Base 64 encode/decode
• MIME quoted printable encode/decode
• UUencode/UUdecode
• XXencode/XXdecode
• HTTP encode/decode - useful with CGI scripts and web pages

Filters for special tasks:

• Search and replace - exact match, multi-line, sounds-like, fuzzy match (detects spelling mistakes), binary characters, perl and egrep patterns, MS Word patterns, Brief patterns, find inside HTML tags, find between HTML tags, find between any text. Lists of search/replace pairs can be loaded from Excel, CSV and Tab-delimited files
• Extract matching text
• Send text as a SQL command to an OLE DB/ODBC database and receive the results as CSV, fixed width, XML or database insert script, with or without header information (very handy for database extracts)
• Sort lines. Options include ascending and descending ANSI and ASCII sort, sort by length of line, numeric sort, date/time sorts. The start column and length can now be specified. Handles files of unlimited size
• Run an external program over the text
• Run scripts written in VBScript, JScript or any scripting language
• Comment filters to document the filter
• Debug filter to show text at each point in the processing
• Split files at a particular size, or after a particular character has appeared a defined number of times, or after a defined number of lines.
• Join (merge) files together. e.g. Merge files to the clipboard.
• Secondary output filters (a Unix T-filter) to output the text at different stages
• Count duplicates or remove duplicates
• Retain lines matching or not matching a pattern
• Retain lines from a list
• Retain lines from a list of patterns
• Move columns or delimited fields (CSV, Tab, Pipe etc)
• Copy columns or delimited fields (CSV, Tab, Pipe etc)
• Repeat a file (duplicating its text)
• Randomize line order - randomize lines in file
• Reverse each line
• Reverse line order
• XSLT filter.

Filters to map single characters or Unicode characters to a new string:

• User-defined mapping. The user can specify a string to be output for each of the 256 possible character codes. Strings can contain embedded control characters. Unicode Maps are also supported.
• UTF-16 maps
• UTF-32 maps.

Filters to restrict changes to subsets of text:

• Restrict matching lines
• Restrict non-matching lines
• Restrict each line in turn (used for its side effects)
• Restrict CSV fields
• Restrict Tab fields
• Restrict line range
• Restrict column range
• Restrict blocks

TextPipe also include over a collection of over 100 demonstration filters and maps for common tasks.

TextPipe is simple to install, and no configuration is required before using it.