The Crux platform is designed to streamline and simplify your data workflows. While it offers broad support for various data sources, file formats, and configuration options (e.g., delimiters and row separators), a few considerations must be made for optimal use. For example, encryption is not yet supported, and there are some specific constraints on file patterns, sizes, and schema versions.
Review these details to ensure your data onboarding process runs smoothly. We're continually enhancing the platform, and most use cases are well-supported. If you have specific needs, our team is ready to help.
Feature | Limitation |
Source | FTP, SFTP, GCS, S3 |
File Format | Delimited text, Avro (flat), Parquet (flat) |
Auto-detected Column Delimiter | Single ASCII character |
User-specified Delimiter | ASCII strings (max 5 characters) |
Auto-detected Row Separator | Newline ( |
User-specified Row Separator | ASCII strings (max 5 characters) |
Supported format: ZIP, TAR, TAR.GZ. | |
Encryption | Not supported |
File Patterns | Max 25 per data product |
Count of Files in Pattern | Max 5k files |
Individual File Size | Max 1.5 GB uncompressed |
Total Size of File Pattern | Max 50 GB (inclusive of all historical deliveries) |
Schema Version | Max 5 total changes per table during onboarding |
Table in a File | 1 table only |
Schedule | 1 schedule per pattern |
Pattern Type | 1 file structure per matched resources |