Common Problems with CSV Import Experiences<\/h3>\n\n\n
Importing CSV data is often one of the first interactions users have with a software application, especially \u201cempty box\u201d products. Unfortunately, there are too many ways that this data import experience can cause customer frustration, or worse, churn.<\/p>\n\n\n\n
For users, an inefficient importer experience will cause them to question the value of the product itself.<\/p>\n\n\n\n
\u201cIf the app can\u2019t import my data easily now, what’s going to happen once my data is finally uploaded?\u201d<\/em> Your customers shouldn\u2019t have to battle with these kinds of CSV import errors (Source: Flatfile<\/a>)Digsy.ai <\/a>shared their experience with handling data imports for their real-estate CRM product prior to integrating Flatfile Portal. Not only was the team strained on resources from building and maintaining a proprietary data importer, but Digsy\u2019s engineers also spent ten or more hours per new user<\/strong> cleaning up and formatting incoming customer data. Occasionally, these users would churn, rendering all that work fruitless.<\/p>\n\n\n\n Thankfully, there’s an out-the-box CSV importer that can do all this on its own, and with just a few lines of code.<\/strong><\/p>\n\n\n We call it Flatfile Portal<\/a>, and it was born out of frustration from continuously re-building CSV importers, parsers, and uploaders. Flatfile provides data normalization, CSV auto-column matching, and a modernized UI component with a few lines of JavaScript. Customers implement Flatfile in as little as a day: a massive improvement over 3-4 engineering sprints with continuous maintenance tacked on.<\/p>\n\n\n\n An animation of an import completed with Flatfile Portal. (Source: Flatfile<\/a>)<\/p>\n\n\n\n Let\u2019s look at issues associated with traditional CSV import experiences and how Flatfile Portal addresses them.<\/p>\n\n\n Users tend to struggle with CSV imports, and usually have questions before the CSV is even uploaded. Here are questions users may ask during an import:<\/p>\n\n\n\n Unless your users spend a lot of time exporting and importing spreadsheets, they\u2019re not going to think about these situations until the moment they import their data.<\/p>\n\n\n\n It shouldn\u2019t be up to your users to read through intimidating data import documentation or watch a 15-minute tutorial on how to import spreadsheets into your product. Surprisingly, developers aren’t spared either! Although engineers understand the complexities of importing spreadsheets into an advanced system (like Microsoft Azure<\/a>), there is still an exhaustive amount of content they need to ingest before their first import even happens. A highly technical product like Microsoft Azure attempts to reasonably present developers with extensive documentation for importing user data. (Source: Microsoft Azure<\/a>)Your product experience should make it simple to import CSV data without requiring users to become data scientists. The same goes for the technical understanding required to build a CSV importer to begin with, a goal we\u2019ve dedicated reaching with Flatfile Portal.<\/p>\n\n\n\n Here\u2019s an example of a simplified CSV import solution for a CRM from Portal, created in less than an hour complete with complex data validation.<\/p>\n\n\n\n A demo of how Flatfile Portal helps software engineers quickly build a data importer with a few lines of JavaScript. (Source: Flatfile<\/a>)Portal is standardized, responsive and customizable to specific branding needs. With Flatfile, users will instantly know to:<\/p>\n\n\n\n There\u2019s no need to overwhelm the user with warnings about file encoding, incorrect date formats, or what fields are required. Portal solely focuses on importing CSV data from the user and making it a delightful experience during this first data onboarding touchpoint. Mapping columns and validating data will be completed at a later stage once the importer has matched the spreadsheet header columns using human-in-the-loop machine learning.<\/p>\n\n\n\n Portal\u2019s JS configuration allows data models to be replicated in minutes instead of weeks. The label matches a CSVs column name, and the key is how you\u2019d like the imported data to be saved in the JSON output. Portal also provides powerful validation options to work with any data model requirement and supports regex, data normalization, and server callbacks for those unique validation use-cases.<\/p>\n\n\n\n Once the data model has been built into the JavaScript config, all that needs to be done is to trigger the importer from within your product, typically by way of a button through a JavaScript call. If your application can execute JavaScript, you can integrate a truly modernized CSV importer, in minutes, and at a fraction of the cost.<\/p>\n\n\n\n With Flatfile Portal, you and your users won\u2019t have to worry about things like file sizes or encoding formats causing problems during import. Portal helps you manage imported data via the browser or through a server-side process, enabling you to split and upload large CSV and Excel files without dropping imported data.<\/p>\n\n\n\n A Flatfile demo that shows how you can reliably split and import data from multiple customer files. (Source: Flatfile<\/a>)Flatfile allows users to import CSV data from multiple files intuitively, without dropping data or doing manual splitting. In this demo, users are allowed to import spreadsheets containing three different sets of data. Not only will this help make their files more manageable, but it\u2019ll make it much easier for your product team to ingest and organize customer data on your backend into a consistent structure.<\/p>\n\n\n\n Customers shouldn\u2019t have to think here.<\/strong> CSV importers should be designed \u2014 error messages and all \u2014 to make data onboarding a quick and painless experience for users.<\/p>\n\n\n The next source of frustration often appears when poor column-matching functionality is in place.<\/p>\n\n\n\n As an example, let\u2019s say that a Mailchimp<\/a> user wants to import as many contact details into the email marketing software as they can. After all, it might be useful to have their business title or phone number on hand for future list segmentation.<\/p>\n\n\n\n However, initializing the import results in some data being skipped or dropped entirely:<\/p>\n\n\n\n This is how Mailchimp\u2019s CSV field-mapping system displays unmapped data. (Source: Mailchimp<\/a>)The app doesn\u2019t recognize three of the four spreadsheet columns in our file. In order to keep the unmatched column data, the user has to go through each field and manually assign a matching label that Mailchimp accepts.<\/p>\n\n\n\n This is the case with many products that ingest user data, especially CRMs. Not all data will be used during an import, however, the decision to allow custom field submissions, in this case, should be left to the user; rather than removing their data prematurely.<\/p>\n\n\n\n We know what you\u2019re thinking: \u201cWhy not just provide a pre-built spreadsheet template?\u201d However, this is hardly a solution to the problem and will only create more work and frustration for your users. Especially those that bring in thousands of rows of data, or have 40 column headers (yes, we\u2019ve seen it!)<\/p>\n\n\n\n The problem here lies within the customer data onboarding experience.<\/strong> Optimizing CSV import features within a product has been a difficult and expensive project to take on for product and engineering teams. That\u2019s where Flatfile\u2019s machine-learning, auto column-matching solution comes in handy.<\/p>\n\n\n\n Portal automatically learns which incoming fields are matched to which columns from each of your users. This results in a \u2018human in the loop\u2019 machine-learning experience that is truly unique to your product\u2019s data model. Portal will automatically match imported CSV columns to your data model based on user inputs and consistently learns over time.<\/p>\n\n\n\n The importer also caches column assignments regardless of session, so a user that uploads ten CSV files in a day will get most if not all of their columns matched automatically.<\/p>\n\n\n\n ClickUp<\/a>, powered by Flatfile Portal, has leveraged human-in-the-loop for its productivity web app.<\/p>\n\n\n\n The main data import modal for users that want to import tasks and projects into ClickUp. (Source: ClickUp, powered by Flatfile Portal)Portal is designed to help users of all technical expertise. Instructions are clearly provided as to what the user can upload, and the fields required. In addition, Portal\u2019s manual data entry feature allows users to preview what sort of data can be imported into productivity software. This helps users preserve as much data as possible, rather than realize too late that the data importer didn\u2019t recognize their columns and dropped the data out without any notice.<\/p>\n\n\n\n Configuration flags extend Portal\u2019s functionality even further. For example, the allowCustom flag specifies whether you\u2019d like users to add custom columns during the matching step. Using ClickUp as an example, one could add a column for \u201cBillable\u201d? As a boolean field to track whether a task is billable. Allowing users to place columns on the fly results in a shift of control not seen with legacy CSV importers \u2013 shaped by a product\u2019s unique data model requirements.<\/p>\n\n\n\n Portal\u2019s column matching step:<\/p>\n\n\n\n Portal asks users to identify column names for accurate mapping. (Source: ClickUp<\/a>)This step asks users to indicate where their column names live. This way, the importer can more effectively match them with its own or add labels if they\u2019re missing. Some CSV data may only contain values rather than column headers.<\/p>\n\n\n\n Next, users get a chance to confirm or reject Portal\u2019s column matching suggestions:<\/p>\n\n\n\n Portal uses a machine-learning column-matching system to automatically map users\u2019 imported data. (Source: ClickUp<\/a>)<\/p>\n\n\n\n On the left, users will find the columns and values they\u2019ve imported. The white tab to the right provides them with automated column matching within ClickUp\u2019s data model. So, \u201cTask\u201d will become \u201cTask name\u201d, \u201cAssignee\u201d will become \u201cTask assignee(s)\u201d, \u201cStatus\u201d remains as is and so on.<\/p>\n\n\n\n If one of the labels in the CSV doesn\u2019t have any match at all, the importer calls attention to it like this:<\/p>\n\n\n\n Flatfile Portal calls attention to labels or CSV values that don\u2019t have an exact match in a product. (Source: ClickUp<\/a>)<\/p>\n\n\n\n In this example, the Priority response of \u201cHigh\u201d was detected. But \u201cMedium\u201d was not. However, there\u2019s no need for the user to guess what the correct replacement should be. The importer provides relevant options like \u201cUrgent\u201d, \u201cNormal\u201d and \u201cLow\u201d to replace it with. No need to re-import their CSV, or change the cell value prior to importing the data.<\/p>\n\n\n\n Once all spreadsheet mappings have been resolved, the user can easily \u201cConfirm Mapping\u201d or discard the column altogether if it\u2019s proven unnecessary.<\/p>\n\n\n\n Users get a chance to confirm CSV labels and clean up their spreadsheet results before importing their data. (Source: ClickUp<\/a>)Finally, users get a chance to review any errors detected with their data:<\/p>\n\n\n\n Validation errors detected in ClickUp\u2019s data import appear in red. (Source: ClickUp<\/a>)In this ClickUp example, \u201cTask Name\u201d is triggered with the \u201cisRequired: true\u201d flag which requires users to submit data for that particular column. Whenever any validation fails in Portal, you have full control over how error messages are displayed to the user – all inline within the importer.<\/p>\n\n\n\n The \u201cOnly show rows with problems\u201d toggle in the left-hand corner makes error rows easy to spot and address quickly.<\/p>\n\n\n\n This keeps users from having to:<\/p>\n\n\n\n How do you set up this system of column-mapping and error detection? Portal does most of the work for engineers.<\/p>\n\n\n\n There\u2019s no need to build a custom data importer with Flatfile Portal. (Source: Flatfile<\/a>)Portal is configured via a JavaScript code snippet. Any labels, keys, or validation rules specified in this code will be reflected on the customer-facing importer. This allows for complex validation on things like phone numbers or normalizing multiple date formats:<\/p>\n\n\n\n This Portal demo provides the pre-written JavaScript code on the left and a sample of the importer output on the right.Flatfile Portal is truly a turn-key CSV importer built for SaaS applications. The critical part to integrating Portal is configuring the JS snippet to your product\u2019s required data model. In other words, tell Portal what data needs to upload from users, what is the correct value, and whether you\u2019d like them to add their own custom data.<\/p>\n\n\n\n Here\u2019s a JS snippet you might use to customize Portal for a basic contact list:<\/p>\n\n\n\n An example from Flatfile on how to configure the JavaScript with your own keys and labels for a basic contact list import. (Source: Flatfile<\/a>)You can then use validators to set strict rules for what can appear in the corresponding fields:<\/p>\n\n\n\n
<\/p>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-io-common-spreadsheet-import-errors-examples-photo-min.png?resize=1024%2C728&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\nIntroducing: The elegant import button for web apps<\/h3>\n\n\n
![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-csv-importer-for-web-apps-animation-photo.gif?resize=1280%2C720&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\nIssue 1: Unclear Guidance<\/h3>\n\n\n
<\/p>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-io-importing-data-into-microsoft-azure-training-example-photo-min.png?resize=1024%2C567&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-csv-importer-for-crm-applications-photo-min.png?resize=1024%2C705&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-multi-file-spreadsheet-imports-photo-min.png?resize=672%2C498&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\nIssue 2: Inefficient CSV Column-Mapping<\/h3>\n\n\n
![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-io-importing-csv-to-mailchimp-matching-photo-min.png?resize=1024%2C494&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-import-csv-to-clickup-photo-min.png?resize=1024%2C561&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"From](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-auto-column-mapping-clickup-column-headers-photo-min.png?resize=1024%2C561&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"From](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-clickup-column-matching-spreadsheet-headers-min.png?resize=1024%2C561&ssl=1\" "\\"\\"")
![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-clickup-column-matching-spreadsheet-validation-photo-min.png?resize=883%2C196&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"From](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-clickup-confirm-spreadsheet-header-column-mapping-photo-min.png?resize=1024%2C501&ssl=1\" "\\"\\"")
![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-clickup-review-imported-csv-errors-photo-min.png?resize=1024%2C388&ssl=1\" "\\"\\"")
![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-turn-messy-spreadsheets-into-product-data-photo-min.png?resize=1024%2C638&ssl=1\" "\\"\\"")
![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-demo-javascript-configuration-importer-photo-min.png?resize=1200%2C658&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-define-config-schema-json-photo-min.png?resize=673%2C389&ssl=1\" "\\"\\"")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/i0.wp.com\/css-tricks.com\/wp-content\/uploads\/2020\/08\/flatfile-portal-validators-regex-photo-min.png?resize=670%2C281&ssl=1\")
<\/figure>\n\n\nTo recap:<\/h3>\n\n\n