![\"\"](\"http:\/\/bondel.hu\/wp-content\/uploads\/2020\/03\/data-consolidation.png\")
<\/figure>\n\n\n\nData consolidation best practices<\/h2>\n\n\n\n
Organizations should plan and execute data consolidation projects carefully. These best practices promote effective data consolidation:<\/p>\n\n\n\n
- Check to see whether data types in your source and target are compatible<\/strong>: If they\u2019re not, you\u2019ll have to transform data to address differences among data types.<\/li>
- Maintain copies of your data<\/strong>:\u00a0Data lineage\u00a0allows an organization to understand exactly what was done to the data \u2014 and how \u2014 during the consolidation process. You may need information to demonstrate regulatory compliance, or for retracing steps to understand the results of analytics and any business decisions based on them.<\/li>
- Standardize character set conversions<\/strong>: If you work with an application that allows you to store single-byte characters \u2014 such as Western languages \u2014 and double-byte characters \u2014 such as some Asian languages \u2014 in a database, the application can convert between these character types. However, when you move the data, the tools processing the data may be unaware that the data is stored in a different format. By standardizing character set conversions, you increase the likelihood of consolidating data for a reliable outcome.<\/li><\/ul>\n\n\n\n
Data consolidation challenges<\/h2>\n\n\n\n
There are challenges in the data consolidation process. The most common ones include:<\/p>\n\n\n\n
- Limited resources<\/strong>: Hand-coding consolidation techniques require data engineers who must write code and manage the process, and write more code every time a new data source comes online. The more sources and data types involved, the longer the process becomes.<\/li>
- Security issues<\/strong> (real and perceived): Security concerns include guarding data from breaches before and after consolidation, and developing backup and disaster recovery capabilities if data is compromised, corrupted, or deleted. Companies also must guard against \u201cinside jobs\u201d like exfiltration \u2014 the unauthorized copying, transfer, or retrieval of data from a computer or server.<\/li>
- Data spread across multiple locations:<\/strong> Today\u2019s decentralized data landscape can make data integration challenging. Having data in different locations \u2014 including in the cloud, on premises, and at remote locations \u2014 adds to the complexity of data consolidation. For instance, data stored in legacy systems may be missing times and dates for activities, which more modern systems commonly include; and data from external systems may not contain the same level of detail as internal sources.<\/li><\/ul>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
Data consolidation refers to the collection and integration of data from multiple sources into a single destination. During this process, different data sources are put together, or consolidated, into a single data store. Because data comes from a broad range of sources, consolidation allows organizations to more easily present data, while also facilitating effective data … <\/p>\n
- Security issues<\/strong> (real and perceived): Security concerns include guarding data from breaches before and after consolidation, and developing backup and disaster recovery capabilities if data is compromised, corrupted, or deleted. Companies also must guard against \u201cinside jobs\u201d like exfiltration \u2014 the unauthorized copying, transfer, or retrieval of data from a computer or server.<\/li>
- Maintain copies of your data<\/strong>:\u00a0Data lineage\u00a0allows an organization to understand exactly what was done to the data \u2014 and how \u2014 during the consolidation process. You may need information to demonstrate regulatory compliance, or for retracing steps to understand the results of analytics and any business decisions based on them.<\/li>