Changing business condiitions drive changes in business requirements (sometimes like wildfire, utterly unpredicatable) I have been through this situation numerous times, as I am sure you have been as well. So how to respond to these changing conditions? What if we have to break the ETL architecture that we have built?
As I was thinkging about this, a thought occured to me. May be we don’t have to be slaves to one set of standards but just be proactive in our approach. I hope the following table would be helpful in giving you a good insight or at the very least start a new thought process…
| Factor | Efficient ETL Architecture Requirement |
| Architecture strategy | Standardized Reports; emphasize high volume usage |
| Data Throughput cushion | Low; Generally data sizes are well known and do not change arbitrarily |
| Data Storage | Low; enable high inventory turns |
| Lead time | Shorten, but do not increase costs |
| Source System selection | Emphasize low volumes (deltas), consistent quality, on-time delivery |
| Demand | Predictable, Amount of data sourced does not change dramatically |
| Priorities | Low cost, consistent quality, on-time delivery |
| New Data Source introduction | Infrequent |
| Data variety | Low |
| Factor | Responsive ETL Architecture Requirement |
| Architecture strategy | Assemble-to-order, make-to-order, or customized service or products; emphasize variety |
| Data Throughput cushion | High |
| Data Storage | As needed to enable fast delivery time |
| Lead time | Shorten aggressively |
| Source System selection | Emphasize fast delivery time, customization, variety, volume flexibility, top quality |
| Demand | Unpredictable; Generally in the form of data dumps (of large sizes) |
| Priorities | Development speed, fast delivery times, customization, volume flexibility, variety, high reliability |
| New Data Source/s introduction | Frequent |
| Data variety | High |