Data Unit – Performance
As you can gather from this section, the Data Unit design is important. If any single Data Unit gets to 1 million intersections of data and the Data Unit grows, you will see an increasing impact on performance. The largest Data Units should never be above 2-3 million intersections.
Early systems (before OneStream) were designed for 5,000 records per Entity. The next generation started at 10,000 and could not consolidate with more than 100,000. The sweet spot for OneStream is between 250,000 and 500,000 records. As you increase above that number, the performance will deteriorate at an increasing rate. The processor speed becomes more critical at that point. The higher the frequency of the processors, the faster they can create these Data Units.
As you build the application, if you find Data Units this size, you should consider a couple of issues. Firstly, you should look to see if you can break up the Entity or Cube to spread out the data. Secondly, you should explain the impact on performance. You will have some idea if you are doing a prototype. If you have not, then I would consider building one. This is an issue that is easily managed if you have performance data; avoid the surprises! Set the expectation with the client.
I have had clients who think a two-minute Consolidation is exceptionally long, and others who are amazed we can do the Consolidation in less than a couple of hours. They have different expectations based on the data volumes and calculations and the systems they have used before. If you can explain that on each occasion you ‘give up some time, for some benefit’, you will help them understand how the entire process is improving, and hopefully they will be happy with whatever times the system generates.
One last note on Data Units as it relates to new features such as Dynamic Attributes. Now that you understand the dynamic nature of the Data Unit, it should be obvious that adding Members can create millions of aggregation points that can impact performance. Dynamic Attributes can create an explosion of intersections. If you are creating trillions of cells of calculations, expect this to slow reporting and Excel. Consider this impact and test changes in terms of impact as you move forward.
*
An excerpt from the OneStream Foundation Handbook
