In a complex data warehousing environment, we create different placeholders to store data.
We can visualize multiple layers where we deal with data.
1. AS IS Layer
is the layer where we store the data which we get it from the source system with out transforming any of the data. This typically acts as the source which we consumed for the DW, even if there is a change happens in the operational system, we can cross consider this layer to reconcile the data between the staging and source.
2. Staging Layer
Typically staging layer contains the data which you are transforming, used to convert the heterogeneous data sources to homogeneous. For example, in some cased you get XML data from one hand held device, tables from an RDBMS, a CSV file from some other business. In this case we covert all three sources into common area. The common area typically gets created in an RDBMS so that handling of data is easy.
3. ODS Layer
Operational Data Store typically exists reduce the data latency. Typically when the requirements are coming from business to get the real time data integration, it becomes difficult to add the operational data to the summarized data in the Data warehouse. So, in this scenario the architecture of DW recommends to add the ODS layer database which typically contains the less granularity of data compared to DW data. Usually in the ODS we store records worth of 2 to 3 months.
4. EDW Layer
The data which is processed (Gold data) based on the business requirement, cleansed, ready to use for analysis gets stored in the EDW layer. In some implementations you see EDW in the form of 3rd Normal form, in some cases you see EDW in Star Schema's (Constellation).
5. Data Mart Layer (Physical Data Mart, Logical Data Mart)
Based on the architecture of the DW, this layer is treated differently from organization to organization. Typically in Teradata based implementations, the business views are formed and exposed to the BI tools (BO, MSTR and Cognos).
If the architecture of EDW follows the 3rd NF, then the data mart typically has the star schema which has dimensions and fact tables. If we have the data in the data mart layer, then we call it as physical data mart. If the data mart is made of views only which gets exposed to BI Layer then we call it as Logical data mart.
I hope you are getting an idea of how many different data layers can exists in the DW platform.
We can visualize multiple layers where we deal with data.
1. AS IS Layer
is the layer where we store the data which we get it from the source system with out transforming any of the data. This typically acts as the source which we consumed for the DW, even if there is a change happens in the operational system, we can cross consider this layer to reconcile the data between the staging and source.
2. Staging Layer
Typically staging layer contains the data which you are transforming, used to convert the heterogeneous data sources to homogeneous. For example, in some cased you get XML data from one hand held device, tables from an RDBMS, a CSV file from some other business. In this case we covert all three sources into common area. The common area typically gets created in an RDBMS so that handling of data is easy.
3. ODS Layer
Operational Data Store typically exists reduce the data latency. Typically when the requirements are coming from business to get the real time data integration, it becomes difficult to add the operational data to the summarized data in the Data warehouse. So, in this scenario the architecture of DW recommends to add the ODS layer database which typically contains the less granularity of data compared to DW data. Usually in the ODS we store records worth of 2 to 3 months.
4. EDW Layer
The data which is processed (Gold data) based on the business requirement, cleansed, ready to use for analysis gets stored in the EDW layer. In some implementations you see EDW in the form of 3rd Normal form, in some cases you see EDW in Star Schema's (Constellation).
5. Data Mart Layer (Physical Data Mart, Logical Data Mart)
Based on the architecture of the DW, this layer is treated differently from organization to organization. Typically in Teradata based implementations, the business views are formed and exposed to the BI tools (BO, MSTR and Cognos).
If the architecture of EDW follows the 3rd NF, then the data mart typically has the star schema which has dimensions and fact tables. If we have the data in the data mart layer, then we call it as physical data mart. If the data mart is made of views only which gets exposed to BI Layer then we call it as Logical data mart.
I hope you are getting an idea of how many different data layers can exists in the DW platform.
No comments:
Post a Comment