In business intelligence projects there are three different tasks:
- ETL. Extraction, transformation and loading of data. This process takes place in numerous steps of different areas (in most cases database), resulting in filled data warehouse.
- Reporting. Reporting and analysis of business relevant information, based on the data warehouse.
- Data entry. If not all relevant data is available in source databases, for instance for referential types, the data in the ETL process needs to be enriched. We therefore sometimes create simple data entry applications.
Stereotypes
We therefore distinguish smart use case types to support these three different tasks. All having a striking resemblance with regular software development. Therefore the standard types of smart use cases from regular projects also apply to business intelligence. Types identified for Business Intelligence include:
- Collect. Which can have similaritues with File import when a push strategy is implmented and flat file are populated into a staging area (ETL).
- Integrate. Integrates data from different sources into one definitions by using conversions, look-ups and defaults (ETL).
- Aggregate. Aggregates and denormelizes data as a general preperation for multiple data marts into a staging out layer. erent sources into one definitions by using conversions, look-ups and defaults (ETL).
- Calculate . Applies business specific rules and filtering to prepare infromation to be loaded into the dependend data marts (ETL).
- Present .
- Publish .
- Data Entry. Which should be developed using the traditional stereotypes.
Collect (5 SUCP)
Quite often flat files are produced by one of the source system, containing relevant input to the ETL process. File import describes loading such files into a database (for instance in a staging area). In a file import no direct communication between the two system takes place, so no additional actors are involved. Collect is explained in more detail using an activity diagram in
Collect.
Considerations
Take into account:
- Is the file present in the predestined location?
- Does the file actually contain data?
- Is the data valid, e.g. is all referential data correct, are mandatory fields filled in?
- On load, make sure records do not overlap matching records already present in the database.
====Estimation considerations====
To estimate the complexity of a file import, please consider the following: - How many types of records does the file hold?
- Does the project need to create the files? If so, model additional smart use case that produce the file. Use the producing system as primary actor. Do not model additional actors, as it is just a file export.
Integrate (10 SUCP)
Integrates different definitions into a single definition.
Considerations
Take into account that:
- Uniform. Check and implement integrity of offered data.
- Convert. Convert different definitions to singular format.
- Lookup. Validate data against lookups, and complete data to match these lookups (or conversively add new records to the lookups first).
- Delta. Perform a delta analysis. If a definition in the target area is missing, but is present in the source area, or vice versa, or the source area holds a updated definition, make sure the target area is updated accordingly. Add a (included) smart use case to the model, stereotype it as delta.
====Estimation considerations====
Please consider the following: - Data from how many tables gets pulled?
- How about your referential data?
A perfect example for integrate would be that of a large financial institute that had over 36 different definition of the term customer. That is, definition of customer existed in 36 different systems.
Sander Hoogendoorn
Aggregate (3 SUCP)
Considerations
Take into account:
- Is the file present in the predestined location?
- Does the file actually contain data?
- Is the data valid, e.g. is all referential data correct, are mandatory fields filled in?
- On load, make sure records do not overlap matching records already present in the database.
====Estimation considerations====
To estimate the complexity of a file import, please consider the following: - How many types of records does the file hold?
- Does the project need to create the files? If so, model additional smart use case that produce the file. Use the producing system as primary actor. Do not model additional actors, as it is just a file export.
Calculate (8 SUCP)
Quite often flat files are produced for data mining.
Considerations
Take into account:
- In general this type of smart use case does not apply very often as source system are fully operational, such as a bank's transaction system.
- Model both systems as actor on the use case diagram. The target system is the primary actor, the source system is the assisting actor.
====Estimation considerations====
Please consider the following: - Data from how many tables gets pulled?
- How about your referential data?